Why product teams need experimentation powered by AI prompting
Product teams are always asked to ship faster with the same teams. As companies grow, demand increases, backlogs fill, and experimentation becomes de-prioritized. Then features start going live without much evidence that anyone is interested.
A Pendo study from 2024 confirms it: for the average product, 6% of features generate 80% of clicks.
Prompt-based experimentation (PBX) gives product teams a way to test ideas on real users without contributing to the engineering backlog. Simply describe the change in conversational language, and PBX creates an on-brand prototype ready to be tested with all users on your live site.
In this article, we’ll discuss how PBX fits your product loop and improves workflows for you and your engineering teams.
PBX: an overview
For product teams, PBX acts as both an experimentation platform and a Figma Make. It allows you to turn an idea into a prototype that you can run actual experiments with with users on your live site.
It works in a simple chat interface. You tell it what you want. It:
- Uses your prompt to create a new variation that inherits your design and components.
- Sets up targeting, traffic split, and metrics inside Kameleoon.
- Launches the experiment (ideally after a quick QA from dev)
- Summarizes results and suggests next steps.
Growth leaders will often argue that every product feature has to be developed and tested, with the losing features dropped.
But product and engineering teams don’t want to throw away thousands of dollars in working hours building something that may not bring in value, if it gets deployed at all.
PBX is designed to reduce this friction, allowing engineering teams to focus on their own work up until you know what actually works.
“PBX is designed to change how product teams used to work. By letting them build and test most of their backlog of ideas, they can then bring the winning ideas to the table and involve engineers that understand and know why they are building the next features.”
—Fred De Todaro
Chief Product Officer, Kameleoon
How PBX fits your product loop
A typical product loop looks like this:
Discover > Decide > Build > Release > Observe
With PBX, however, you can instead approach it like this:
Discover > Experiment > Decide > Build > Release
PBX focuses primarily on the decide stages, allowing you to test variations on your live site and collect user data. Once you have a clear winner, you move on to building and releasing your features, using feature flags to control the rollout.
PBX is built on the same statistics engine as Kameleoon’s broader experimentation platform, allowing you to use sequential, Frequentist, Bayesian, and CUPED to keep test data secure and statistically significant.
A quick example: signup flow
Imagine you own a startup and want to improve trial starts.
Your typical process may be to write a ticket, submit to design, wait for the next available sprint, and hope the new flow works once it reaches production.
With PBX, you can explore a lighter version in a single working session:
- Open PBX on your signup page.
- Describe the change you want to explore. For example: “Shorten the signup form to only use email and password fields; move the other, optional fields to a second step after the user presses submit. Keep styling and validation consistent.”
- Review the output, using chat to further tweak details as needed.
- Perform QA on key devices.
- Target new visitors, split traffic 50/50, and set your primary metric to trial starts.
- Launch the experiment and watch progress. Use sequential testing to inspect trends without breaking the math.
- Once you have enough data, generate a summary and decide what to do from there.
If the lighter flow works, you now have a clear, data-backed request for your developers. If it doesn’t, you learn a valuable lesson at a low cost.
Get started with PBX this quarter
You don’t need a big rollout to get value from PBX. You can even run your first month for free using our Starter Plan! Just click here to get started.
