How Prompt-based Experimentation is making experimentation easier and faster
Prompt-based experimentation, or “vibe experimentation,” turns plain-English prompts into real, testable A/B test variants on your live site.
It combines the power of using AI to build digital products, including the code, with the practice of experimentation.
Unlike building prototypes or variants on vibe coding tools like Lovable and finding awkward ways to “test” those variants with focus groups or in staging environments, prompt-based experimentation allows you to describe the change you want to see directly on your live site.
With prompt-based experimentation, you can create a new web feature or revising an existing one, and watch it be built in real time on your site. Once built, you can test the new digital experience on existing traffic using your existing tech stack.
Since Kameleoon launched Prompt-based Experimentation (PBX) in June 2025, we’ve seen thousands of PBX experiments run, helping teams learn more about their customers while providing meaningful wins faster than they ever did before. .
In this article, we share a clear read on the immediate impact of Kameleoon PBX on businesses today.
Headline numbers: the first 1,000 vibe experiments
After beta users ran the first 1,000 experiments in PBX, we collated data and found three numbers that stood out:
- 97% reduction in time to create testable variants
- 10x increase in test velocity
- Less than three months for payback
These are significant metrics; when build time drops by so much, your whole roadmap changes. You can afford to test structure, flow, and microcopy, rather than just choosing one over the others. You can run follow-up experiments while the original insights are still fresh.
Your engineers can focus on the hard problems, while PBX allows non-technical teams to cover fast, repeatable work.
What PBX builds well
After its release, an independent reconstruction study examined how PBX performs against traditional experimentation by asking it to rebuild historical tests. The pattern is consistent: PBX delivers notable time savings and high visual fidelity for content and layout edits, and does well on interactive patterns from specific prompts.
“Deploy AI-assisted experimentation for routine test scenarios immediately,” the authors of the study recommend. “The 89% time savings and 93.6% success rate for static content modifications represent clear efficiency gains with acceptable risk profiles.”
From this study, we can think about scope in three tiers:
- Start with atomic changes. Headers, CTAs, banners, cards, tables, and forms.
- Add well-specified, light interactions. Sticky bars, tooltips, sort orders, pagination, and simple modals.
- Use a short QA pass for multi-step logic. Edge cases and conditional states are easy to handle with an extra prompt.
This all means that the goal is not to replace product sense or QA, but rather to let PBX build the first draft and free all teams to focus on other important work.
What worked best: repeatable patterns
Since August 2025, over 100 unique companies have built experiments with PBX, with teams averaging roughly three prompts per variant. Which prompts worked best?
- Layout beats copy. Structural changes move metrics more readily than small text edits. Try tightening hero sections, simplifying product cards, and improving scannability.
- Be specific. Name the element, the page area, and the intent of each change.
- Make your prompts atomic. The prompts that generated high-quality outputs focused on one specific change, which is why the average variant uses three.
Practically, plan for two to four prompt iterations, especially for complex UX changes, and remember that multi-step logic changes need human QA.
How PBX has already transformed existing sites
A retail team used PBX to turn long mobile product pages into decision-ready screens. They used PBX to add a sticky “Add to Cart” bar, lift reviews higher, and simplify size selection. The whole process took only minutes; after a quick QA pass, the team launched the variant and saw more taps on the primary CTA and a smoother path to checkout.
How to get your first PBX wins in the first week of your trial
You can try PBX for free before choosing the best plan for you. Once you do, here’s a simple guide to follow to get started.
- Pick three tests you want to run.
- Write one clear prompt per change.
- Iterate twice; preview your variant and tweak it once or twice until it matches your intent.
- Check with your developer, then launch and learn. You can use sequential testing in free and starter trials to limit risk of peeking.
- Keep a backlog and archive the prompts that work best.
PBX, and vibe experimentation as a whole, is not a replacement for craft, but it is an accelerator. The stats we’ve seen show real gains in speed, scale, and payback for both technical and non-technical teams.
These stories show repeatable wins your team can run this week, for more insights in hours. Sign up for the free trial; your next round of insights can be only a few hours away.
