How to test an e-commerce website using prompt-based experimentation

Most e-commerce funnels lose prospective customers in the same two places: the buy box on product detail pages and the checkout.

So it’s no surprise that a lot of e-commerce leads, marketers, and developers run experiments on these two areas, aiming to collect customer insights, reduce friction, and make it as easy as possible for buyers to buy.

In this article, we’ll explore how e-commerce teams have used Prompt-based Experimentation to generate test-ready variants that improve PDP-to-checkout drop-off on their sites; what prompts they used, what metrics they tested, and the improvements they saw from each test.

Three e-commerce experiments you can launch today with PBX

Prompt-based Experimentation (PBX) uses AI to allow teams to ideate, build, and launch experiments using natural language.

The three experiments listed below were all designed and launched through PBX using the prompts listed in each section. Each one was used by several different e-commerce and retail companies to improve their conversion funnel. 

Experiment one will help you test your product detail pages and discover potential uplift on add-to-cart. From there, you can launch experiment two, a checkout CTA test, on the same segment to fill in the rest of the story.

We’re also sharing an optional third experiment that tests an improvement to the mobile-specific flow.

Experiment one: making “add to cart” the only obvious next step

In our first use case, testers hypothesize that a PDP buy box that offers competing actions invites shoppers to pause. Without removing options, they wanted to make the next step as clear as possible. 

{{blue-block-1}}

In 25 instances of this experiment run through PBX, we see an average lift of 12.43% on e-commerce and retail websites.

{{pink-block-1}}

Experiment two: make “continue” or “place order” the only prominent action

On checkout screens, distraction leads to abandonment. Testers theorized that if the primary CTA competed with extra links like “continue shopping,” shoppers would lose focus. They wanted to simplify the hierarchy to improve checkout.

{{blue-block-2}}

In 16 instances of this experiment run through PBX, we see an average lift of 7.53% on e-commerce and retail websites.

{{pink-block-2}}

[Optional] Experiment three: add a sticky checkout bar on mobile

When mobile users scroll to review totals and delivery details, they often lose the CTA. Testers used PBX to implement a sticky bar to boost its visibility.

Start segment: Visitors to checkout on mobile devices

Primary KPI: Completion rate/Purchase conversion rate

Guardrail metrics: Error rate, time to complete checkout

PBX prompt:

Hypothesis: Persistent access to the primary action reduces drop-off on mobile.
Task: Keep the checkout action visible as users scroll.
Target: Mobile checkout page.
Change:
- Add a sticky action bar with the primary CTA and a minimal summary (total price).
- Show it after the user scrolls past the default CTA position.
Constraints:
- Do not cover fields, error messages, or system UI.
- Keep it responsive across common devices.
- Allow dismiss if it blocks content.

In 33 instances of this experiment run through PBX, we see an average lift of 2.16% on e-commerce and retail websites.

What to do with the winners

After launching each test, monitor your conversion rates and, if purchase conversion rates improve and guardrails stay flat (or improve), use feature flags to ship your changes. Don’t forget to wait until you have statistically-significant results! 

On the other hand, if your KPIs improve, but error or time-to-complete rates rise, iterate on your tests to find the friction points before committing to your variation.

If you see no meaningful movement or your guardrails worsen, that’s your cue to avoid shipping and investigate new theories.

No matter what happens, be sure to keep your PBX prompt as documentation and record the results of each test. 

{{cta-block}}