What is group sequential testing?

Traditional experimentation and A/B testing worked with a simple rule: Don’t peek.

If you looked at results before an experiment reached its planned sample size, you risked inflating false positives and making the wrong call. So teams were told to wait, analyze once, and decide at the end.

Modern experimentation works differently. Mid-test analyses are common; even necessary. But both extremes carry cost. Waiting unnecessarily delays learning, slows growth, and increases opportunity cost. Stopping prematurely risks acting on noise, scaling false positives, and eroding trust in experimentation.

Group sequential testing sits between these two extremes, allowing data to be reviewed while an experiment is still running but without sacrificing statistical rigor. Often viewed as the simplest form of adaptive design, group sequential approaches enable principled early stopping while maintaining frequentist error control.

What is group sequential testing?

In group sequential testing, instead of analyzing experiment data only once at the end (as in a fixed-sample design) or continuously monitoring results after every observation (as in fully sequential testing), experiment data is analyzed at multiple pre-planned interim analyses as it accumulates.

The “group” in group sequential testing refers to the batch of observations collected between these interim analyses. This is staged evaluation built into the experiment design.

{{pink-block-1}}

Group sequential testing vs fully sequential testing

Fully sequential testing sits at the opposite end of the design spectrum. Rather than scheduling interim analyses for groups of observations, a running test statistic is monitored after every observation, and the experiment stops when it crosses a pre-specified boundary. 

In our healthcare example, this would mean updating the evidence continuously as each additional patient session occurs, rather than waiting for structured interim looks.

Importantly, this is not the same as informally checking a p-value and stopping when results appear favorable. In fully sequential testing, stopping boundaries are mathematically derived in advance to account for continuous monitoring. They are constructed so that the overall probability of a false positive across the entire sequential process remains within the target error rate.

In this design, the stopping rule is embedded directly into the experimental framework, and sample size is determined by the evolving evidence rather than fixed in advance. In theory, this means no predetermined maximum sample size is required, as the experiment concludes once sufficient evidence accumulates.

In practice, however, fully sequential designs can introduce operational complexity:

• Continuous monitoring requires infrastructure capable of near real-time updates. 
• Boundary calculations are more involved than in fixed-sample or group sequential designs.
• Interim estimates remain volatile until a stopping boundary is reached. Early engagement signals may fluctuate as different patient cohorts move through the redesigned coverage explanation.

Modern experimentation platforms generally handle this complexity behind the scenes. Solutions such as Kameleoon, for example, continuously compute the running test statistic, apply stopping boundaries, and surface signals as evidence approaches decision thresholds.

Here’s Kameleoon powering a fully sequential test:

With Kameleoon, you could even set up alerts to get notified as experiments move toward potential decision points.

While fully sequential testing enables continuous monitoring, many teams adopt group sequential testing to balance statistical efficiency with operational simplicity. This balance has direct implications for how experiments are planned.

How experiment planning changes with group sequential testing 

Designing a group sequential experiment requires more upfront specification than traditional fixed-sample testing, yet remains operationally simpler than fully sequential testing, which evaluates incoming data continuously. Because interim decision-making is built into the design, adaptability must be engineered into the experiment from the outset. Here is how the design work changes.

{{blue-block-1}}

Who sees interim results can matter more than it appears. Even directional signals can influence how teams interpret or support the redesign, subtly shaping the data that follows. Managing access to interim results can therefore serve as a safeguard for experimental validity.

Group sequential testing and the shift to adaptive experimentation

Group sequential testing sits naturally within the broader move toward adaptive experimentation. It introduces flexibility into experimentation without abandoning structure, allowing decisions to evolve as evidence accumulates rather than waiting for a single “final” analysis.

When implemented thoughtfully, this approach can shorten time to decision, reduce the amount of data required to reach confident conclusions, and increase trust in the outcomes organizations act upon. 

The deeper shift, however, is organizational. You move from running experiments as isolated events to designing controlled decision systems. Interim analyses are no longer informal check-ins but planned evaluation moments, and stopping is no longer reactive but built into the design itself.

{{cta-block}}