Primary metrics and their BFFs: how to build a test story that holds up

Many experimentation programs default to conversion rate as the primary metric on every test. It’s a habit that feels safe. 

It’s also why so many test results fail to tell a story leadership can act on.

Marcela Gutierrez leads digital analytics, experimentation, and consumer behavior at Fossil Group. Her team runs experiments across Fossil’s global portfolio. Her team has built a measurement discipline that produces results that are easier to trust.

The frame they use is simple: pick the metric closest to the change, then give that metric a “best friend” that corroborates the story.

In this article, we’ll walk through how that works, why session-based measurement can distort results, how to pick a primary metric that fits the test, and what a "BFF metric" looks like.

The move to user-level measurement

According to Gutierrez, the first move in maturing a measurement practice is leaving session-based metrics behind.

{{quote}}
‍

‍

A test that succeeds often means more return visits. That means if you only count sessions, the same user shows up multiple times in your results, dampening or masking the test’s true lift. 

Switching to user-level measurement means each visitor counts once, and your math better reflects what actually happened. 

{{blue-block-1}}

Pick the metric closest to the change

Conversion rate is not the right primary metric for every test. The primary metric should always be the one your change most directly influences.

{{quote1}}
‍

‍

If, for example, you test a filter widget and measure conversion as the primary KPI, you are asking the test to carry every other variable downstream of that filter. Which metric will the change actually move? That’s what will give you a clean read on whether the change worked. 

Then, use the next metric down the funnel to confirm the change had a real consumer effect, not just a click. This is what Gutierrez calls the “BFF.”

Every primary metric needs a secondary

A single metric never tells the whole story. The secondary metric is what proves the primary metric meant something.

{{quote3}}

‍

The BFF model produces a small set of metrics that build a narrative, as opposed to “just” an observation. The primary metric tells you the change worked, while the secondary tells you the change mattered. 

{{blue-block-2}}

Set the hypothesis before the test

Marcela's team has one non-negotiable rule: always set the hypothesis before the test begins.

Without a stated hypothesis, you can rationalize almost any result. With one, you have a clear standard for what a win looks like, what a loss looks like, and which metrics earn the primary slot. 

The team that writes "we believe X will move Y, because Z" has already done the hardest thinking. 

Learn more about A/B testing vs hypothesis testing to explore this underlying logic.

{{cta-block1}}

Pivot when you know more

It’s always important to remember that measurement discipline is not static. The frameworks that work for your program today may not work next year. That’s not a bad thing for a team with a culture of learning and experimentation-led growth:

{{quote4}}
‍

‍

The teams that hold this disposition produce better measurements over time. Tests that utilize primary metrics, “BFF” metrics to corroborate them, and hypotheses stated before the test begins hold up under scrutiny. 

Once you’ve built a few of those, the program can start speaking for itself.

{{cta-block}}