If AI does the junior work, who becomes the senior experimenter?

As Claude, ChatGPT, Gemini, and many more language-based AI models increase in both popularity and effectiveness, more CRO specialists can integrate AI into their daily workflows. In doing so, many eliminate the need for the “tedious” aspects of the job: writing test copy, documenting outcomes, reviewing user behaviors, and similarly quiet tasks.

Outsourcing the lower-stakes task to agents that can work on them in the background while keeping the human in the loop where needed is unquestionably efficient. A good plan in the short-term.

In the long-term, however, it raises an important question: what are the junior experimenters doing in a world where AI performs the “easy” work? 

Katie Kelly, an experimentation expert and former COO at Speero, raised this question during her recent appearance on Unite Voices:

{{quote}}
‍

‍

The data is already here

Several studies confirm what Kelly is thinking. Less than one year ago, Stanford University published a study indicating that “early-career workers (ages 22-25) in AI-exposed occupations experienced 16% relative employment declines” following widespread adoption of generative AI, while experienced workers in the same roles held comparatively steady. This is especially true in digital roles where AI is automating basic functions. 

A pipeline problem framed as a productivity win

When we look at the CRO specialist role holistically, the suggestion is that the role is moving “upwards,” which is to say more experienced, more strategic, and higher-level. Kelly agrees, saying that “the position for the agencies is now often more high-end, it’s, ‘how can we add the most value from a strategic point of view,’ more consultancy, that type of thing.”

The issue? Also in Kelly’s words, “juniors aren’t coming in able to offer strategic advice and consultancy.” 

New ideas have to come from somewhere, but eventually, everyone is going to leave the industry they’re in, whether for new challenges, different opportunities, or simply retiring. And then what?

There’s an uncomfortable idea in there that everyone is aware of but few seem to be discussing openly: most everyone was a junior at some point, and most everyone learned something crucial as a junior they carry with them to this day. The mistakes we make as new workers shape the strategies we compile as professionals, seniors, and leaders.

The creativity and diversity cost

Human creativity has always been the edge that sets some of the best workers apart. As AI becomes increasingly prevalent, that creativity is needed more than ever before. 

Junior specialists, young people, and workers from the “new” generation have always brought in that fresh eye to capture new angles that everyone else is used to overlooking. CRO, more than many other specialties, needs those new perspectives. As a field, it constantly evolves. Without new roles, however, it’s likely to stagnate.

{{quote1}}
‍

‍

Automation versus augmentation

The Stanford study highlighted an interesting distinction between companies that automate processes and companies that augment the capabilities of the workers carrying out those processes.

Notably, the study found that “entry-level employment has declined in applications of AI that automate work, but not those that augment it.”

On the contrary, the study found that occupations where AI use is primarily augmented see growing levels of employment.

The opportunity: how to augment the future of CRO

The junior CRO role should not simply vanish, replaced forever by AI so the only people entering the field come from marketing, product, or other specialties.

Instead, think of how AI can augment the learning of fresh minds eager to apply creative solutions to problems you might have thought were solved already.

• Redesign (or reinforce) the junior role around QA, research synthesis, and hypothesis quality. These tasks are less done by LLMs but can be enhanced by a clever user.
• Build deliberate apprenticeships, pairing juniors with seniors on live tests and making a point of teaching the “why”s of experimentation: why did this test win? Why didn’t this one? Why are we doing this, and what have we learned?
• Hire for soft skills and teach the hard ones. Curiosity and creativity go further than any teachable techniques or software. Focusing on those makes your team likelier to find talent no AI can replicate.
• Platform new voices by sharing work publicly, nominating team members for initiatives like the Who’s Who in Experimentation, and support the broader practice as much as you can.

AI is an opportunity; new testers are too

Teams that continue to hire, train, and support new CRO specialists are the ones that will own the senior talent everyone else wishes they had. 

{{quote3}}
‍

‍

Fresh eyes and fresh perspectives are the lifeblood of any competitive program, whether in marketing, sales, or CRO. 

Short-term efficiency is always tempting, but in the long-term, the potential problems with stagnation may be better off avoided. Continuing or improving your hiring practices may well be what keeps you ahead of everyone else who figured AI alone would be enough.

{{cta-block}}