About me

Over the past 15 years, Erin has designed and developed over 1,500+ experiments for brands such as Booking.com and Deliveroo. Erin wrote the book “Design for Impact: Your Guide to Designing Effective Product Experiments,” which was published by Rosenfeld Media in June 2024. The book aims to help readers with complex concepts such as Sample Ratio Mismatches (SRMs), null hypothesis tests, and p-values.

Q&A

What’s one thing people often get wrong when trying to get different teams to run experiments together?

“I see a lot of product teams take an approach to testing that’s too academic out of the gate.

‍
When data science teams own the tools and processes, they sometimes unintentionally discourage people from testing with big words, fear of doing it wrong, and over-complicated tooling.

‍
Data science shouldn’t gate-keep. They should enable and educate without judgment or hurdles. People learn best by doing—and by making mistakes. Data science should make room for people to make mistakes.

‍
So allow experimenters to access tools, numbers, and knowledge, and guide them as they go. It’s better to have people enthusiastically start and mess up along the way than to begin with fear.

‍
Fear is harder to overcome than process problems or miscalculations.”