Bad ideas can win: what that means for ego in CRO

Transcript

Meet Linda Bustos: The Mind Behind Ecom Ideas

Katie Green: Well, welcome to Unite Voices. I'm so excited to be joined today with Linda Bustos. I'm Katie Green, the principal advocate at Kameleoon. I'm responsible for maintaining our wonderful community of experimentation experts, and one of the people that is an expert in the digital field is Linda. Linda was nominated for our UXpert category of the ETLA, the Experimentation Thought Leadership Awards, and we were so happy to have you. So I want you to introduce yourself. Take a minute, tell everybody who you are. Thank you so much for taking the time to be on Unite Voices.

Linda Bustos: Well, thank you. Happy to be here. So yeah, I run a project called Ecom Ideas, which is a website that's totally free for anyone to use who's in the space. It's just a massive collection of innovative features and functionality inside e-commerce design patterns, all cataloged by type of page and type of feature experience, that kind of thing.

I've spent quite a long time covering the e-com, UX, and A/B testing space across various roles in different companies, and I'm just very, very passionate about e-commerce experience, web design, usability, and product. So yeah, happy to nerd out on that today.

Beyond Testing: How UX Expertise Shapes Ecom Ideas

Katie Green: And for people who don't know Ecom Ideas, it really, I feel like, doesn't come from a total background of testing, right? Is that fair to say? It comes from your background of UX and user behavior, right?

Linda Bustos: Yeah, I think where it falls is that everyone in the experimentation world knows that you can't just look at a site over there and go, "that's cool," plug it into your site, and call it a day. You need to have some rigor and discipline and critical thinking around everything. Even stuff that works on your own site might not work six months from now, once other elements enter the mix.

So I think my job is just to find and surface the ideas — it's up to you to decide whether it's a good idea in general. Some of the stuff is a little controversial, a little wacky, and that's where I kind of feel like I'm creating content for different personas. There's definitely the experimentation crowd, the people responsible for P&L, revenue, and performance. But there are also brand teams, and that's a completely different animal — they want to represent the brand digitally, they want the website to look like a fancy magazine or a video game in some cases.

So I try to just find the ideas and put them out there, no comment — it's up to you whether it's a good idea or a bad idea. I try not to post stinkers, but sometimes I think something's great and I have a little blooper reel. For example, I recently posted something where badges, when you're scrolling through the PLP, kind of slip behind the product card. It's hard to describe, but it totally looked like one of those edgy, design-forward effects you'll see on indie fashion sites that really push the boundaries of design. I posted it and said, "look at this theme effect," and people were like, "I'm pretty sure that's broken, I don't think that passed QA."

Sometimes you can't tell, because designers are really taking big risks, and it's always fun to find those in the wild — I love sharing that. So circling back to your question: is it just for CRO? I think CRO should be involved with any design decision, or at least weigh in with an opinion and have some kind of guardrails around things. But in terms of how we bring digital experiences to life, there's a lot of creativity out there.

Page Blindness and the Value of Fresh Eyes

Katie Green: Well, I think as an experimenter myself, one of the biggest things I've struggled with is staring at a page I've looked at for three years — and that's not an exaggeration, I've been working on the same exact landing page for three years, and I am out of ideas. It's just where you end up as a human. You look at the same thing and run out of ideas — I call it "page blindness," though I think multiple people probably use that term.

Because you have such robust experience in e-com, anybody dealing with that, who's been looking at the same PLP, PDP, or navigation — I did see a couple of your recent posts on navigation that were super interesting — they should look to your ideas. We struggle so much with ideation, and AI is becoming more involved in that too, but having someone like you who shares ideas and sees things out in the wild and says "this is why I think this is cool" is really invigorating and inspiring for experimenters.

I wanted to make sure we said that here at the top: this isn't technically a test-idea platform, but it can be, and that's how I'd expect our industries to collide. There's more than one way there, but experimenters should be looking to people like you and Ecom Ideas, because we all just think differently. We bring different perspectives to the table, and that's what makes CRO so exciting — we're able to come up with these different ideas. Your philosophy of "observation without prescription" is really important to bring into conversion rate optimization in general.

Great, so thank you so much for letting us dive into that for a quick second. I realize we spent more time than we probably intended to on just talking about what Ecom Ideas is, but I wanted to make sure people understood the context.

Why Mobile Navigation Still Struggles

Katie Green: If I can, I want to ask you one quick question before we get into it — I realize this wasn't part of our prep session, so this is just off the cuff. When you're coming up with ideas, do you ever find that all websites look the same to you? I saw your recent post on Bulgari's navigation, and another one on American Eagle's mini navigation. I have personally struggled with navigation tests so many times, so I'm curious — do you ever run into the issue that many experimenters do, where all e-com experiences just feel the same?

Linda Bustos: Yeah, in multiple ways. I think I could probably answer this in three different ways. I'll start with the navigation thing. In the post you're talking about with Bulgari, I said something I say often: desktop has more fun.

We know that the majority of e-commerce traffic is mobile, and mobile-first is a real and important thing. But once you get to a mobile surface, things like mega menus become a lot less sexy — they become more of a stack of links. And it's risky to start "menu-izing" things in there, like blocks of images, animations, or video. How do you handle your hierarchy and taxonomy? Is it an accordion, a slider, do you go three or four levels deep and back?

So mobile navigation is kind of vanilla. One thing that really surprises me is that a lot of brands just don't give it a visual hierarchy — it's the same font, the same color, everything is all the same. There's no featured category, no clear direction for where you want to send people.

Some companies do a little twist on that — they push actual product recommendations into the menu. If you're a small catalog with only a couple of hero products that 90% of people come for, that's a great strategy, but it doesn't work for everyone. You can also get banner blindness in menus if you put in two or three featured categories — it's just like the homepage, you're taking up real estate hoping to funnel people into a category, and who knows if that works. So there's a lot of testing opportunity for menus, but they do tend to look the same.

The Shopify Effect: Why So Many Sites Look Alike

Linda Bustos: And in terms of websites looking the same generally — especially since we're in a Shopify world — a lot of apparel sites just use a regular theme without adding much of their own. You get the theme, and it has the same standard: the PLPs look the same, the PDPs essentially look the same.

Homepages are where you see a bit more brand storytelling, more brand-specific content. But the rest of the site, and of course carts and checkouts, tend to look very much the same. So it's gotten harder for me, especially since I've been collecting content for Ecom Ideas for three years, to actually find things that are truly novel, truly different, or that truly solve a problem.

Websites start looking the same to me, but I take for granted that if you're not looking at 100 different websites every week, something I've seen a hundred times might actually be the first time someone else has seen it. So it's a little tricky for me to pick featured ideas, because sometimes something shocks me or I think it's really cool, but I know it's not an actual value-add to the user experience — it's just that a designer did something different in a very unusual and uncommon way. Maybe that's still worth sharing.

It's especially true in certain industries — themes tend to look the same. Go look at snacks, CPG, and beverage brands, and they get really creative with colors and theme style, but they all have a similar cartoony kind of vibe. You'd never see that on an apparel site.

AI, Vibe Coding, and the Design System Dilemma

Katie Green: No, this is great, and I'd like to talk a little more about the AI piece of it all, because you're kind of in the Shopify world — I realize the Shopify checkout experience is so homogenous, and they're saying it's the most streamlined experience for everybody. It's starting to get more personalized as you get more advanced. But I want to talk more about AI, because more and more people are using it to design, to vibe-code, whatever it is. Are you seeing any trends in UX as it aligns with the elevated abilities AI has on websites?

Linda Bustos: So, I am a confessed vibe coder. The new iteration of Ecom Ideas that I launched about a month and a half ago was purely vibe-coded — I built it in Lovable. One thing I learned about designing with AI, especially if you're building a product, is that it's not really a respecter of design systems.

It's not really consistent with UI elements as you're building out a new experience — it's a generative engine, and it can kind of invent things on the fly without keeping a record. Now, someone could probably educate me and say there's a proper way to kick off a project where you actually canonize the design system and enforce it everywhere. But I found myself going around and around in circles designing the product.

And once you get enough chat history, what AIs and LLMs do is compact it — they don't keep track of everything you've ever asked or told them, they compact it into a summary and hand that off to the next batch you talk to. Sometimes it drops and loses context, and that can be very frustrating. That's true for designing products, so if you're trying to vibe-code a Shopify theme, you might experience the same kind of pain.

Where AI Actually Unlocks CRO Work

Linda Bustos: When you're talking about using AI tools to do one specific thing for an experiment, though, I think that's a completely different unlock, because you're working on one small component. That can really empower someone in CRO, which is a discipline where you have to wear so many hats — you have to be good at heuristic evaluation, understand UX, understand data and statistics, understand the methodology of testing.

Oftentimes, in my own experience running A/B testing programs as a consultant, I really relied on support from a designer and a developer. Not every test I thought was a good idea could actually be implemented, because maybe the data wasn't layered that way inside Shopify, or the uplift wasn't a simple CSS show/hide situation. Sometimes we didn't even have a design system yet, especially with smaller companies — we needed to get a designer first. Developers didn't want to code something that wasn't going to last.

So I think AI can empower marketers and business folks — people who have an eye for what they want to test but don't have their own coding skills — to move quickly and create more variations. That's a really good unlock. But you have to make sure the AI is staying within a design system.

Katie Green: Sure. That's the tricky part, right? That's where the human in the loop is so important, to make sure the AI isn't running off doing things you don't want it to do. It's really helpful to hear your perspective, because a big part of my job is talking to people — customers, prospects, people who don't even work with us at all — and asking what's going on in the experimentation industry at large. And something a lot of people are hearing is "use AI," and they're asking, "okay, how?" They don't know where to begin, or whether they're wasting their time with it.

But I think it comes down to this: AI enables more people to have access to testing, design, and UX, but it still needs a person to implement that rigor. It comes back to your first point — you have to make good decisions. AI can show you ideas, but it can't make good decisions for you. It's a matter of training yourself to be able to do that.

Curiosity Over Ego: Building a Culture of Learning

Katie Green: But I think what's beautiful about the testing industry overall is that it tends to be low-ego — we're here because we have a genuine curiosity about what is or isn't going to work, that's what experimentation is. So I'd love to hear your perspective on people with UX ideas who are very passionate about them, where testing becomes more about proving a hypothesis than genuinely learning and failing forward. Do you have any advice for people dealing with a higher-ego approach to UX? How do we make sure UX and behavior online stay aligned with a culture of learning and continuous improvement?

Linda Bustos: I think you opened with an important point — people inside the CRO industry generally accept that you have to be curious, you have to let the data speak, and you never really know going in. No peeking, no trying to manipulate tests in your favor by not letting them reach the right confidence interval or stretching the timeline. Methodology and test design are important to that rigor.

Not everybody with access to an A/B testing tool has that background — anyone can set up an A/B test. If you're in a smaller or less-resourced company, you might be the marketer, or the VP of e-commerce, as I've been. You have a lot of ideas, but you need to actually sit down and talk with the team, because there's more to consider than just "I want to test this idea," especially if it needs to be developed and it's really a net-new feature.

Then it becomes a decision: is this going to be enough of a lift to warrant putting it into a development sprint, possibly knocking off something else important or urgent? Then you get a designer with a point of view, and this goes for analytics people, math heads, design heads, and development heads — everybody comes from their own background of expertise, and when you look at something through a different lens, everybody can be right.

Navigating Multiple Stakeholders and Trade-Offs

Linda Bustos: But then you're dealing with trade-offs — a development effort, or "we shouldn't do this because of this reason," whether that's accessibility, web performance, or brand stability. I remember one of my first A/B testing experiences: I worked with a company called Elastic Path, and we ran, in-house, the Vancouver 2010 Olympic store. So this was ages ago, but we had a real challenge with A/B testing because we had to answer to the Olympic committee, an international committee that was deeply committed to brand consistency. Brand was fixed in stone — it didn't matter what you wanted to try, it had to fit the international guidelines. So sometimes you can hit up against constraints like that.

Coming back to your question about ego and wanting certain things to win — it's a reality of digital product management and marketing that there are going to be multiple voices, and sometimes they're all right from where they sit. I think the biggest challenge for an A/B tester is sometimes letting a good idea go, or managing multiple expectations and maybe diluting the strength of a hypothesis in order to satisfy everybody. It's tricky.

Katie Green: It's a real — I feel like that's a question neither of us could fully answer in a podcast. I feel like I almost set you up for failure there, but I didn't mean to. It's a very complicated problem. As an experimenter, I've gone into a room where people with ten more years of experience than me are guaranteeing something's going to be right, and I'm like, okay, we can test it — but I'm telling you it's not going to work, because I have thousands of tests in my backlog that they don't have that context for. It really comes back to the fact that we all bring different perspectives.

How to Build a Stronger Case for New Ideas

Katie Green: I do want to back up the workflow a bit for the second-to-last question here. Taking a step back — when people come to the table for a roadmap discussion with ideas, what would you recommend they do to make sure their ideas aren't just flashy or adding noise? What guardrails can people put in place before they even get to "hey, CRO expert, I want to test this thing" — what can they do before they get there to make a really strong case, beyond "I feel like it's pretty"?

Linda Bustos: That's a tricky question to answer, because if you default to what every consultant says, it depends — everything kind of depends a little bit. But I think sometimes things that look like bad ideas actually win. I remember pushing back on a team that wanted to put a scrolling ticker at the top of a page — I find those incredibly annoying, and I never would have thought it would win. But it did win the test, having a scrolling ribbon across the top. Then we did another round of testing shoving emoji into it, and for that particular brand and customer, that was a second round of lift.

So if you've experienced that once or twice, where something you thought would never work actually did, it goes both ways — the things you think are really great can also fail when you test them. For someone facing that on a Monday morning, I think it's nice to knowledge-share and show practical examples, since that can sometimes open somebody's mind.

But the other thing, if you don't want to push back and you want to foster a "no bad ideas" culture, there are always ways to mitigate risk. You can test something on a PDP that gets medium traffic — not your best-converting page, not your hero product, but something with enough volume to get a test through without introducing too much risk. Or you can use a throttling method, sending a test to only one in every five visitors and protecting the rest. There are workarounds. I don't like to shoot down ideas — I like to keep an open mind. But for somebody who's bright-eyed and bushy-tailed on Monday morning with a handful of great ideas, you can also say something is deprioritized because you're busy with other things, or defer the decision to later.

Katie Green: That's my favorite method personally — "you know what, let's think about that tomorrow."

Linda Bustos: Yeah, exactly.

Advice for Data-Driven Teams New to UX

Katie Green: I think a lot of CRO people, like you said, there are all these different personas in CRO. For my very last question here — do you have any advice for somebody who might not be in the UX space within CRO, someone who's more of a math head, data head, or engineering head, who wants to integrate more UX thinking into their day-to-day practice? Say it's an analytics person who's focused on goals and wants to come to the roadmap meeting with more than just "we really need to increase click-through rate on this button" — what would be their first step in getting their feet wet with the kind of work you do every day?

Linda Bustos: I think usually when people are data-driven and start with the numbers, you want to protect the numbers. There's a KPI you've been told is important, and you're trying to figure out what that means and what you can do about it. Then you have to look at the features and functionality — not just look at a page and consider what you could do, but get a little more curious about what users are actually doing.

So you have the quantitative data you're seeing in your analytics reports, but can you also go look at something like Hotjar and actually watch people? That might flip your understanding of the metric. Then you might want to layer in a prototype, or get some user testers on a call to test your hypothesis there — a little more curiosity than jumping straight from a number to a diagnosed problem.

And then looking at things in context, because once you actually look at a page, you might start seeing the points of friction. If you're a pure numbers person, it really comes down to communication, curiosity, and engaging with your designers and developers. Everybody today kind of knows what usable and unusable feels like, just from our own use of the web — we know that constant animation is annoying, that page refreshes aren't as nice as a smooth transition, that struggling with a mobile menu is frustrating. We know what we don't like. So I think it's just communication and teamwork.

Katie Green: Communication and teamwork — that's what we're calling the episode. Period. No, that's great. I just want to make sure, because you have such a breadth of knowledge — you've been in leadership positions and had to work with people across the whole spectrum of personas when it comes to CRO and UX. Thank you so much for offering that advice, because I can imagine somebody listening thinking, "okay, I'm going to set up a one-on-one with my designer" — maybe someone they don't work with all the time. If we could inspire one person to have a conversation with somebody they might not have previously, and get a little more into one of the other personas that's critical for CRO and UX, that would be a huge win for us.

Where to Follow Linda and Ecom Ideas

Katie Green: But Linda, thank you so much for your time — I realize we're at the end of our episode here. If anybody wants more ideas from Linda, you can follow Ecom Ideas on LinkedIn. Linda, are there any other places people should follow — Substack, YouTube?

Linda Bustos: Right now it's LinkedIn and the website. There is a newsletter sign-up if you want to get in on that — I haven't published one in a while, but once that's revived, that'll be a third channel.

Katie Green: Maybe this will force you to! I tried to get on Substack myself and thought, whoa, this is really intimidating, I'm going to leave this for tomorrow — and I started that back in February. So, you know, I hear you.

No, that's great — everyone follow Ecom Ideas and Linda on LinkedIn for more thoughts on UX from our UXpert. It's great to hear from you. Thank you so much for your time, Linda.

Linda Bustos: Thanks, Katie.