How to move from
client-side to server-side A/B testing
- Typical evolution of most optimization programs
- Two approaches same goal: make better decisions using data
- Three main reasons why A/B testing programs move from client-side web testing to server-side testing
- Before you move to server-side testing
- 9 ways to know if you should run a server or client-side test
- 6 steps to move to server-side testing
As experimentation programs mature, they build increasingly more sophisticated tests.
Building more sophisticated tests requires more agile testing expertise, dev support, or new technologies.
At some point in your optimization journey, you will (hopefully) make the leap from pure client-side to server-side experimentation. When you do it and how you do it depends on your organization and priorities.
This guide and checklist are meant to help experimentation programs learn when, if, and how they should adopt server-side testing, in addition to running A/B tests client-side.
Typical evolution of most optimization programs
Most programs start with a graphic or front-end code-based approach. Mature ones leverage both. Advanced programs and/or teams just beginning with server-side testing may rely on hybrid experimentation. While dedicated programs may evolve to also practice server-side testing, keep in mind that the real goal is to know which method is right for the intended test and have the means to execute well.
Two approaches same goal: make better decisions using data
The main benefit of client-side experimentation or web experimentation is that it is “easy” on the front-end for anyone to build, launch, and run A/B tests.
Experimentation on the backend, however, is a different story.
While it is more powerful, server-side A/B testing requires backend developer resources. As most companies know, getting dev support can be a huge challenge.
This friction has made server-side A/B testing, which can give organizations several significant advantages, a more challenging practice than standard web, or client-side, optimization for most teams.
It’s important to note that client and server-side A/B testing are NOT exclusive. They are complementary. Brands do not need to choose between one and the other. They should be able to decide which approach is best for their optimization goals and strategies.
This guide is meant to help programs understand the pros and cons of each approach and how to adopt server-side testing.
Three main reasons why A/B testing programs move from client-side web testing to server-side testing
Server-side testing is not just for engineering-led experimentation programs. Programs choose or evolve to practice server-side testing because it overcomes key client-side limitations. By enlisting developers (see the checklist below), server-side testing can help DevOps and IT teams get behind testing, as they are able to see the testing process in action.
1. Server-side A/B testing enables feature experimentation. For those companies that rely on products to also spur growth (product-led growth (PLG)), server-side testing enables organizations to test and optimize product and feature variations.
2. Server-side A/B testing allows teams to build experiments that have a high degree of complexity and/or that test:
- Dynamic content, especially relying on external data sources
- Omnichannel experiences
- Engineering, network, and database performance
3. Server-side A/B testing also helps experimenters better manage:
- Web browser restrictions such as Apple’s Intelligent Tracking Prevention (ITP)
- Performance and/or security issues caused by poorly built or installed client-side A/B test management tools.
Before you move to server-side testing
What is client-side experimentation?
In a client-side test, your A/B testing script modifies web pages directly in the visitor's browser (whether Chrome, Firefox, Safari, Opera, etc.). Client-side web testing is also called web experimentation because the browser must be involved, i.e., it is only for websites. What is usually called client-side testing is a test built with a graphic editor or with JS/CSS code.
To do client-side A/B testing:
- An experimenter must install a snippet of code from the tool into their website HTML to set up a client-side testing solution
- Use a graphic editor to build tests
- Or use a JS/CSS code editor to build tests
Benefits of client-side A/B testing:
- Client-side testing allows anybody to create and deploy tests and personalization campaigns quickly (for example, changing text and CTA button placement, or switching blocks around).
- Brands can also run more complex experiments like testing several elements simultaneously (a multivariate test), personalizing the visitor experience across multiple pages, or using targeting criteria based on real-time, in-session browsing data.
- As the A/B test is only adjusting the website front-end i.e., UX and graphical elements, it’s easier to hire professional freelancers and CRO agencies to ideate and build tests. These external resources do not need access to the backend code.
- Programs are able to more easily build tests based on browser data e.g., pages viewed, time spent, new vs. returning visitors, etc.
Client-side testing has its disadvantages:
- Client-side web code testing cannot optimize mobile applications, or anything else dependent on the backend.
- If the snippet is poorly constructed or installed, visitors can experience a “flicker” effect where the control loads before the variation.
- While the initial page load impact of a client-side web A/B testing script is negligible, the more experiments you launch, the “bigger” the test script will get. If not well designed, the weight of the script can add up and slow your web page load performance.
- Data privacy measures like Apple’s Intelligent Tracking Prevention (ITP) limit what data experimentation programs and agencies can rely on since they restrict web visitor data access. (Note that ITP challenges can be mitigated by the solution.)
- The graphic editor (WYSIWYG) from poorly built client-side web solutions may struggle to function properly on a single-page application (SPA), which many sites are moving toward.
What is server-side A/B testing?
In server-side A/B testing, test variations are rendered directly on the server, not on the visitor's browser, like in client-side testing. Changes are directly generated before the HTML pages load.
To create server-side A/B tests:
- Unless they build an in-house experimentation solution from scratch, they must rely on an experimentation solution’s software development kits (SDKs).
- These kits translate a tool’s processes and capabilities into whichever backend language the developer will test on. An outdated or incomplete SDK will severely limit what a team can do, which is why it’s so important to work with a vendor with powerful and up-to-date SDKs.
Benefits of server-side testing
- Teams that practice server-side testing methods have absolute freedom to test whatever they wish to optimize.
- Server-side testing eliminates the flicker effect.
- There is no impact by an external snippet affecting page load performance.
- As more browsers restrict website visitor data (see Apple’s ITP), adopting server-side testing can act as a hedge against possible client-side complications.
- Expands your ability to conduct mobile tests.
The downside of server-side testing
- It requires programmers to code the experiments and establish a test environment
- Your program will only be as strong as the SDKs provided by your experimentation solution.
- Testers won’t be able to as easily rely on an external agency or service provider to help you build tests, since server-side testing requires a developer who understands your site’s architecture and engineering.
- It highly limits what you can do in terms of targeting, as it’s difficult to access and use visitor in-session i.e., browser data to build A/B tests.
Companies that can leverage full-stack testing are different
James McCormick, VP of Marketing at Creativex, shares with Kameleoon what practices and qualities leading companies that practice full-stack experimentation possess.
It’s not organic. There’s a plan to get it right.
Leading companies that practice server-side testing are mature and purposeful. They know how valuable experimentation is and they want to scale it. They understand the importance of data but also grasp how critical it is to have empathy for the human engaging in the digital experiences the company is creating.
I like to use the acronym, “DART” when describing these types of leading companies.
They know how to capture and extract value from data. In addition to understanding data, they can activate it by building experiments, enriching their customer profile, sending personalized messages, etc. Leading companies can do this in real-time, so the experience feels fresh and relevant. Lastly, they know how important trust is, so they’re transparent about how data is used. They’re proactive with consent management and protecting customer data.
Management is engaged, but practitioners know how to sell their value
Becoming data-driven and scaling experimentation always requires a top-down approach but that doesn’t mean senior executives are data experts. Executives know that a customer is more than a click, a visit, or an uplift. When experimentation leaders from leading companies engage executives, they present performance data but also include in-session excerpts from real customers that demonstrate how an experience is failing or winning. They present the real person, and that resonates with management, who in turn keeps investing in optimization.
It’s the scale of server-side testing that appeals
Digital-first companies know that if they’re going to create an optimized customer experience, they’re going to have to analyze and test everywhere they can, not just on their website. Full-stack testing gives them the ability to achieve testing at scale. Powered by human-centric analytics that reveals the entire customer journey across multiple touchpoints, they’re able to build, release and test features on every channel, while carefully controlling performance and security.
3 ways hybrid experimentation makes server-side testing easier
Many experimentation programs are beginning to run “hybrid experiments” which combine the benefits of client-side with server-side testing.
A hybrid experiment is a server-side test that benefits from key client-side capabilities. While developers are required to build the test, they are not needed to code tracking, pull in external segments for targeting purposes, or to configure integrations to third-party testing tools.
Hybrid experimentation saves developers’ time. A backend developer only has to engineer the logic and function of an experiment that is part of your test plan.
For example, the back-end developer would code when and how a pricing page would change based on visitor characteristics, whereas a front-end developer or product manager would design the look and language of the pricing options.
With hybrid experimentation, non-technical teams can still make front-end changes (such as designing CTAs, colors, or wording) using client-side technology, all within the same experiment. You’ll be able to run concurrent tests if you wish.
Developers don’t need to set up tracking or build complex integrations with the tools you use every day. While developers build the experiment code server-side, experimentation leaders, marketers and/or product managers can set up KPIs for each test using client-side technology or use pre-built integrations to target users based on data available in their preferred tools (Contentsquare, Mixpanel, Segment, Amplitude, etc.).
Anyone can iterate, analyze, and activate data derived from the test. With hybrid experimentation, once developers configure the test, anyone can iterate on and analyze it without needing developer involvement. Marketers and/or product managers can activate data derived from the server-side test.
Not all A/B testing platforms can provide hybrid experimentation. Only when the underlying architecture to create client and server-side tests are the same, like Kameleoon’s, can an A/B testing solution provide hybrid experimentation.
Unfortunately, most software testing tools have one testing platform to create client-side tests and another for server-side. This means programs must consider two different user interfaces, data models, and more. It’s not recommended.
Learn what is a hybrid experiment.
9 ways to know if you should run a server or client-side test
Remember, server-side testing doesn’t replace client-side testing. They’re complementary, and good test managers will know which methodology makes sense for a given experiment. Smart programs know when they should build a client or server-side test.
To help you determine when you should build a server-side test, consider these use cases:
1. You want to run “big” front-end tests
Server-side testing is not limited to optimizing hidden digital experiences i.e., algorithms. You can run complex tests on the front-end with server-side testing. For instance, if you’re changing many UX elements on a page and/or how visitors navigate your site, your test is likely to be quite “heavy.” Heavy client-side tests may have performance and quality assurance (QA) issues that may cause a bad user experience.
Server-side testing not only broadens what you can test, but they’re more scalable, which is helpful when you’re making complex client-side tests.
2. Your customers interact with you via multiple channels
Customers may use different devices and channels to visit your site or mobile app. Server-side testing enables you to roll out the same experiments across every channel without having to design them individually for each platform.
With server-side testing, you’ll be able to more easily measure how customers responded to a test on your website, but also how they reacted to a similar experience when delivered through other channels, such as mobile apps and email.
3. Experimenting with the cost/price of your goods and services is important
Since server-side tests render variants directly from the back-end server, it also allows you to test more complex, dynamic content beyond the scope of the user interface of a “static” website. This could include pricing and shipping costs on an e-commerce site generated on the server via a database rather than on the front-end.
4. You believe contextual and/or algorithmic targeting creates better customer experiences
You may wish that your search engine produces results that prioritize products that you know your customer has purchased, “liked” or viewed in the past. Spotify does something like this when it prioritizes your “Liked music” in the search results.
Creating complex “rules” that determine what digital experiences, services, and products your visitors receive often requires server-side testing because of the number of conditions, including external ones, that affect the test.
These sorts of test scenarios and personalized experiences are the hallmark of mature and advanced optimization teams, which means you likely see them at fast-growth and/or leading companies.
5. Product and content recommendations affect what experiences you want to deliver
Like tests that aim to optimize how pricing or algorithms work, teams may use server-side testing to test how product and content recommendations affect customer behavior. While this is also possible via client-side testing, the recommendations are often based on visitor browser data. Server-side testing of recommendations may rely on multiple data sources.
Did you know? A/B testing software like Kameleoon makes it easy for non-technical teams to pull in “custom data” for targeting purposes.
6. You’re a product manager and want to optimize the products you manage
As digital products are coded in the back-end, product managers rely on server-side testing to optimize their products and features. In some cases, they’ll build a server-side test to optimize a particular product or they may create a feature flag and build a feature experiment.
Client-side testing isn’t as ideal for product-led growth as server-side testing.
7. You’re an engineer and want to test a technical capability
Back-end and full-stack developers rely on server-side testing to validate a product, app, or site’s load speed, stability, and data accuracy. Client-side testing cannot do this.
8. Server-side testing eliminates flicker and the performance issues that can affect client-side testing
With server-side testing, the flicker effect is eliminated, and digital experiences load as quickly as the server call is set up.
Server-side testing also makes it easier to release a winning variation faster, as it is already coded into your back-end language.
9. You are confident enough in an idea that you’re willing to use dev resources to test it
As a rule of thumb, client-side tests are faster and cheaper to test but slow when it comes to implementing a “winner.” Server-side tests, however, can be slower and more expensive to execute but quick to put winners into practice.
If you want something quick and directional to help you understand which direction to build your business, a client-side test may be right. If you have confidence in an idea and are willing to spend development effort to test it, then going server-side may be the right choice.
6 steps to move to server-side testing
Follow this checklist to gain the buy-in and resources you’ll need to ensure your test strategy can generate ROI from server-side testing quickly and reliably.
Gain access to 28 tips on how to move to server-side testing. Access the full checklist.
1. Understand the backend languages of your websites and mobile apps
Software development kits (SDKs) are critical for server-side testing because they provide developers with all the components they need to quickly take your experiments to all channels and platforms.
2. Know how developers release code
Executing this step is critical to setting up a positive and informed conversation with development leadership, which is next. Just like you do your research before proposing a test, you need to do your research about how your development process works.
3. Meet with the development team. Position server-side testing to address their needs
Developers value rolling out and back features and products with confidence and ease. They like stability and efficiency. They may not value experimentation, so it’s important to speak to their needs and jobs to be done vs. your own.
4. Get buy-in from senior leadership
You will need back-end developer support and a server-side solution to practice server-side testing. Allocating these resources often requires getting approval from senior management.
5. Find the right solution that fits your needs
Now that you have the initial buy-in of senior leadership and developer resources, you’ll need a server-side testing solution.
Don't pick test software that will force your team to cobble together different products and scramble to scale its experimentation capabilities to fit your test plans. Pick a software solution that offers both client and server-side testing. Pick one that has dev-friendly features, in addition to the SDKs for your back-end.
Bonus: Look for a vendor that can offer hybrid experimentation, which makes server-side testing easier.
6. Crawl, walk, run
The only difference between client and server-side testing is the process of building a test. In the beginning, continue to use the same testing methodologies to ideate, prioritize, assign goals, and analyze your server-side tests as your client-side ones. Apply most of your scarce time to supporting the developers building your test.
Server-side testing is not better or worse than client-side testing.
Server-side testing is another means to learn about your customers and optimize and personalize your digital products. It broadens what you can test and comes with performance benefits. It also requires heavy developer involvement. If your optimization program feels server-side testing can help you learn more about your customers, then consider starting out with hybrid experimentation. It makes server-side testing easier.
Server-side tests are rendered on your infrastructure before any code is sent to your users’ browsers or before they download your app. You can design a server-side test either from scratch with the help of your own development team or by using an experimentation solution’s SDK. Client-side tests, on the other hand, insert a snippet of code sent to a user’s browser that controls how your site loads.
You can test practically anything you want about your site or app when you design tests server-side, as everything about your code and UX is in your control. And since every change needed to conduct experiments processes on your infrastructure there is minimal performance impact on what the users see.
The most commonly used technique is to configure a feature flag. These are pieces of code that determine which of two or more features a user sees when they load your site. Feature flags—sometimes called feature toggles or release toggles—allow you to include both A and B versions of your site in a single codebase. A developer can hardcode a feature flag or you can design flagged tests using an experimentation solution.
The two most common methodologies used in server-side testing are A/B testing and multivariate testing. A/B tests are a popular choice when you want to test across multiple channels, for example, web and mobile, at the same time. Multivariate tests are also popular server-side as it is easier to manage their increased complexity when processed on infrastructure you fully control. Therefore, you don’t have to worry about how users’ devices will interact with what you send them.
Consider the size and skill levels of your experimentation team members. Do you have front-end development experience available to you? Back-end experience? Also, consider the volume of tests and traffic you’ll need to work with. The greater the scope and complexity of your tests and traffic, the more likely you are to benefit from using a fully-featured testing solution like Kameleoon Hybrid.