What is the flicker effect and how can you get rid of it in your A/B testing?

March 22, 2017

Reading time:

6 minutes

Fred De Todaro

Fred is Kameleoon's Chief Product Officer and leads the company's A/B testing, feature management, and personalization product strategy. Leading product teams across Europe and North America, he regularly shares his advice on product trends in experimentation and how best to deploy Kameleoon technology.

During A/B tests, the flicker effect is when the original page appears for an instant before the variation appears. Also called Flash of Original Content (FOOC), it impacts the experience for visitors and can lead them to abandoning a site, hurting your test results and overall engagement with consumers. While flicker may only appear for a fraction of a second, reducing it is therefore a key aim for brands looking to improve testing performance.

This blog explains what causes the flicker effect, a checklist of how you can overcome it, and the innovative way that Kameleoon eliminates it from your A/B testing.

1 What causes the flicker effect?

Essentially there are two sets of reasons that trigger the flicker effect:

Flicker effect reason 1: The script installation

The effect can be caused by the loading of the testing script itself. It appears the moment before the script becomes available and is ready to execute the necessary changes to load your A/B test. The best practice advice provided by A/B testing tool vendors should help remove this issue but follow our best practice checklist to track down and solve any problems.

Best practices for installing your A/B testing script

Essentially, the earlier your A/B testing script is available, the better for performance. These different factors all cause delays, with the first two being most important, as they’re linked to the way the script is integrated into your site's code:

The script is hard-coded into the header of your site rather than integrated via a tag manager
The position of the script within the page, compared to other tags (i.e. is at that top or bottom of the page?)
The weight of the script and performance of the hosting server. The lighter the script and the more powerful the server is, the faster the script will be loaded by the visitor’s browser.
The browser’s caching capacities.

Synchronous or asynchronous script: does it impact on the flicker effect?

There’s a lot of talk about synchronous and asynchronous scripts and their relative influence on page loading times and the flicker effect. Based on our experience, here are our insights.

The pros and cons of synchronous scripts

With a synchronous script the browser blocks the execution and the display of the rest of the page until the script has loaded. It treats the script’s instructions as primary, meaning it must be executed before any other action is taken. Therefore, nothing else is executed while the script is still loading.

The consequence is simple: for as long as the script is loading, a blank page will be displayed to your visitors. And if the server hosting the script doesn’t respond, the page remains permanently blank: for visitors, it looks like the site is no longer available. Clearly, nobody wants that to happen to their site.

However, the use of synchronous scripts is recommended by certain optimization tools, leading to their customers occasionally serving blank pages to their customers.

The pros and cons of asynchronous scripts

As the name suggests, an asynchronous script is loaded at the same time as other resources. So there’s no blocking or risk of blank pages being shown. However, neither is there a guarantee that the script will be available before the other elements of your page appear. That means there is a potential risk of the flicker effect, as elements supposed to be changed within your A/B test may appear before the script has loaded, and then quickly be replaced by the test variants.

Achieving the advantages of synchronous and asynchronous approaches

To deliver the best possible compromise between synchronous and asynchronous scripts, we have developed a method called “asynchronous blocking integration.” In this the script is loaded asynchronously, but the page remains blank until it is available, or for a certain time (a second, for instance) before loading the other elements.

This method offers the advantages of the synchronous script (a blank page), but with additional security - if Kameleoon’s Content Delivery Network (CDN) server doesn’t answer, the page itself loads after a second.

synchronous and asynchronous approaches

The result is performance similar to synchronous script in terms of removing the flicker effect, but with greater reassurance.

FLICKER EFFECT REASON 2: THE A/B TESTING SOLUTION

The second reason for the flicker effect is how the A/B testing solution interacts with the web page to implement changes. You might think that once the page has loaded, the risk of a flicker effect disappears - but it doesn’t.

To reduce the risk of flicker focus on:

The quality of the code within your test
The technologies you use - the A/B testing solution should use as little CPU and RAM as possible while offering the fastest possible interaction with the necessary elements of the page.

2 Checklist : Best practices to eliminate the flicker effect

In the final section of this post, we’ll explain how Kameleoon 100% removes the flicker effect. For those using other tools, following these best practice rules will help reduce the risk of flicker:

Install the Kameleoon script correctlY

Install the script of your A/B testing solution as high up as possible in the header of your site.
Install the script using asynchronous blocking integration. If your solution doesn’t offer this, we recommend using synchronous script. While not as strong as asynchronous blocking integration, it is more likely to prevent the flicker effect than an asynchronous script. However, it does create a (small) risk of crashing your entire site.
Don’t use tag managers so that you can keep control of the order in which scripts load.
Reduce the size of the script. Kameleoon has developed pre-processing that accelerates the script by loading only what is strictly necessary. If, for example, your A/B test does not need advanced targeting, there is no need to load the corresponding criteria.

Optimize the loading time of your tested pages

Optimize the loading time of your site and all its elements.
Use a CDN to accelerate script loading.
Most of all don’t use redirection for your tests. Unfortunately, A/B test redirection can only be done via JavaScript. Unlike a “real” server redirect, (HTTP 302 for instance), which is very light, JavaScript redirect is heavy and will slow down page loading significantly. In order to execute a JavaScript redirect with an A/B testing solution, the first page and all its resources (images, scripts, etc.) have to be loaded prior to the loading of the second page. No flicker effect for sure, but an extremely long initial page load. You should actually limit redirects to situations where there’s no alternative. A good example would be a test for a complete overhaul of your website, with the site hosted on a different server. In this case, redirects are almost mandatory. In all other cases, please avoid using JavaScript redirects!

Use a style sheet whenever possible

This makes only a minor difference if you use Kameleoon, where you are free to choose the implementations that suit you best without worrying about performance as this is taken into account automatically.

Don’t use jQuery with Kameleoon

Kameleoon doesn’t use external libraries, thus reducing the size of the script even further. New sites are based on new frameworks such as Angular or React, which most of the time make jQuery an unnecessary burden. Moreover, you should be careful: if your site already uses jQuery, it may not be sufficient to withdraw it from your A/B tests. If the jQuery script loads later, you can still have a flicker effect.

Optimize your code

The code should be optimized in order not to add unnecessary JavaScript instructions. Kameleoon’s graphic editor doesn’t generate JavaScript instructions and uses an engine to apply modifications made by the user. So this is not an issue for our users.

Modify the setInterval on the page

The setInterval is used to change the targeted elements as quickly as possible. The idea is to test different values to find the ideal parameter that makes the flicker effect disappear without overly increasing the CPU time your A/B testing solution uses. Kameleoon’s solution eliminates the need for setInterval as modifications are applied to a given element as soon as it is injected into the page’s DOM.

3 How Kameleoon 100% eliminates the flicker effect

As we’ve shown the flicker effect is caused by running A/B tests, Rather than just recommending best practices to reduce the risk of flicker we decided to attack the problem at its root and solve it once and for all.

How it works on a technical level

Our technology innovation impacts how Kameleoon interacts with the elements on the page once the script is available, taking into account any difficulties linked to the refreshment cycle of the browser. Changes are directly applied to the elements as soon as they appear on the page, without the need for a blank page, and without the risk of your website crashing.

For the technically minded our technology is based on the DOM MutationObserver standards. This is an industry standard implemented in all (mobile and desktop) browsers supported by Kameleoon, apart from IE 9 and 10.

That means that our flicker-free technology is compatible with 97% of all existing browsers. No change of code is needed so if the browser isn’t compatible, the flicker-free technology simply won’t be installed, and your tests continue to run normally.

To learn more about our flicker-free technology visit our documentation page