This is a static snapshot of a website, so only a few pages are working

Frequently asked questions

What exactly counts as "visitors tested"?

One visitor is counted when the visitor is introduced to a new test for the first time. All the subsequent times when that same test is presented to the same visitor are not added to this number. If you run out of this resource all active tests will be stopped.

Example: You are running one test on the element that is present on every page of your website. Jane visits your website for the first time and explores 10 pages. Jack visits your website, but quits after checking out 2 pages and comes back the next day. All this action will count as 2 visitors tested.

Important! A/B Testing Lab does NOT provide any new visitors to your website. This service provides the tools to run content experiments with the traffic that your website is already getting by itself. When your plan ends or you stop the tests, your visitors will see the default version of the website without any changes. Simple as that.

Do I need a lot of technical expertise to use A/B Testing Lab?

Not really. Installing our snippet onto your website is rather straightforward if you have at least some knowledge on HTML. Creating tests with visual editor is a breeze. If you know some CSS and HTML, it will allow you to create more complex tests, but that's not necessary to conduct effective testing.

Are there any limits to the amount of tests?

You can run as many tests as you want.

Will A/B Testing Lab work with any browser?

Visitors with all modern mobile and desktop browsers (IE6+) will be able to see your tests. The visual editor will work fine with most browsers, but it is recommended to use the latest version of Chrome.

How does A/B Testing Lab work?

  • Load your website into visual test editor.
  • Choose the element you want to optimize.
  • Describe a variation. You may specify different styling too.
  • Choose a conversion event. How will you measure success?
  • Choose the reach of your test. How many visitors should participate?
  • Let the test run for a while. We will notify you when it ends.
  • Review the results and make changes accordingly.
Once you create, edit or delete tests, they are immediately available to your visitors. The little code snippet you copy to your website loads a script from our servers, which scans every page on your website as it loads, replacing specified content with variations. This happens so seamlessly and quickly that your visitors do not notice anything.

Does A/B Testing Lab affect the loading time?

Barely if at all. The script is compressed and takes about 35Kb, depending on the number of tests you are running. Compare that to the average web page which is 1,935Kb and uses 95 HTTP requests. It is completely static and it will be cached in the browser, so a file is downloaded only on the first visit or when you change the tests.
To avoid "flashing" effect (when the original content is visible for a split second until it is changed), which might occur on slower computers or bigger websites, script hides web page content for 500ms. This is rarely noticeable to the end user, but gives enough time for the script to put variations in their place. If you have problems with this approach or simply do not like it, you can adjust or remove the content of the <script> tag while keeping the src attribute untouched.

Will A/B tests affect my SEO?

No, according to Google. Just make sure that your variations are not completely different versions of the page.

Why don't you allow to specify a test goal instead of reach? I want to stop testing once noticeable improvement is reached.

This might sound like a good idea, but in reality the results of such tests might not be statistically meaningful. If you run a test up until you get a result you want, the risk of getting a false-positive is greatly increased. Instead you should decide on test limits in advance and then stick to it. If you do not get a definitive result, then run a new, improved test, instead of running the same test until you see the desired result. Read more on this topic here.

Did not find the answer you were looking for? Contact us now.