Your guide to AB testing and how to get started

AB testing (also known as split testing) has proven to be an effective data-gathering tool for companies, as they try to interpret how people behave. This could be anything from landing on a webpage, opening an email, or responding to a Facebook ad. 

Testing different variables allows the marketer to experiment and find out what performs best before they launch the campaign or marketing asset.

AB testing is one of the most popular methods of optimizing marketing materials, with over 77% of companies running AB tests at any one time. 

The reason for its popularity is simple: AB testing works. Over 70% of businesses saw increased sales from AB testing materials before launch.

Here’s everything you need to know to conduct your own AB tests. 

What is AB testing or split testing in marketing?

Often AB testing and split testing are used interchangeably, as the principles and objectives of both are the same.

Think of AB testing (or split testing) as a contest where two players face off to test their strength. 

You put two versions of your marketing assets against one another and segment your audience to see which one emerges as the winner. 

Segmenting audiences should be a familiar process to most businesses. Marketing segmentation is a strategy used to tailor products, services, and marketing materials to different subsets of consumers based on their behaviors and needs.

In marketing segmentation, the audience segments are created because the behavior has already been determined. AB testing or split testing is the method that determines said consumer behavior.

The players in the AB test arena are categorized as:

• Challenger: When you start a test, you create a new version (called a variant) to challenge your existing champion page. This is the Challenger.
• Champion (Winner): After running the test, the version with the best conversion performance is crowned the winner. It becomes the Champion variant. 

In an AB test, a web page or mobile app screen is amended to create a second model of the same page. This can be a simple change in the headline or hero shot or a complete redesign. 

Then, half of the website’s traffic is shown the original version (the control) and the other half is shown the modified version (the variant). 

Audience engagement is analyzed and measured to find which fared better. 

The winning version is deployed as good-to-go and targets the wider audience pool. 

Even minor changes can impact conversion rates. The color of the call-to-action button, for instance, is a hotly debated topic among conversion rate optimization (CRO) experts. Some say red works wonders. Others vouch for green. 

There could be other changes like the size of the CTA button or removing the banner from the top to changing the form on the sign-up page and then assessing performance. 

Or it could be a change in the Add to Cart/Purchase button text or font to see if conversion rates increase.

Ultimately, you shouldn’t need to guess what works best. You should AB test it.

The benefits of AB testing

AB testing is not new. It’s been used for experimental results in other fields since the 1920s. However, it was brought to marketing in the 1960s and 70s to evaluate direct response campaigns (source: HBR). 

One big benefit of AB testing is that it’s low on cost, but high on returns. 

Here are several more reasons why companies should make AB testing a part of their testing portfolio:

Boost your bottom line

Controlled testing and analysis can determine which marketing strategies create high user engagement and work best for your product. 

In the current, highly dynamic digital landscape, continuous experiments will help maximize ROI.

Improve conversion rate

Any positive change, be it in the text or the CTA color, can enhance user experience and garner more clicks. 

This means more conversions and more lead generation. For example, eCommerce companies often find users abandoning items in their shopping carts. You can help tackle this with AB testing. 

Lower bounce rate

 Does your website have a high bounce rate? AB testing can help make the right changes to retain them.

Solve pain points

Users may face common problems while navigating a web page. It may be text that’s hard to understand or an elusive CTA button. AB testing helps figure out and resolve these pain points.

Go by statistics, not gut

Don’t rely on guesswork. Take data-driven decisions on which headline, image or video works best on your landing page.

Test new features for optimum outcome

Want to introduce a new feature change to your webpage? If you AB test, the outcome becomes predictable and value-adds to your page. 

This is especially relevant if the change affects customer data or the purchase funnel. A feature change done without testing can be a gamble—it may or may not pay off. Testing brings certainty to the outcome. 

Manage a balance between the marketing team and developers

Since the changes for AB testing do not require hard coding, it simplifies life for both marketers and coders. 

Coders are not constantly making changes without any data to back that request for change. Marketers can manage variations all by themselves and optimize performance. 

Why AB testing tools are vital for growth managers

As innovative businesses find new ways to optimize conversion rates, AB testing has moved from being an occasionally used strategy to a continuous, structured process that marketers rely upon.

For best results, it’s ideal for marketers to work with service providers who specialize in this space. 

Running in-house testing without the right technology backup can be a challenge for companies: The effectiveness of the tests would be lower, the processes longer, the analysis not fully reliable, and the post-test actions taken may not be completely accurate. 

With a specialist tech vendor, companies can run five to ten tests in a week. 

Alternatively, if they opt for in-house testing, they would probably manage two or three tests a month, at best. 

When companies work with a reputed service provider, they don’t just gain access to technology – but also buy into their know-how, experience, consultancy prowess, agility, and benchmarks with other industries and countries. 

It’s critical to select a tech provider who comes with all these capabilities and supports you in every step of your testing journey: 

Coming to AB testing tools, there is no lack of choice in the market. As a growth marketer, you need to pick your tool with care.

Besides prioritizing effectiveness and ease of use, opt for a tool that best fits your business size, resources, and learnings from any past experiences.

There are a few key features that fully-loaded AB testing tools come with: 

A/B/n Testing: For multiple versions of a page. You may want to test more than just one version of your page – say, an A/B/C test. Your tool must cater to this.

Multivariate Testing: For multiple elements on the same page.

Visual Editor: For building variations without any code.

Full Stack: For AB testing both web pages as well as mobile apps. 

Autopilot: Automatically manages traffic, to shift more traffic to the pages that are performing better. 

Heatmaps: To decipher how visitors are actually using a webpage.

Step-by-step: How to Perform AB testing

Wondering how to do AB testing? From start to finish, here are the steps to follow:

Step 1: Do your homework

Know everything about your website’s current performance, from how much traffic it generates and the most viewed pages to the conversion rates and bounce rates.

Step 2: Get an AB testing tool 

Find an AB testing tool that is comprehensive, intuitive, and easy to use. The tools should be designed specifically for the Web, Mobile Web, and Mobile App platforms. Keep a checklist ready and see if it matches your requirements.

Step 3: Create a challenger

As you work on optimizing your web page, there may be many variables you want to test. Only pick one or two at a time, otherwise, you won’t know what worked. Build a Variant or Challenger page incorporating this change.

Step 4: Split your sample group or wait for the numbers to add up

Split your sample group equally and randomly if you’re testing an email. If you’re testing a web page, there’s no fixed group, just let the test run its course. However, ensure it’s long enough to clock in a definitive number of views.

Step 5: Run the test (and test both variations simultaneously)

If you test Variant 1 today and Variant 2 a week later, you’ll be left guessing whether your web page’s performance change was driven by design or timing. 

Step 6: Measure the test results using an AB testing calculator

Measure which Variant emerged as the winner. You can either do this manually or feed the results into an AB testing calculator. Deploy the winning variation.

What type of content can you AB test?

AB testing allows you to test almost anything in your marketing material. But since time and resources are limited, it’s best to test the most impactful elements. 

Here’s what you can (and often should) AB test:

• Headlines and sub-headings
• Hero shots and images 
• Design elements including the menu bar, cart page, and success page 
• Homepage and landing page
• Size, color, and font of CTA button
• Forms 
• Banner areas
• Product recommendation areas
• Onboarding steps
• Lead capture form length
• Navigation bar
• Social Proof
• Videos on website and app
• Various combinations to showcase Product pages 

AB testing examples

Test hypothesis: Cart page with top menu bar vs. Cart page without top menu bar

Let’s take an example of an eCommerce company. It’s new on the market, has just launched its website, and is looking to attract visitors to its webpage to drive more sales. 

E-retail is a crowded marketplace and the company wants to create a website that stands out and garners eyeballs.

The current version of the company’s cart page comes with a menu bar running across the top of the page. 

The bar lists all the product categories that users can browse through before they arrive at a purchase decision. It’s a handy tool, but is it making the page look cluttered? The company decides to find out.

It runs an AB test, testing two versions of the cart page—one with the menu bar and one without.

The results start flowing in over the next three weeks, as visitors are randomly assigned to visit one of the two pages.

The KPI being measured is purchase performance and based on the data, the test without a top menu bar is performing 32% better than with.

The company can then make the informed decision to refrain from using top menu bars on future cart page designs to continue improving purchase performance on a wider scale.

Test hypothesis: Buy now vs. Cart icon

When this eCommerce company launches a mobile web application, it uses AB testing again to find the best-fit Call to Action button.

In the original version of the webpage, the CTA button read `Buy Now’. It’s an unambiguous CTA, but the marketing team wants to test it against something more subtle.

So it sets up an AB test, where two versions of the web page are built – one with a `Buy Now’ CTA and another with a `Cart Icon’ CTA.

The results provide the company an invaluable insight into the consumer: Does a subtle call to buy works better than a direct call to buy. 

Again the measurement metric is purchase performance.

The AB tests show that having a Buy Now CTA has increased purchases by 16%. Going forward, the eCommerce company rolls out Buy Now buttons across the mobile application. 

Text-based onboarding process vs. banner/image-based onboarding process

AB testing can help companies make important decisions on what visuals or text appeals to consumers on the mobile application platform as well. 

Here’s another example: 

A new food delivery company launches an app and begins its marketing campaign, to get consumers to download it. However, two months later, it finds that traction isn’t high, despite the continuous increase in demand for food delivery. So the company decides to go back to the drawing board to see if it can change its application to attract more downloads. 

The app currently has a text-based onboarding process. Are consumers finding it too boring to proceed with? So, the company decides to start AB testing to find answers. 

An alternate, banner-based onboarding process is created for the app. The two are tested for registration conversion to see which one performs better.

The results come in, and they’re proven correct: the image-based onboarding process sees a completion rate 27% higher than the one without.

How Insider can help you start AB testing

Insider has worked with companies across the board to test new features and add-ons on their webpage and mobile apps, increase conversion rate and deliver unique user experiences. 

Partnering with Insider goes beyond a testing tool. The partner organization also benefits from strategy, expertise, and account management. 

Every step of the journey, from ideation and strategizing, to implementation, analysis, and iteration, is covered.

You don’t need to just take our word for it, here are some real-life examples from our partners. 

How airBaltic increased CVR by 5.8% 

airBaltic, a Latvia-based regional airline, came on board with Insider to use its AB testing tools. Its goal was to optimize the user experience on its digital channels. 

Goal 1: Build a unique experience for each traveler on their digital channels.

• Goal 2: Improve main KPIs like conversion rate and average session duration on their website.

The AB test process:

Personalize the homepage for different audience segments.

• AB/n testing the homepage where it used 3 different variations, also keeping the original version.

Test result: The three-week experiment showed an alternate page was performing better than the original, both in terms of user experience and average session duration.  

Number crunching: A conversion rate uplift of 5.8% on the winning variation.

Read airBaltic’s story.

The First Group guide to AB testing

A Dubai-based property developer, First Group, worked with Insider to AB test its investment guide:

Goal 1: Get visitors to their website, so they can start talking with the brand. Get the visitors to engage with the content. Nurture leads.

Goal 2: Get visitors to download a copy of their investment guide.

Goal 3: AB test notifications for better conversions.

The AB test process:

• Personalize all pages of the website.
• Send notifications to visitors, suggesting they download the guide.
• AB test five different messages, sent to different portions of visitors.

Test result: Substantial increase in conversion rates for customers who received notifications to download the guide. 

Read First Group’s story. 

Two Insider tools to support your AB testing strategy

To make the process seamless, Insider has two platform-specific tools, designed to optimize AB testing on the Web and on Mobile Apps:

1. Content Optimizer

Built for mobile apps, Content Optimizer AB tests in a variety of interesting ways, such as: 

• Personalizing Mobile App content to boost conversion rates.
• Changing the order of a food menu list (for example) or product description details to enable easier navigation.
• Optimizing sliders on the homepage to facilitate ease of use 
• Tests various layouts to find which one has the highest conversion rate.
  

2. Onsite Experiment

Catering to Web platforms, Onsite Experiment enables businesses to make changes to elements, test new ideas, and improve the user experience. Its feature portfolio includes:

• Enables users to add, remove, move, or resize elements with ease.
• Edit color, background, border, and layout using any parameter, be it font style or font family.
• Its high-end JS and CSS editor allows users to explore advanced editing options.  
• Its undo and redo features enable users to see their footprints while editing a page.
• It makes selecting the right element a breeze—as the tool’s Editor lets you select elements with respect to their hierarchy.

Conclusion

AB testing makes one thing crystal clear to marketers: you don’t know what works with your end user until you’ve tested it.

If you’re ready to start optimizing your user experience, increase conversion rates, and help out your bottom line with AB testing, Insider can help. Why not schedule a demo and chat with one of our experts?

⭐️⭐️⭐️⭐️⭐️

“The best solution for A/B testing. Through Insider, we were able to implement multiple A/B tests at the same time on our website without any technical requirements or problems. We were easily able to understand the campaigns that performed better, we got insights into our consumers’ preferences and we were able to optimize our site to meet their needs easily. It’s so easy to set up and understand, and it doesn’t take long to see the results.

• Senior Digital Marketing Manager

⭐️⭐️⭐️⭐️⭐️

“Insider has been a massive success for us thanks to it’s testing capabilities. We use Insider’s A/B testing to test various elements on our website to improve conversion rate using the suggestions we got from the winning test results. We are happy to report that Insider has been a success for us, having tremendously improved our lead collection and helping us close more deals and drive profits up.

• Verified G2 Review

Read more reviews from happy customers on G2. 

AB testing FAQs