How to Do A/B Testing to Improve Your User Experience in 2017
6 min read
What is A/B Testing
A/B testing (also referred to as split testing) is a kind of statistical testing used to determine which of the two variables (A or B) is more effective. This method identifies the best marketing and promotional strategies for an online business by comparing two variants of a web page or app based on key parameters such as purchases, downloads, and clicks. According to the narrower A/B testing definition, learning which website version performs better allows excluding guesswork from your website optimization process. Moreover, the method provides accurate statistical data indicating which version increases conversion rate.
All websites focus on involving their visitors in gainful activity. Every commercial website wants to increase its sales. For this, such sites need a visitor to buy their products or services hence comes the necessity to test the employed marketing models such as purchase funnel. Media and news websites expect their readers to sign up for chargeable subscription or click on ads. SaaS apps usually offer sign-ups for a trial version of their products and then work on persuading the user to purchase a premium account. Therefore, testing variations of a website that can have an impact on a user’s behavior brings substantial financial benefit to your business. Furthermore, the continuous process of A/B testing enables to utilize the latest resources, follow current trends, and update your websites and other tools.
Rigorous A/B testing can result in the increase in revenue if websites and landing pages are optimized accordingly. To succeed in getting a higher conversion rate, make sure to analyze the most relevant elements of the page based on the metrics you have previously defined.
There is an advanced version of this testing method called multivariate testing. It uses a more extensive set of metrics to compare and analyze more variables. Despite following A/B testing procedure, multivariate testing runs numerous tests that produce more specific results and exhaustive data. Check out Brian Massey’s article to learn more about this type of testing.
What is a b testing in email marketing
A/B email testing is used to compare two email campaigns and figure out which one works better. Since successful email marketing relies on an effective promotional campaign, A/B testing becomes an indispensable tool for designing a smart marketing strategy.
To carry out an A/B test of an email campaign, you need to devise two versions of one campaign and distribute them among your recipients equally. It is advisable to send out these email variations to a small number of your subscribers so as to involve the minimum number of people from your target audience in the testing process.
After variant A is sent to the first test group and variant B ends up in the second test group inbox, you can start counting clicks and opens. The version with more opens and clicks will be your winning campaign. Now you can send this variant to the remaining recipients.
There is a wide variety of A/B testing software available for both professionals and beginners. Most marketing tools include A/B testing option, which makes it easy and convenient to organize your email campaign. Check out the list of the best email marketing software of 2017 to find the package that meets your objectives.
If you are new to A/B testing, it is wise to adopt some of the commonly used practices of testing an email-marketing program. Here are the practical guidelines to rely on:
- One variable per test
You will only be sure which variable worked and which one failed if you test them one by one. Your test should check a single modified element at a time. Otherwise, you will risk getting lost in your results, as you will not be able to identify which variable performed better.
- Statistics matter
Before deciding on a winning version, remember to exclude a possible error or random chance. It is always better to double-check, so make sure to establish the statistical significance of test results.
- Never-ending search for new variables
Having covered the list of variables to test, you should continue looking for more options. Investigate and try out new aspects of the same variables. For instance, when testing an email subject line, you can also examine how using the subscriber’s name or mentioning a promotion affects the campaign. You can learn more about things to include in your A/B email testing in the article by Campaign Monitor.
How to Do A B Testing
What to test depends entirely on you. Before launching an A/B test, you need to define your goals and decide on the expected results so that you can create the necessary variables and customize the process. Once you have settled on things to test, your next step should be choosing a testing tool. Most of the available A/B testing software have a similar core test setup, so you can switch from one tool to another if needed.
According to a b testing best practices, there are two ways of setting up an A/B test:
- Substitute the variable before the web page loads
This method is used for testing only one element on a page. If you want to test a call-to-action button, for example, you should create its versions in your testing program. The A/B tool will then replace the original button with its variation at random before the visitor sees a fully-loaded page.
- Use a redirect to a different page
When testing an entire web page, you need to create an alternative page and upload it to your website. Say, you want to test a new design. If your original design is located at http://www.test.com/design.html, the new one should be placed at http://www.test.com/design1.html so that a part of your visitors is redirected to the page with the original design, and the other part gets to see the new one. Your testing tool will make the necessary redirect to your alternate URL.
After your variations are set up, you need to establish a conversion goal. For instance, if you need to test which sign-up button works better, your conversion goal will be a “Thank you for subscription” page shown to your visitors after they have signed up. The A/B user testing program records the events of conversion every time this page is displayed to a visitor. You can check which button performed better and resulted in more sign-ups after a sufficient number of visitors and conversions.
To have a more profound understanding of how to do such tests, check out some practical A/B testing examples
How does A/B testing work
To run an A/B test, you need to choose a web page and change some of its elements to create another version of this page. The modifications can be either simple like changing a button or headline, or more complex like a full redesign. After creating the required variants, you will need to divide traffic equally between the two versions of a web page – control (the original page) and variation (the modified page).
Analytics dashboard collects and measures visitors’ engagement with each version, and a statistical engine performs a comparative analysis of the processed data. The results determine whether any of the alterations influenced visitor behavior, and what kind of effect it had.
For instance, you need to identify which ad poster on your website will work better – black-and-white or color. So, you design two different posters and put one on the original web page (control) while the other one goes to an alternative page (variation). Then you direct 100 visitors to page A (control), and another 100 people get to see page B (variation). If your ad poster gets 35 percent clicks on page A and 20 percent on page B, it means A outperforms B. Now you can switch to laading page a b testing to ensure all bases are covered.
You can apply the same pattern of testing to any element on your website: headlines, call-to-actions, images, buttons, etc. When A/B testing Facebook ads, you might want to include links, texts, titles, interests, age, gender, locations, and other relevant keys in your metrics. Since there are endless options for ad tests, make sure your experiments remain useful and consistent.
Despite being a statistical technique of a promotional strategy, the concepts and outcomes of A/B testing can be easily understood by a non-statistician. However, before running tests, you need to ensure you have collected enough measurements for your sampling to have statistical significance. You should also monitor whether tests are executed correctly.
Prior to focusing on A/B testing, you should go through the basic testing a website checklist to see if major testing aspects are covered such as performance, usability, compatibility, security, and functionality.
A/B Testing Tools
Since the software industry offers a wide choice of A/B testing tools, it is totally up to you which one to choose. All testing programs can be divided into two categories: those making hypotheses based on the collected data and those running A/B tests. Conversion Rate Experts created a chart comparing key features of most popular A/B testing software where you can see which one will serve your purpose.
If you are looking for a free tool of reasonable power, Google Analytics A B split testing will meet your requirements. Experienced Google Analytics users who enjoy discovering its advanced features favor a testing function called Google Experiments. It allows storing all your testing data under your Google account, which is very convenient. A B testing using Google Analytics can be a good start for you to scale up and master controlled testing little by little.
When using A/B testing, it is crucial to stay creative and experiment with possibilities. Constant search and brave endeavors will lead to the desired results, so you should never stop trying.
