Website Optimization: A/B and Multivariate Testing 101
Joel Lockwood, Partner and Chief Marketing Officer
A/B and multivariate testing is a lot like going to the gym.
You know you should be doing it. You understand the benefits of doing it. Your friends who do it swear by it.
But getting started is, well, painful. So it’s much easier just to keep procrastinating.
Keep in mind, though, that when you put off testing, you’re really just putting off getting better results from your website or online marketing campaign.
The best way to get started is to learn the ins and outs of A/B and multivariate testing, and when to use each. Here’s a basic overview.
A/B Testing
In A/B testing, you test a change in a single website element against the original (control) version. For example, you might tweak the headline on a landing page, or change the size of the product image on a product page.
A/B testing is a quick way to determine whether one idea works better than another. For best results, test new ideas that are drastically different from your control. Got a dull, gray “Buy” button? Test canary yellow, or fire engine red.
Here are some tips on executing an A/B test:
- Test a change to only one element. If you change your headline copy, while also bumping up the font size and changing it to red text, you won’t be able to tell which change caused the increase (or decrease) in your conversions. And that would defeat the purpose of testing.
- Split your traffic 50/50, so that equal numbers of visitors will see option A and option B.
- Test enough for statistical relevance – but strike a balance. Too many marketers either pull a test too early, or let it run beyond the point when it is yielding useful results.
Multivariate Testing
A/B testing is clear, crisp, and to the point. There’s no arguing with the results. But if you have a variety of elements to test and want to speed up the process, consider multivariate testing.
In multivariate testing, you test multiple elements simultaneously. For example, on a product page, you might want to test two new headlines, two new product images, and two new order buttons. Assuming you use your current headline, image, and button as the control, that means you’ll have 27 different combinations of elements to test (3 x 3 x 3).
One of the goals here is to determine which combination of elements results in the most conversions. It’s all well and good to know that headline A beats headlines B and C, but multivariate testing also helps you determine that headline A works best with image B and button C.
Another goal is to find out which individual elements influence visitor behavior – and which don’t. Using your control as the baseline, you can then determine what change you caused by swapping in different headlines, versus different images or buttons. You might find, for example, that neither of your new headlines caused much of a change, whereas your new order buttons both caused significant spikes in conversions.
Because multivariate testing involves many variants, you’ll need more traffic to gather a significant sample size. This may mean you’ll need to run your test for a longer period of time. But because you’ll be testing multiple elements at once, you’ll probably still find that multivariate testing yields faster results than running a series of A/B tests.
Building a Testing Strategy
If you’re just getting started with testing, we recommend a simple, five-step approach:
1. Figure out where you’re at.
Determine your current conversion rates for the pages on your site that seek to drive conversions. Keep in mind that a conversion doesn’t necessarily involve a sale – it can also be someone signing up for your email list, filling out your form to request more information, or downloading your white paper.
Next, you’ll need to assign a value to each conversion. When a visitor downloads a free white paper, you don’t make any money up front. But if 2% of white paper downloaders typically go on to become customers, and the average lifetime value of your customers is $1,000, then you can say the long-term value of a white paper download is $20.
2. Decide where you want to be.
Identify your most important conversion goal for a section of your site – or for the entire site. Whether you want to collect more email addresses, convince more people to friend your company on Facebook, or persuade more people to actually buy a product, focus your testing efforts on this one goal. Also, make sure you know exactly which metrics you’ll use to track your results.
3. Get buy-in.
When you start trying to make changes on a website, you’re going to step on some toes. One of your executives may love the headers you’re using on your landing pages. Another colleague may insist that large product images are a waste of space.
You’re going to have to manage these expectations – and be prepared to slay some sacred cows – before you start testing. Before you dive in, have a meeting in which you clearly articulate the goals of your testing project and give everyone a chance to discuss. Take comments and suggestions – but avoid letting “committee-think” clutter up your testing program with secondary and tertiary objectives that distract you from your true focus.
4. Determine a testing order.
You can’t test all pages at once – but whatever you do, don’t start “safe.” Start your testing on the pages that have the most potential to deliver greater revenue. You’ll find them close to the end of your conversion funnel – your lead-gen forms, opt-in pages, and shopping cart pages.
Your next testing priority? We’ve all heard the expression, “You only get one chance to make a first impression.” Open Google Analytics (or a similar program) to determine which pages are your most popular entry pages. Are there any with high bounce rates? Focus on turning that around by pulling people deeper into your site.
Next, optimize any pages that have high abandon rates. By tweaking specific elements, can you prevent more visitors from jumping ship?
5. Identify which elements to test.
On the pages you’ve decided to optimize, give some thought to which elements may be driving visitors away, rather than compelling them to take the action you want them to take. Test some better alternatives.
Focus on eliminating clutter and distractions, making calls-to-action more visible, rewording copy to increase urgency, and reassuring the prospect about their decision to convert.
Testing Tools
Check out some of these testing tools to help you in your optimization process:
Parting Thoughts
Testing isn’t just about confirming what you already know – it’s about taking chances and being willing to fail in pursuit of the Next Big Breakthrough. You’ll make greater progress if you “fail faster.” That means you should run your tests on high-volume pages, and, if possible, use multivariate testing to execute multiple experiments at once.
Once you’ve learned what you can from your high-volume pages, apply those lessons to lower-volume pages. And don’t forget to share what you’ve learned with your email marketers and web designers.
After all, just like going to the gym, testing and optimizing your online marketing is much easier to do in a group.
Categories:
- Trends
- Best Practices
Sign up for regular updates from Ozone Online.
|
Joel Lockwood, Partner, President Joel brings over 20 years of marketing experience to Ozone Online. He has held executive marketing positions at large corporations such as Computer Associates, Sun Microsystems, Netscape and America Online, as well as several successful start-up companies. Joel excels at helping Ozone’s clients sync their online campaigns with their corporate marketing objectives. Email Joel. |
