How A/B Testing Changed Digital Marketing in 2012
A year-in-review look at A/B testing in 2012, as Google folded Website Optimizer into Analytics and testing spread onto mobile.
Google retired its standalone Website Optimizer this year and rolled the functionality into Google Analytics as Content Experiments, a small technical change that reflects a bigger shift: testing has stopped being a specialty tool bolted onto a site and become an expected feature of the analytics platform itself. That consolidation, along with a handful of other developments, made 2012 a genuine turning point for how businesses think about testing.
Testing tools matured past the early adopters
Optimizely and Visual Website Optimizer, both a couple of years old now, spent this year moving beyond the early wave of tech-forward marketers who adopted them first. Agencies built testing into standard client offerings, and businesses that would never have described themselves as “data-driven” a year or two ago are now running their first split tests on a landing page or a checkout flow. The tooling stopped being the differentiator; knowing what to test became the actual skill in short supply.
Mobile broke the easy assumptions
As mobile traffic climbed to a meaningful share of most sites’ visitors this year, a lot of testing programs discovered their results didn’t hold across devices. A headline or layout that won decisively on desktop sometimes did nothing, or actively hurt, on a phone screen. Businesses running mature testing programs started segmenting results by device rather than trusting one blended number, which added real complexity to what used to be a simpler process.
Facebook’s public offering put spend under a microscope
Facebook’s IPO in May put a spotlight on digital advertising economics generally, and marketers spending real budget on the platform came under more pressure to prove it converted. That pressure pushed more businesses toward testing ad-driven landing pages specifically, rather than treating the page after the click as an afterthought to the campaign itself.
What businesses changed this year
- Testing moved from an occasional project to a standing item on the marketing calendar for businesses with real traffic.
- Device-level segmentation became standard practice rather than an advanced technique reserved for the largest sites.
- More businesses started testing earlier in the funnel, ad creative and landing pages together, instead of only testing checkout and signup pages.
- The conversation shifted from “should we test” to “what should we test next,” a sign the underlying habit has taken hold.
The gap that’s still there
For all the tooling progress, plenty of businesses running tests this year still aren’t reaching statistically sound conclusions, either stopping tests too early or drawing conclusions from too little traffic. The tools made running a test easy; they didn’t make interpreting one correctly automatic, and that remains the actual skill gap heading into next year.
Looking at where this leaves things
A/B testing in 2012 stopped being a differentiator reserved for large, sophisticated marketing teams and became table stakes for anyone spending real money to acquire customers online. The businesses pulling ahead aren’t the ones with the fanciest testing tool, they’re the ones who’ve built the discipline of testing consistently and reading the results honestly.
Keep exploring: browse more Marketing Analytics guides, see everything we published in 2012 or check out the Digital Business Marketing Awards.
