The Rise of A/B Testing
Where A/B testing came from, from Claude Hopkins's mail-order coupons to the tools every marketer can run a test with in 2012.
A/B testing feels like a product of the internet age, but the core idea is closer to a century old. Advertiser Claude Hopkins was tracking coded coupons in newspaper ads back in the 1920s, comparing which headlines and offers pulled more responses and writing the results down in what became his book “Scientific Advertising.” Statistician Ronald Fisher formalized the underlying logic of controlled experiments not long after, in agricultural research of all places, giving marketers decades later the statistical foundation they’d eventually borrow for split testing on the web.
The direct mail era proved the concept
Long before anyone had a website to test, direct mail marketers were running the same basic idea at scale: mail one offer to half a list, a different offer to the other half, and count the responses. It was slow, expensive to run and limited to a handful of variables at a time, but it established the principle that would later define web testing: don’t guess which version works, measure it.
The web made testing fast enough to matter
What changed on the internet wasn’t the concept, it was the speed. A direct mail test took weeks to print, mail and tally. A web test could show a meaningful result in days, sometimes hours, on a high-traffic page. Early testing platforms like Offermatica, founded in 2004 and later acquired by Omniture, brought this capability to marketing teams who previously only had it available through custom engineering work.
The moment that made testing famous
The 2008 Obama presidential campaign became the case study everyone in marketing eventually heard about. Dan Siroker, who led the campaign’s testing efforts, ran experiments on the campaign’s donation and signup pages, testing different combinations of images and button text against each other. The lift in signups from finding the right combination was large enough that Siroker went on to co-found Optimizely two years later, built specifically to bring that kind of testing to businesses that didn’t have a political campaign’s engineering resources.
2010 was the real inflection point
Optimizely and Visual Website Optimizer both launched in 2010, and that’s the year testing stopped requiring a developer for every experiment. Before that, running a split test usually meant writing custom code or paying for expensive enterprise software. After it, a marketer could open a visual editor, change a headline or a button and launch a test the same day.
Where it stands now
By this year, A/B testing has become closely associated with how the largest technology companies operate. Amazon, Google and others are known for running enormous numbers of concurrent experiments on their own products, and that visibility has made testing culture aspirational for businesses of every size. What started as a newspaper advertiser tracking coupon codes by hand has become an expectation: if you’re not measuring which version of something works, you’re behind the businesses that are.
Why the history is worth knowing
Understanding where this practice came from is a useful check against treating it as a magic trick unique to software. It’s the same discipline Hopkins used in the 1920s: form a specific hypothesis, run a controlled comparison and trust the result over your own opinion. The tools got faster; the underlying idea hasn’t changed at all.
Keep exploring: browse more Marketing Analytics guides, see everything we published in 2012 or check out the Digital Business Marketing Awards.
