Case Study: Winning With Website Personalization

The scenario below is a composite drawn from the kind of personalization work retailers have been running this year, not a single named company, but it reflects a pattern that’s become common enough to be worth walking through in detail.

The starting problem

A mid-size online retailer selling home goods noticed something in its analytics that didn’t add up: returning visitors, people who had already browsed or bought before, were converting at roughly the same rate as brand-new visitors landing on the site for the first time. That’s backwards. Returning visitors should convert better, they already know the brand and have shown intent. Instead, everyone was seeing the identical generic homepage, regardless of what they’d looked at before.

The hypothesis

The marketing team’s theory was straightforward: a returning visitor who’d previously browsed, say, kitchen items shouldn’t land on the same generic “shop now” homepage as someone arriving for the first time from a search ad. Showing that visitor products related to their last visit, or a banner reflecting the category they’d shown interest in, should shorten their path back to a purchase.

What they built

Using a personalization platform, the kind that had become accessible to mid-size retailers over the past couple of years without requiring a custom engineering build, the team set up a simple rule: returning visitors with a known browsing category saw a homepage module featuring related products and a relevant offer, while first-time visitors continued to see the standard homepage. They deliberately kept the change contained to one module rather than redesigning the whole page, so any result could be traced to a specific cause.

Running the test properly

Rather than rolling personalization out to everyone at once, they split returning traffic: half saw the personalized module, half saw the standard homepage as a control. This mattered more than it might sound, because without a control group it’s impossible to know whether a change in conversion is due to the personalization or just normal week-to-week variation in traffic quality.

What happened

Over several weeks, the personalized group converted at a noticeably higher rate than the control, and average order value ticked up slightly as well, likely because the featured products were already closer to what those visitors wanted. The lift wasn’t dramatic in isolation, a few percentage points, but applied across the volume of returning traffic the site already had, it added up to a meaningful revenue increase without spending an extra dollar on acquiring new visitors.

What they’d have gotten wrong without the discipline

Had the team skipped the control group and just rolled out personalization everywhere, they’d have had no way to separate the effect of the change from ordinary fluctuation, and no defensible number to bring back to leadership. Had they personalized everything on the page at once instead of one module, they wouldn’t have known which element drove the result, and might have quietly hurt something else on the page without realizing it.

The takeaway

Personalization worked here not because the technology was novel, but because the team treated it like any other test: a specific hypothesis, a contained change, a real control group and honest measurement of the result. That discipline is what turns personalization from an interesting feature into something with a provable return.

---

Keep exploring: browse more Marketing Analytics guides, see everything we published in 2014 or check out the Digital Business Marketing Awards.