You Are What You Measure

The Ecommerce Success Metric That’s Holding You Back

Peter Drucker once said, “What gets measured gets managed.”

Well, actually, I've just been told he didn't say it. Someone else did. It doesn't matter. Someone said it, and it's still a simple truth. Whatever you choose to measure, you inevitably optimise for. And that’s precisely the problem in ecommerce today.

For years, conversion rate has reigned as the KPI to end all KPIs. It is a single number that is easy to track, easy to understand, and easy to compare, determining how many convert online.

Although, also for years, it's been hailed within the conversion optimisation community as folly. For many reasons; it's an average of average of averages, it's binary, it's retrospective, it's a measure of an output, not performance; there are too many to list. It's just a bit crap.

But the biggest reason I think conversion rate is the wrong (yes, you heard me, wrong) measure of success is that it's a perverse incentive. It pushes e-commerce brands to optimise for one-way, short-term transactional wins, often at the expense of long-term, two-way customer value.

And once you see it, you can’t unsee it.

The way you measure success dictates your strategy. The wrong metric breeds the wrong behaviours. It’s time to rethink how we measure success online, because conversion rate isn’t just flawed, it’s fundamentally misaligned with how people actually buy.

The Measurement Trap

This isn’t just an e-commerce problem; it happens everywhere. Let's go broad before we go deep.

Disney

My beloved brand. We all know it's a love-hate relationship for me.

I wrote about this recently, but to recap, Disney changed its measure of success in 2020. The company stopped measuring for attendance and bringing people to the park and began prioritising per-guest spending. That change in measurement changed everything, specifically the things they began optimising for.

Like most theme parks, it has always optimised for attendance and, by virtue, capacity. Increasing the park's capacity means that more people can safely enjoy it. The behaviours to optimise that measure of success include building new rides, character meet-and-greets, attracting people to the park through resort offerings, offering perks, and anything that involves guest satisfaction to ensure attraction and a great experience for repeat visits.

One of those perks was the free FastPass system, a perk that allowed you to skip the queue of three rides of your choosing (give or take, it got more complicated than that). Perks are perks, though. They're immeasurable and hold indirect attribution.

When the measure of success shifted in 2020 towards something more attributable, per-guest spending, their behaviours to optimise that measure also changed. Disney replaced its free FastPass with a paid, tiered system. The question was no longer "How do we increase capacity by flowing guests freely throughout the park?" but "How do we monetise each moment within it?". Lightning Lane became a new norm and the standby line became a deterrent.

They created a perverse incentive. Buy the Fastpass system by demoting an existing behaviour; the standby line. And so standby lines became slower, perhaps intentional to psychologically encourage guests to think "we'll buy the fastpass". Why would that sign saying a 40 minute wait be accurate?

Google

You're not escaping this one, Google.

In 1998, Google’s founders, Larry Page and Sergey Brin, were loved. They received fan mail. Some of which contained money to ensure they carried on doing what they were doing; that's how much people loved them.

Their mission was clear from the start: "to organise the world’s information and make it universally accessible and useful." And alongside of that mission they held a belief that we need to read three times to make sure we're not misreading, as we laugh along.

“Advertising income often provides an incentive to provide poor quality search results.”

Oh, how times have changed. Fast forward 20 years, and Google’s core revenue stream is, you guessed it, advertising.

Compare a Google search result from 2016 to today. Back then, if you searched for "wedding dresses," the first organic result appeared just 240 pixels down the page. On average, there were seven ads for every eight results.

Fast forward to 2024, and that first organic listing is now buried 500 pixels down. The number of ads? An eye-watering 32 per 8 results. Optimising for their measurement protocol and ad revenue. How long before the entire first page is ads?

With our Disney example, think of the adverse behaviour they'd want to demote to psychologically promote the behaviour they optimise for. I'm not saying this is true, but why would the organic listing be relevant? The promoted (paid for) listings would clearly be the most relevant to the search term, as Google wants you to click on them. So what would that mean if the organic listing just so happened to be less relevant?

Tin-foil hat time. I think, like Disney, Google created a perverse incentive. Ensure and click on a relevant promoted ad by demoting an existing behaviour, a potentially irrelevant organic listing. A question that makes you ask: Does Google optimise for search quality, or does it optimise for ad revenue?

We've seen it in education; does the education system optimise for bringing children up in the world who will thrive in society when it measures in short-termist exam results?

We've seen it in selling a car; does a car salesman optimise to get you the best car possible when they are measured and incentivised on commission?

We've seen it in politics; is a political party's true purpose to enact long-term policies aligned with their ideological vision for the country, or is their success measured by polling numbers, soundbites, and short-term popularity wins? Let's move on, shall we?

Conversion Rate is a perverse incentive

Charles Goodhart once said “when a measure becomes a target, it ceases to become a good measure"

He definitely said that one.

Despite it’s flaws, conversion rate has become a measure of success. But it has taken a step beyond that and become a target, which Charles would therefore render as a poor indicator of success. Something that brands chase down to optimise, report on a weekly basis in trade meetings, use as a yard stick to compare performance between attribute A and attribute B. And, more than anything, conversion rate is easily gamified.

We could manipulate conversion rate if we wanted to:

Increase PPC traffic to inflate the numerator. All landing on the PDP page, probably.

• Remove brand terms from PPC traffic. I think.

• Use aggressive discounting to boost short-term sales.

• Add pop-ups, urgency messages, and FOMO tactics that pressure users into buying within session.

And brands do this. If conversion rate is down one week, it’s time to put on a sale. It’s why some can’t get off the discount drug. It’s why most of the tactics we talk about in our Intent Gap Report focus on the upside - the net aggregated conversion rate improvement, not the downside - the erosion of long-term trust, LTV and repeat visits.

Conversion rate is an aggregate metric that hides real customer intent. A 2% conversion rate doesn’t tell you who converted or why. It’s a binary state that ignores the 98% who didn’t buy (let’s face it, conversion rate is always 2%) and what they actually needed.

Because it is an aggregate binary measure of success, the behaviour it breeds is also aggregate and binary. Binary in that it only aims to convert one-way, and aggregate in that all the solutions before it are also aggregated; in other words experiences are aggregated, optimising for “pages,” not “buying stages.”

This last point is crucial. People don’t buy in pages. They buy in stages. There’s our famous line again.

A product page visit doesn’t mean someone is ready to buy. They could be browsing, refining, evaluating, or deciding. Yet, ecommerce teams keep optimising aggregate pages, ignoring the reality that a homepage visitor might be closer to buying than a checkout visitor (that’s rare, I’ll admit, but possible). They do this because they measure on an aggregate metric.

That’s the intent gap. A digital disconnect.

A New Way

“If conversion rate is the wrong metric, what’s the right one?” I hear you ask.

It must be said, we’re not here to go against conversion rate despite all the above. It’s entrenched and needed. There’s a rebalance needed is all. So we should really be saying stages AND pages, rather than stages not pages.

That small disclaimer aside, the answer lies in understanding where customers are in their buying journey. Instead of optimising for a single event (purchase), we should measure and respond to their intent stage. We should understand where users are in their buying stage, what intent they hold, and optimise to support them to move from one buying stage, to the next.

1. Low intent – Just looking, no clear intent. Bottom of the funnel. Your job is to engage them in order to build their intent.

2. Building intent – In the process of shopping in some form, likely has an affinity towards something or finding their feet within the process. Middle of the funnel.

3. High intent – Ready to buy or could be ready to buy. Certainly in market and top of funnel. Your job here is to ensure they convert, they should be prioritised.

Each stage has different intent predictions within it and each stage requires different interventions.

What does this look like in practice?

• Instead of measuring conversion rate, measure composition and stage progression; how many users move from low intent to building intent, from building intent to high intent. Even backwards - from high intent to low intent.

• Instead of showing the same pop-ups to everyone, trigger engagement based on real-time intent signals (e.g., offer help to those in building intent, showing signs of struggle, with a high intent to exit)

• Instead of discounting broadly, tailor offers based on buying stage, reserving them for those truly on the fence, not eager buyers who would have paid full price. Protect your margin.

The goal? Optimise for progress, not just purchases. Treat your website as it’s meant to be; a method of supporting users by building their intent.

You are what you measure.

This is why we built Made With Intent.