User Experience Metrics to Track in CMS Projects

Most teams pick metrics that sound smart… But under the hood, they’re just noisy, slow, misleading, or biased. But today, I'm giving you a framework to avoid that trap. It’s called STEDII and it’s how to choose metrics you can actually trust: — ONE: S — Sensitivity Your metric should be able to detect small but meaningful changes Most good features don’t move numbers by 50%. They move them by 2–5%. If your metric can’t pick up those subtle shifts , you’ll miss real wins. Rule of thumb: - Basic metrics detect 10% changes - Good ones detect 5% - Great ones? 2% The better your metric, the smaller the lift it can detect. But that also means needing more users and better experimental design. — TWO: T — Trustworthiness Ever launch a clearly better feature… but the metric goes down? Happens all the time. Users find what they need faster → Time on site drops Checkout becomes smoother → Session length declines A good metric should reflect actual product value, not just surface-level activity. If metrics move in the opposite direction of user experience, they’re not trustworthy. — THREE: E — Efficiency In experimentation, speed of learning = speed of shipping. Some metrics take months to show signal (LTV, retention curves). Others like Day 2 retention or funnel completion give you insight within days. If your team is waiting weeks to know whether something worked, you're already behind. Use CUPED or proxy metrics to speed up testing windows without sacrificing signal. — FOUR: D — Debuggability A number that moves is nice. A number you can explain why something worked? That’s gold. Break down conversion into funnel steps. Segment by user type, device, geography. A 5% drop means nothing if you don’t know whether it’s: → A mobile bug → A pricing issue → Or just one country behaving differently Debuggability turns your metrics into actual insight. — FIVE: I — Interpretability Your whole team should know what your metric means... And what to do when it changes. If your metric looks like this: Engagement Score = (0.3×PageViews + 0.2×Clicks - 0.1×Bounces + 0.25×ReturnRate)^0.5 You’re not driving action. You’re driving confusion. Keep it simple: Conversion drops → Check checkout flow Bounce rate spikes → Review messaging or speed Retention dips → Fix the week-one experience — SIX: I — Inclusivity Averages lie. Segments tell the truth. A metric that’s “up 5%” could still be hiding this: → Power users: +30% → New users (60% of base): -5% → Mobile users: -10% Look for Simpson’s Paradox. Make sure your “win” isn’t actually a loss for the majority. — To learn all the details, check out my deep dive with Ronny Kohavi, the legend himself: https://lnkd.in/eDWT5bDN

How well does your product actually work for users? That’s not a rhetorical question, it’s a measurement challenge. No matter the interface, users interact with it to achieve something. Maybe it’s booking a flight, formatting a document, or just heating up dinner. These interactions aren’t random. They’re purposeful. And every purposeful action gives you a chance to measure how well the product supports the user’s goal. This is the heart of performance metrics in UX. Performance metrics give structure to usability research. They show what works, what doesn’t, and how painful the gaps really are. Here are five you should be using: - Task Success This one’s foundational. Can users complete their intended tasks? It sounds simple, but defining success upfront is essential. You can track it in binary form (yes or no), or include gradations like partial success or help-needed. That nuance matters when making design decisions. - Time-on-Task Time is a powerful, ratio-level metric - but only if measured and interpreted correctly. Use consistent methods (screen recording, auto-logging, etc.) and always report medians and ranges. A task that looks fast on average may hide serious usability issues if some users take much longer. - Errors Errors tell you where users stumble, misread, or misunderstand. But not all errors are equal. Classify them by type and severity. This helps identify whether they’re minor annoyances or critical failures. Be intentional about what counts as an error and how it’s tracked. - Efficiency Usability isn’t just about outcomes - it’s also about effort. Combine success with time and steps taken to calculate task efficiency. This reveals friction points that raw success metrics might miss and helps you compare across designs or user segments. - Learnability Some tasks become easier with repetition. If your product is complex or used repeatedly, measure how performance improves over time. Do users get faster, make fewer errors, or retain how to use features after a break? Learnability is often overlooked - but it’s key for onboarding and retention. The value of performance metrics is not just in the data itself, but in how it informs your decisions. These metrics help you prioritize fixes, forecast impact, and communicate usability clearly to stakeholders. But don’t stop at the numbers. Performance data tells you what happened. Pair it with observational and qualitative insights to understand why - and what to do about it. That’s how you move from assumptions to evidence. From usability intuition to usability impact. Adapted from Measuring the User Experience: Collecting, Analyzing, and Presenting UX Metrics by Bill Albert and Tom Tullis (2022).

Strong signals bring user needs into focus. Over the years, I’ve worked with many teams that create user personas, giving them names like “Cindy” and saying things like “She needs to find this feature” to guide their design decisions. That’s a good start. But user needs are more complex than a few traits or surface-level goals. They include emotions, behaviors, and deeper motivations that aren’t always visible. That’s why we’re building Glare, our open framework for data-informed design. We've learned a lot using Helio. It helps teams create clear, measurable signals around user needs. UX metrics help turn user needs into real data: → What users think → What users do → What users feel → What users say When you define the right audience traits and pick the helpful research methods, you can turn vague assumptions into specific, actionable signals. Let’s take a common persona example: Your team says, “Cindy can’t find the new dashboard feature.” Instead of stopping there, create signals using UX metrics to define usefulness better: → Attitudinal Metrics (how Cindy feels) Usefulness ↳ 42% of users say the dashboard doesn’t help them complete their tasks Sentiment ↳ Users overwhelmingly selected: Confused, Frustrated, Overwhelmed Only 12% chose Clear or Confident Post-Task Satisfaction ↳ 52% of people are satisfied after completing key actions → Behavioral Metrics (what Cindy does) Frequency ↳ Only 18% of users revisit the dashboard weekly, down from 35% last quarter → Performance Metrics (how the product supports Cindy) Helpfulness ↳ 60% of users say they needed help materials to complete a task, suggesting the experience is unclear With UX data like this, your team can stop guessing and start aligning around the real needs of users. UX metrics turn assumptions into signals… leading to better product decisions. Reach out to me if you want to learn how to incorporate UX metrics into your team workflows. #productdesign #productdiscovery #userresearch #uxresearch