Part 4: Measuring What Matters: New Metrics for Digital Healthcare Success

By Jennifer Brooks-Mason and Sarah Richardson

“If you can’t measure it, you can’t improve it—but measure the wrong thing and you’ll improve the wrong outcome.” ~Peter Drucker inspired thinking.

Welcome to the fourth edition of Leading Digital in Healthcare. In our ongoing series, Sarah Richardson and I have been exploring the evolving role of the healthcare technology leader. In this edition we are diving into a topic that has never been more important or misunderstood - it’s metrics that matter in digital healthcare.

As we have briefly touched upon in our previous editions, today’s digital leaders must evolve from system maintainers to strategic stewards and measure what matters most across clinical, operational, financial, and human dimension. As we lead through digital transformation, the onus is on us to ensure that our organizations have the metrics and goals in place that align with clinical, patient care and consumer experience priorities.

1️. Why Traditional ROI Is No Longer Enough

Many health system leaders still default to familiar metrics like ROI and uptime. But today’s digital leaders are being asked to deliver more than stability, we are accountable for enabling better clinical outcomes, operational agility, and meaningful patient experiences. That means shifting how we measure success.

Application Rationalization and Value Assessment Across the healthcare industry, studies and industry analysis consistently show that a significant portion of application portfolios, often approaching 30%, are functionally redundant or underutilized. This redundancy leads to unnecessary complexity, cost, and operational friction.

The problem isn’t just the number of tools, but the lack of clear value they provide. Instead of narrowly focusing on whether a project yields immediate hard-dollar savings, healthcare leaders must evaluate whether it delivers proof of value and, ultimately, proof of sustainability.

Strategic application rationalization is not just a cost-cutting exercise; it’s a pathway to long-term operational resilience and smarter digital investment. The mature conversation moves from “What’s the ROI?” to “How does this investment advance care quality, equity, and resilience?”

o   From Proof-of-Concept to Proof-of-Sustainability: Rethinking the Maturity Curve It is no longer sufficient to stop at a successful Proof-of-Concept (PoC). While PoCs demonstrate technical feasibility, HIT leaders are increasingly expected to progress rapidly toward Proof-of-Value, clear evidence that the solution impacts care delivery, efficiency, and/or safety. But the true differentiator now lies in reaching Proof-of-Sustainability. This means ensuring long-term alignment with organizational strategy, workforce capacity, regulatory compliance, and cost efficiency. It is not just about launching innovation; it's about embedding it into the operational fabric and proving it can scale, persist, and evolve.

o   Beyond ROI: The Rise of the Balanced Scorecard for Health IT Investments CFOs understandably continue to press for hard-dollar ROI, how many dollars in, how many out. But HIT leaders must adopt a broader lens. The most strategic leaders are now advocating for a balanced scorecard approach that connects technology spend with clinical outcomes (e.g., reduced readmissions, improved diagnostics), operational improvements (e.g., workflow optimization, staffing efficiency), and human impacts (e.g., clinician satisfaction, patient engagement). This multi-dimensional view helps justify digital investments not only in terms of savings, but in terms of mission-critical improvements to care and workforce resilience.

2. OKRs: The North Star for Digital Health

Let’s talk about Objectives and Key Results (OKRs), a familiar framework in tech-native organizations but still underutilized in many healthcare environments. At their best, OKRs provide visibility, alignment, and agility across digital health strategies, helping teams stay focused on what truly moves the needle.

As John Doerr, one of the earliest champions of OKRs and author of Measure What Matters, explains:

“Ideas are easy. Execution is everything.”

Doerr’s approach emphasizes that ambitious goals, paired with measurable key results, drive not only performance but cultural transformation. For healthcare systems undergoing digital evolution, OKRs can serve as the bridge between strategy and execution, and between mission and metrics.

In digital health, where transformation must happen amid constrained budgets, clinical pressures, and regulatory scrutiny, OKRs provide a compass. They allow teams to move beyond vague aspirations ("become more digital") toward structured accountability. And in a threat landscape where cybersecurity breaches are a persistent risk, often exacerbated by sprawling, redundant systems, OKRs help shine a spotlight on technology governance and risk mitigation.

Here is what some strong digital health OKRs might look like:

Objective: Elevate the Patient Experience

• KR1: ≥75% of patients check in via the digital front door.
• KR2: Improve Net Promoter Score by 7 points year over year.

Why it matters: These KRs ensure that digital tools are not just deployed, but they are also adopted. By tying OKRs to patient behavior and sentiment, teams stay focused on human-centered metrics, not just implementation milestones.

Objective: Shrink the Application Footprint

• KR1: Decommission 20 redundant apps by Q4.
• KR2: Cut annual maintenance spend by 12%.

Why it matters: Beyond cost reduction, application rationalization directly reduces the attack surface for cyber threats. Fewer apps mean fewer vulnerabilities, fewer endpoints to manage, and fewer potential data leakage paths. These OKRs link technology governance to both financial stewardship and risk mitigation.

So how do we make OKRs actionable?

• Integrate OKRs early: As part of annual roadmap development and portfolio reviews, define OKRs that clearly map to each strategic initiative.
• Use OKRs for executive accountability: Incorporate OKR reviews into quarterly business reviews (QBRs), enabling data-driven discussion around progress, blockers, and course corrections.
• Revise only when necessary: OKRs should be stable guides, not moving targets. Adjust Key Results only when core assumptions shift significantly (e.g., a new regulatory mandate, acquisition, or care delivery model).

When used consistently, OKRs become more than just metrics, they become the operational heartbeat of digital transformation.

3️. KPIs Still Matter—They Just Need Context

Let’s be clear: Key Performance Indicators (KPIs) are not obsolete. Far from it. They remain the essential dials on our operational dashboard. Metrics like system uptime, claims first-pass rate, and user adoption help us know whether the engine is running smoothly. But KPIs alone cannot tell us if we’re heading in the right direction.

"In healthcare, it's not just about collecting data; it's about aligning that data with our mission to improve patient outcomes and operational efficiency. Metrics should serve as a compass, guiding us toward sustainable innovation and equity in care delivery."  - Tracy Donegan Chief Information and Innovation Officer at MLK Community Healthcare

Where KPIs are about performance, OKRs are about purpose. KPIs monitor the current state; OKRs stretch us toward what is next.

For example:

·        A KPI might show 99.9% EHR uptime - critical, yes, but does that uptime translate into clinician satisfaction, workflow efficiency, or patient throughput?

·        A KPI might measure the number of training sessions delivered, but an OKR asks whether that training improved digital tool utilization or reduced clinical documentation time.

The magic happens when KPIs are nested inside OKRs, so they are contributing, not competing. This approach ensures that your operational measurements feed strategic intent. In other words, every monitored metric should answer: Are we advancing our goals for digital equity, efficiency, experience, and/or resilience?

This alignment is especially vital in healthcare, where there is often a tension between hitting benchmarks and driving transformation. For instance, a cybersecurity team may track KPIs like vulnerability scan completion rate, but an OKR could elevate that to ask: “Have we materially reduced risk exposure across high-priority systems by year-end?”

By embedding tactical KPIs within strategic OKRs, we empower teams to not just report progress but to create momentum.

Ultimately, the smartest digital health leaders know: KPIs track what is. OKRs guide what could be.

4️. Spotlight Metric Set: Ambient Listening Deployment

Let’s look at a real-world example. Ambient listening tools are AI-enabled solutions that help clinicians document visits with less manual effort. Now let's ground this in an actual deployment.

Here is how a digitally mature organization might assess the full-system value of AI-powered ambient listening, not just in dollars, but in clinician well-being, patient safety, and long-term financial agility:

Pro tip: pair your dashboards with a two-question pulse check:

1. “How mentally drained did you feel after today’s clinic?”
2. “Did technology help or hinder your day?”

Those qualitative insights bring life to the numbers and may surface friction that pure metrics can’t catch.

5️. The New Frontier: Equity, Data and AI Governance

As regulators sharpen their focus on AI transparency and equitable digital access, forward-leaning health systems must start measuring these domains now, not later. The reputational and operational risk of delay is simply too great.

According to the FDA’s AI/ML action plan and ONC's HTI-1 Final Rule, explainability, auditability, and bias mitigation are no longer optional, they are expected. Here are a few additional metrics that deserve a seat at the strategy table:

• Digital Equity Index – % of visits where patients used interpreter services + digital access parity.
• Data Liquidity Score – time to surface a longitudinal record across settings.
• AI Governance Compliance – % of AI models with documented bias + performance monitoring.

These aren’t just feel-good add-ons. They shape system trustworthiness, regulatory readiness, and long-term innovation potential.

6️. Our 7-Point Checklist

So where do we go from here? We have compiled a starter checklist for teams ready to evolve their measurement mindset:

1. Diagnose Redundancy: Run an annual application rationalization audit. Start with Gartner’s 30% benchmark.
2. Draft 2–3 Enterprise OKRs: Think transformation, not tasks.
3. Align KPIs to OKRs: Avoid metric overload and tie each KPI to a strategic objective.
4. Track Human-Centric Measures: Think beyond satisfaction to burnout, engagement, and usability.
5. Refresh Quarterly: Celebrate wins, retire stale metrics, and adapt when assumptions change.
6. Elevate the Conversation: Use metrics to connect technology back to patient care and organizational resilience.
7. Link to Strategic Pillars: Ensure every metric maps to an enterprise priority, which could be quality, growth, workforce, or equity. If it doesn’t, it probably should not be on the dashboard.

Measure what matters, retire what doesn’t, and watch the signal finally outshine the noise.

We hope this edition will spark conversation within your organization or team. Our work as digital leaders is to ensure we are not just collecting more data but collecting the right data to make better decisions, faster.

Next up in our series: we will dive deeper into how we can identify and leverage strategic partnerships to accelerate transformation to support our journey in healthcare technology innovation. Stay tuned!