"e-Patient Dave" - Patient Empowerment evangelist. #PatientsUseAI. No pitches please.
"AI in Healthcare: Digging in the Wrong Spots" [Overseas friends: how much of this is true in your country?] At the HL7 Annual Meeting I just saw what's *easily* the most useful talk about AI in healthcare I've ever seen. It was by John Zimmerman of Carnegie Mellon and focused (FAR more than most of LinkedIn) on which uses actually GET anywhere, which uses CAN actually achieve a useful outcome. That focus is diametrically opposed to the wet-dream culture of seeking unicorns - and PATIENTS need y'all to achieve something USEFUL. Bottom line is to stop looking at the hardest, most amazing things, even though they're fascinating. While listening intently I could only capture a small fraction of points to share here but I hope to get much more from him. A few examples, not prioritized, shown in the photo composite I cobbled together: 1. Despite our imagination that AI will figure out everything from EMR access, IT'S HUGELY DIFFICULT, because the freaking data is heavily skewed, e.g. to whatever is billable. So, although sepsis is a huge killer, IT'S OFTEN NOT NOTED in ICU charts, because it's not billable(!!). 2. He cited Cassie Kozyrkov who has a great YouTube course on "making friends with machine learning - she says to find a really practical application for AI, you need to think of it as an island full of drunk people :-) Eager and friendly and usually pretty good but REALLY likely to make mistakes. So think: what kind of tasks can you give them? (I'm again reminded of the "trust but verify" rule from our paper at Division of Clinical Informatics DCI at BIDMC) 3. A taxonomy of 40 *commercially successful* AI products, segmented by : - How perfect does it need to be, for success? (Y axis) - How perfect *is* AI at the task? (X axis) LOOK: 25 of the 40 are in the left column, where the AI is moderately good at it but not brilliant! So think: what applications can you find where it'd be valuable to be wicked fast and PRETTY smart but not perfect? He also had a photo of a grizzly bear and cited the 2022 study that said 6% of people think they could win a fight with a grizzly ... and he said he's pretty sure all of them are AI product managers :) :) I'm going to take this thinking into the rest of the meeting week and beyond: what CAN we do with genAI, practically, without seeking perfection? Grace Cordovano, PhD, BCPA, Grace Vinton, Liz Salmi, Danny Sands, MD, MPH, Brian Ahier, Jan Oldenburg, Kim Whittemore, Anna McCollister, Amy Price MS, MA, MS, DPhil, James Cummings, Daniel Kraft, MD, Matthew Holt
Brian Ahier
1w
Progress in healthcare AI isn’t about dazzling demos or chasing the hardest unsolved problems—it’s about steady, incremental usefulness. The challenges John Zimmerman highlighted around data quality and skewed incentives are absolutely real. If our training data is incomplete or distorted by billing priorities rather than clinical reality, then no model—no matter how advanced—will generate outcomes we can trust at the bedside. Where I see hope is in the hard, sometimes unglamorous work of interoperability. The ability to connect data across systems, normalize it, and ensure it faithfully represents a patient’s journey is what creates the foundation for AI to deliver meaningful value. Without that plumbing, we’re only ever polishing fragments of the truth. Once that foundation is in place, though, the kind of “pretty smart but wicked fast” applications become transformative—clinical decision support, care coordination, even administrative efficiency can all improve dramatically without requiring AI to be perfect.
Satish Kumar
1w
Dave deBronkart Sir, can you please give the link for the paper mentioned at point no.2 in the post (Lines from the post - "I'm again reminded of the "trust but verify" rule from our paper at Division of Clinical Informatics DCI at BIDMC" )
Scott K.
1w
1) EMR data is mostly garbage. It’s skewed as you mentioned, but also skewed in that it doesn’t give nearly enough health information instead of problem list history. (When’s the last time your doctor asked you what’s your diet or how active you are?) Also, I’ve been on more than a half a dozen plus studies in which the EMR chart had at least one verifable error in it, which means you’re just training on erroneous data. And claims data is worse. 2) most commercial “AI” applications are using LLMs and these are great for textual processing but not as useful in health care settings, where we’re dealing with a lot of time series data as well as correlations back to evidence based research. Machine learning is great for that stuff, and there’s a lot going on here, BUT setting up these kinds of models and interpretating their results takes highly skilled teams of people, not just some devs slapping an LLM onto a product.
Great post! It's refreshing to see a conversation that cuts past the hype. The sepsis example is such a perfect illustration of how skewed data undermines even the smartest AI ideas. We keep imagining breakthroughs at the hardest edge cases, but the real step forward might be much less glamorous: building systems where “fast and good enough” is still a huge upgrade for patients and clinicians. That’s the kind of progress that actually sticks.
Dave, do you have more on that taxonomy? Very interesting
AI is probably smarter than now duggested. If sepsis is not in the data it will diagnose it from complex other data, such as antibiotic prescribed, symptoms, beddays, age, previous history etc.
Excellent. Refreshing as always Dave. Simon
D. D. Sharma
1w
A top-down analysis of healthcare cost data also supports Dave deBronkart's comments. Sharing some cost data I have published earlier (estimates): Largest US Healthcare cost item - Healthcare Delivery: 83% of whole $5T pie Largest healthcare delivery cost item: Non-Physicians - 61% of whole Largest healthcare delivery staffing: Non-Physicians - 94% (of 14M) (Admin roles are 16% vs Physicians 6%) Estimated Savings from GOFAI/ML - $417B Estimated savings from Gen AI: $150B Estimated savings by Open Data Access and Collaboration Platforms: $340B Savings from preventing healthcare delivery waste: $275B Savings from preventing medication/prescription errors: $297B Savings from Second Opinions: $195B References: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4973958 https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4976293
This would be an excellent workshop and way of defining AI usefulness across stakeholders! Could be useful in the expertise categories and where there is considerable variance.. Our paper that covers trust but verify and the coproduction with end users Rozenblit L, Price A, Solomonides A, Joseph AL, Srivastava G, Labkoff S, deBronkart D, Singh R, Dattani K, Lopez-Gonzalez M, Barr PJ, Koski E, Lin B, Cheung E, Weiner MG, Williams T, Thuy Bui TT, Quintana Y. Towards a Multi-Stakeholder process for developing responsible AI governance in consumer health. Int J Med Inform. 2025 Mar;195:105713. doi: 10.1016/j.ijmedinf.2024.105713. Epub 2024 Nov 22. PMID: 39642592. https://pubmed.ncbi.nlm.nih.gov/39642592/
"e-Patient Dave" - Patient Empowerment evangelist. #PatientsUseAI. No pitches please.
1wcc Virginia Lorenzi, Maria D. Moen, Lisa Nelson, Sarah Tonn MPH, CHTS-PW