Cognitive Chasms - A Typology of GenAI Failure Failure Modes
0 likes246 views
Cognitive Chasms - A Typology of GenAI Failure Failure Modes
- 1. Cognitive Chasms: A Typology of GenAI Failure Modes Tathagat Varma Executive Fellow, Indian School of Business, Co ’25 “Cognitive Chasms: A Grounded Theory of GenAI Adoption©”
- 2. Disclaimer This presentation is based on my doctoral research titled “Cognitive Chasms: A Grounded Theory of GenAI Adoption”. You are free to refer to my work and use as appropriate, so long as you cite it ☺
- 3. Evolution of AI: Is third time a charm? https://www.americanscientist.org/article/intelligence-may-not-be-computable
- 4. The field of AI: continually evolving and rapidly growing… …also manufacturing serious hype since 1956! 1956 “Every aspect of learning or any other feature of intelligence can in principle be so precisely described that a machine can be made to simulate it.” – John McCarthy 1967 “Within a generation...the problem of creating 'artificial intelligence' will be substantially solved.” – Marvin Minsky 2016 “People should stop training radiologists...it is just completely obvious that within 5 years deep learning will do better than radiologists.” – Geoffrey Hinton 2019 “Next year, for sure, we'll have over one million robotaxis on the road” – Elon Musk
- 5. Hype continues to build up unabated…
- 6. Though the reality isn’t always as promised…
- 7. Challenges and failures galore…
- 8. And the results leave several questions unanswered…
- 9. So, what are the big questions? Why: Why does AI fail every so often? What: What does adoption mean? How: How do we measure success?
- 10. Theory of Cognitive Chasms: A Grounded Theory of GenAI Adoption”©
- 11. Cognitive Chasms (§7.2) GenAI Adoption Failure Modes Represent the potential discontinuities in adoption 4 distinct chasms represent exponential risks and rewards
- 12. Hype / Reality (H/R) Chasm • "Expectation s Gap" • Right technology must work Pilot / Production (P/P) Chasm • “Scaling Gap” • Pilots must be scalable Technology / Business (T/B) Chasm • “Value Gap” • Business need must exist Business / Social (B/S) Chasm • “Safety Gap” • Societal interests sacrosanct Cognitive Chasms (§7.2)
- 13. Hype/Reality (H/R) Chasm: The Expectation Gap (§7.2.1) Refers to the wide gap between what is sold as a marketing claim or a media hype that leads to heightened expectations about delivering magical performance to the enterprise vs. the on-ground reality in terms of (a) what is realistically possible here and now in the real-world vis-à-vis a research lab or a controlled experiment, (b) the time, effort and money it takes to implement it inside a firm, including the impact on rest of the organization and its overall business strategy that also includes its vendors, partners and customers, and (c) the actual results that are possible on a sustainable basis from a given implementation. When any of these key result areas are missing, the chasm appears, and depending on the extent of the gap and the simultaneous occurrences of gaps in other key result areas, it could lead to a much wider and deeper chasm.
- 14. Pilot/Production (P/P) Chasm: The Scaling Gap (§7.2.2) • Indicates a significant gap where pilot projects undertaken in a small, controlled environment fail at the full-scale production rollout stage. I could also view it as a “Lab to Application” chasm, where the research published in a journal might indicate a very advanced level of technology but translating it into a real-world application could entail significant challenges. • A “happy path” or an ideal use case implementation of a pilot might prove the power of GenAI technologies. Still, it might fail to uncover the commensurate changes required around organizational processes, structure, talent, and other factors critical to integrating GenAI technology into an institutional capability.
- 15. Technology/Business (T/B) Chasm: The Value Gap (§7.2.3) • Represents perhaps the most critical chasm at the enterprise-level adoption. While the technologists and data scientists might find the results of their latest AI experiments highly impressive and extremely valuable unless these translate into business-level performance, they might not allow the business to proceed with those AI adoption initiatives, and the executive leadership support might wither, and the firm might witness an internal AI winter. • At its core, the T/B Chasm represents a measurement gap. Rather than measuring the benefits, value, or ROI of technology, the firm continues to focus on measuring the value of technology itself and loses the focus of differentiating the “means” from the “ends.” However, firms must recognize that technical successes don’t automatically constitute business value.
- 16. Business/Societal(B/S) Chasm: The Safety Gap (§7.2.4) • Represents the challenges businesses must face in creating society-grade AI solutions. If the AI solutions are designed solely with the profit-motive without any consideration to societal expectations such as human needs, social values, and cultural norms, and address key concerns such as ethical and fair usage of cognitive capabilities, I foresee societal backlash in the form of increased rejection of such products and services wherever possible, and increased protection through legislations wherever required. • Essentially, the B/S Chasm captures the ethical gap between the goals of implementing a technology inside a firm and the needs and expectations of society from any technology or its applications. While regulations are catching up with the heightened dangers of GenAI technology, when firms sidestep these considerations, they risk creating solutions unsafe for society.
- 17. Common Properties (§7.2.6) Adoption is an exponential journey Chasms must be bridged Systematic approach is required
- 18. Multistage Scaling Strategy (§7.1) Stage 1: Explore Stage 2: Experiment Stage 3: Improve Stage 4: Transform Stage 5: Innovate
- 19. Multistage Scaling and Consequents: Stage 1: Explore Stage 2: Experiment Stage 3: Improve Stage 4: Transform Stage 5: Innovate
- 20. Multistage Scaling and Cognitive Chasms Hype / Reality (H/R) Chasm Pilot / Production (P/P) Chasm Technology / Business (T/B) Chasm Business / Societal (B/S) Chasm
- 21. Human Mindset Decision Mindset Purpose Mindset Cognitive Mindsets are sequential, accretive, and cumulative Learning Mindset
- 22. Bridging The Chasm: Multistage Scaling, Cognitive Chasms and Cognitive Mindsets Human Mindset Decision Mindset Purpose Mindset Learning Mindset
- 23. More details • Theory of Cognitive Chasms: A Grounded Theory of GenAI Adoption: My doctoral dissertation at Indian School of Business (ISB): https://eprints.exchange.isb.edu/id/eprint/2340/ • GenAI in Business: Invited talk for Emerging Leaders Program at ISB, May 2025:- https://www.slideshare.net/slideshow/generative-artificial-intelligence-genai-in-business/278708699 • Theory of Cognitive Chasms: Failure Modes of GenAI Adoption: Lecture for Executive MBA students at Walton School of Business, Apr 2025: - https://www.slideshare.net/slideshow/theory-of-cognitive-chasms-failure-modes-of-genai- adoption/278463456 • PhD Colloquium at Indian Institute of Management, Bangalore (IIMB), Dec 2024: https://www.youtube.com/live/UBmkl7Vpr0Q?si=lqw2JJUVhUI8cWL7 • GenAI Value Spectrum – delivering the “true value” from GenAI initiatives: Keynote at IEEE Bangalore Technology Conference (BTC) 2024: https://www.slideshare.net/slideshow/genai-value-spectrum-delivering-the-true-value-from-genai- initiatives/273734671 • The role of Cognitive Mindset in GenAI Adoption: Invited talk at Project Management Institute (PMI) Bangalore Chapter Footprints: https://www.slideshare.net/slideshow/the-role-of-cognitive-mindset-in-genai-adoption-pdf/274055306 • Managing Fast-Evolving GenAI Adoptions: Invited Keynote at Siemens Agile Conference 2024: https://www.slideshare.net/slideshow/managing-fast-evolving-genai-technology-adoptions/273470070
- 25. Recap
- 26. References • https://www.cdotrends.com/story/4558/decoding-ai-roi-paradox-why-85-projects-die- production • https://fortune.com/article/ceos-ai-initiatves-fraction-deliver-return-on-investment-roi-study/ • https://www.prnewswire.com/news-releases/ai-costs-climb-36-but-roi-still-unclear-report- finds-302463333.html • https://www.entrepreneur.com/business-news/klarna-ceo-reverses-course-by-hiring-more- humans-not-ai/491396 • https://www.pcmag.com/news/amid-backlash-duolingo-backtracks-on-plans-for-ai-pivot • https://tech.yahoo.com/ai/articles/next-assignment-babysitting-ai-081502817.html • https://www.nytimes.com/2025/05/21/business/media/chicago-sun-times-ai-reading-list.html • https://www.nytimes.com/2025/03/29/business/media/bloomberg-ai-summaries.html