Bringing clarity to analytics projects with decision modeling: a leading practice
2 likes1,092 views
The document emphasizes the importance of decision modeling in analytics projects, advocating for clearer collaboration between data scientists and business stakeholders to improve decision-making. It outlines practical recommendations for implementing decision modeling, such as examining stagnant projects and using the CRISP-DM methodology. The value derived from effective decision modeling includes enhanced clarity, operationalization of analytics, and faster project recovery.
Bringing clarity to analytics projects with decision modeling: a leading practice
- 1. BRINGING CLARITY TO ANALYTICS PROJECTS WITH DECISION MODELING A Leading Practice
- 2. • CEO of Decision Management Solutions • Works with clients to improve their business by identifying and modeling decisions, and applying analytic and business rule technology to automate & improve these decisions. • Spent 14 years championing Decision Management. • Consultant, author, speaker • IIA Faculty leader Copyright © 2017 IIA and Decision Management Solutions. All Rights Reserved JAMES TAYLOR 2
- 3. • The Big Ideas • The Value of Decision Modeling • Clarity of Purpose • Effective Collaboration • Operationalizing analytics • Practical Recommendations 3 AGENDA Copyright © 2017 IIA and Decision Management Solutions. All Rights Reserved
- 4. • Hard to connect business environment to analytics • Business users lack data science experience, vocabulary • Data scientists/analysts are not familiar with business problem • Decision Modeling: • Builds a shared understanding with business clients • Revives projects that have lost their purpose • Brings clarity to problems long thought difficult • Delivers value quickly Copyright © 2017 IIA and Decision Management Solutions. All Rights Reserved THE BIG IDEA: DECISION MODELING 4
- 5. What kind of analytic/data science group are you in? A. Individual contributor B. Dedicated to a single business function/business team C. Supporting multiple business functions/business teams D. Center of Excellence supporting multiple analytic teams POLL #1 Copyright © 2017 IIA and Decision Management Solutions. All Rights Reserved 5
- 6. Data Business Understanding Data Understanding Modeling Data Preparation Evaluation Deployment THEORY: PROJECTS USE CRISP-DM • Central Data Science Team • Provide diverse skillset • Support multiple projects across operations • Use CRISP-DM • Cross-Industry Standard Process for Data Mining • Most widely used approach for managing analytic and data science efforts Copyright © 2017 IIA and Decision Management Solutions. All Rights Reserved 6
- 7. IN PRACTICE: NOT EXACTLY • Business Objectives not Business Understanding • A lack of clarity • Rework not reevaluation • A lack of collaboration • IT handoff not deployment • No operationalization Data Data Understanding Modeling Data Preparation EvaluationIT Business Objectives A lack of clarity A lack of collaboration No Operationalization Copyright © 2017 IIA and Decision Management Solutions. All Rights Reserved 7
- 8. THE VALUE OF DECISION MODELING: CLARITY OF PURPOSE 8
- 9. • Value of data science comes from improved decisions • Any data science project must influence decisions to add value DATA SCIENCE IMPROVES DECISIONS Copyright © 2017 IIA and Decision Management Solutions. All Rights Reserved 9
- 10. WHICH DECISION MUST BE IMPROVED? • Define a Question • Define Allowed Answers • This • Focuses on discrimination • Resolves disagreements • Reveals inconsistency • Clarifies different assumptions • Approve Claim? • In what way should this claim be processed once complete details have been received? • Auto-approve • Fast-track • Regular process • Reject • Fraud investigation unit Copyright © 2017 IIA and Decision Management Solutions. All Rights Reserved 10
- 11. Copyright © 2017 IIA and Decision Management Solutions. All Rights Reserved MODEL DECISION REQUIREMENTS Source: DecisionsFirst Modeler 11 Specific Analytic Decision Analytic Models Overall decision to be improved Other Relevant Experience Regulations and Policies Data Sub-decisions
- 12. • Costs increasing • Organizations that drove cost increases were not accountable - could not allocate costs • Decision Model for Allocation Decision • Identified data needed for decision • Identified how that data was derived • Revealed (much simpler) original data sources • Allocation could be managed Copyright © 2017 IIA and Decision Management Solutions. All Rights Reserved CASE: COST ALLOCATION 12
- 13. Copyright © 2017 IIA and Decision Management Solutions. All Rights Reserved OTHER EXAMPLE RESULTS • Life insurance provider • Decision model of life underwriting decision • Identified that a planned medical risk prediction would not be usable • Project was redirected • No wasted effort 13
- 14. THE VALUE OF DECISION MODELING: EFFECTIVE COLLABORATION 14
- 15. How much of an issue is collaboration for you and your team? A. We don’t really collaborate with anyone B. We collaborate with business partners well, IT not so much C. We collaborate with IT well, business partners not so much D. We collaborate well with business partners and IT – no problems Copyright © 2017 IIA and Decision Management Solutions. All Rights Reserved 15 POLL #2
- 16. Three Legs 1. Business 2. IT 3. Analytics Copyright © 2017 IIA and Decision Management Solutions. All Rights Reserved 16 COLLABORATION
- 17. • Automated lead qualification • Need to predict size also for prioritization • Decision Model • Revealed that analytic would modify estimate • Showed how prediction would be used • Focused analytic effort on business value not just accuracy Copyright © 2017 IIA and Decision Management Solutions. All Rights Reserved CASE: AUTOMATED LEAD SIZE PREDICTION 17
- 18. Copyright © 2017 IIA and Decision Management Solutions. All Rights Reserved OTHER EXAMPLE RESULTS • Healthcare insurance provider • Developed a decision model for healthcare claims adjudication • The model showed where and how to use a fraud analytic • Increased usage • Improved results 18
- 19. THE VALUE OF DECISION MODELING: OPERATIONALIZING ANALYTICS 19
- 20. DECISIONS HAVE CONTEXT • Decisions are… • Made during Business Processes • Triggered by Events • Made by Systems • They impact KPIs, Objectives • Various organizational roles Copyright © 2017 IIA and Decision Management Solutions. All Rights Reserved 20 System Decision Process Event Performance Indicator Objective Organizational Unit
- 21. OPERATIONALIZING DATA SCIENCE Decision Traditional New External Internal Analytically enrich it Make Consistent, Precise, Real-Time Decisions Across All Channels Learn Continuously Start with customer data Add Policies, Regulations Best Practices and Preferences Copyright © 2017 IIA and Decision Management Solutions. All Rights Reserved 21
- 22. • Predict if customer will renew service contract • Why is that the prediction? • Here are the predictive variables • I’m still not sure. Maybe if you just added this…. • Actually: • Needed to prioritize selling opportunities • Churn prediction just part of this • Stay focused Copyright © 2017 IIA and Decision Management Solutions. All Rights Reserved CASE: SERVICE CONTRACT RENEWAL 22
- 23. Copyright © 2017 IIA and Decision Management Solutions. All Rights Reserved OTHER EXAMPLE RESULTS • Manufacturing company • Identified predictions of quality risk • Could change task assignments • Developed decision models around product quality • Found opportunity to analytically focus daily activities of quality team 23
- 24. How much of a challenge is operationalizing your analytics? A. We struggle to operationalize any analytics B. We operationalize analytics too slowly C. We can only operationalize simple analytics D. We effectively operationalize everything up to real-time embedded analytics Copyright © 2017 IIA and Decision Management Solutions. All Rights Reserved 24 POLL #3
- 26. Review • Examine portfolio for stagnated analytics projects • Identify functions that need but don’t have analytics Learn and adopt • CRISP-DM as an approach • Decision modeling as a technique Approach • Hold one-day decision modeling sessions with stakeholders to kick off • Work with people affected by decisions across business, IT, analytics Copyright © 2017 IIA and Decision Management Solutions. All Rights Reserved 26 IMPLEMENTATION CHECKLIST
- 27. LESSONS LEARNED • Natural impatience to rush to the data • Not just a way to rescue projects • Diverse teams are a powerful asset • Many projects don’t know how data science can help • Bring in decision modeling at start of the project • Teach data scientists to use as standard approach • Decision modeling focuses these teams • Use a facilitated one-day decision modeling session Copyright © 2017 IIA and Decision Management Solutions. All Rights Reserved 27
- 28. 1. Performance Measures 2. Identify Decisions 3. Clearly Define Decisions 4. Specify Business Context 5. Define Data Requirements 6. Define Knowledge Requirements 7. Define Decision Requirements 8. Iterate and Refine Copyright © 2017 IIA and Decision Management Solutions. All Rights Reserved DECISION MODELING APPROACH 28
- 30. Copyright © 2017 IIA and Decision Management Solutions. All Rights Reserved TAKEAWAYS 30