Big Data Berlin – Automating Decisions is the Next Frontier for Big Data
4 likes4,196 views
The document discusses the role of big data in automating decision-making across various sectors such as marketing and finance. It contrasts traditional human decision-making processes with automated machine learning processes that can enhance decision quality by minimizing cognitive biases. It highlights the challenges of demand prediction in supply chains, using the ground beef dilemma as an example to illustrate the complexities of inventory management.
Big Data Berlin – Automating Decisions is the Next Frontier for Big Data
- 1. LARS TRIELOFF – BLUE YONDER – I’M @TRIELOFF ON TWITTER AUTOMATING DECISIONS – THE NEXT FRONTIER FOR BIG DATA
- 2. WHAT NOBODY TELLS YOU ABOUT THE FRONTIER
- 3. WHO IS USING BIG DATA TODAY
- 4. WHERE BIG DATA IS USED Effective Use Marketing Finance Everyone Else SOURCE: OWN, POTENTIALLY BIASED OBSERVATIONS
- 5. WHERE BIG DATA IS USED Potential Use Marketing Finance Everyone Else SOURCE: OWN, POTENTIALLY BIASED OBSERVATIONS
- 6. Better Decisions Faster Data More Data THREE APPROACHES
- 7. DIGITAL MARKETING’S APPROACH: MORE DATA
- 8. FINANCE’S APPROACH FASTER DATA
- 9. BUT WHAT ABOUT BETTER DECISIONS?
- 10. Self-Actualization Love/Belonging Safety Physiological Esteem
- 11. Storage Collection Prediction Analysis Decision
- 12. Storage Collection Prediction Analysis Decision Self-Actualization Love/Belonging Safety Physiological Esteem
- 14. “… all others bring data.” ! — W. EDWARDS DEMING
- 15. HOW DATA-DRIVEN DECISIONS SHOULD WORK COMPUTER COLLECTS COMPUTER STORES HUMAN ANALYZES HUMAN PREDICTS HUMAN DECIDES
- 16. HOW DATA-DRIVEN DECISIONS REALLY WORK COMPUTER COLLECTS COMPUTER STORES HUMAN ANALYZES COMMUNICATION BREAKDOWN HUMAN DECIDES
- 17. COMMUNICATION BREAKDOWN Communication Breakdown, It's always the same, I'm having a nervous breakdown, Drive me insane! — LED ZEPPELIN • Drill-down analysis … misunderstood or distorted • Metrics dashboards … contradictory and confusing • Monthly reports … ignored after two iterations • In-house analyst teams … overworked and powerless
- 18. HOW DATA-DRIVEN DECISIONS REALLY WORK HTTP://DILBERT.COM/STRIPS/COMIC/2007-05-16/
- 19. HOW DECISIONS REALLY SHOULD WORK COMPUTER COLLECTS COMPUTER STORES COMPUTER ANALYZES COMPUTER PREDICTS COMPUTER DECIDES
- 20. 99.9% of all business decisions can be automated
- 21. HOW DECISIONS ARE MADE Human Decisions Business Rules No Decision, Decision-by-default
- 22. BUSINESS RULES = PROGRAMMING • Business rules are like programs – written by non-programmers • Business rules can be contradictory, incomplete, and complex beyond comprehension • Business rules have no built-in feedback mechanism: “It is the rule, because it is the rule”
- 23. HUMAN DECISION MAKING • • System 1: Fast, automatic, frequent, emotional, stereotypic, subconscious • • System 2: Slow, effortful, infrequent, logical, calculating, conscious DANIEL KAHNEMANN, THINKING FAST AND SLOW
- 24. HOW TO DECIDE FAST FREQUENT DECISION MAKING MEANS FAST DECISION MAKING, MEANS USING HEURISTICS OR COGNITIVE BIASES Anchoring effect IKEA effect Over-justification effect Bandwagon effect Confirmation bias Substitution Availability heuristic Texas Sharpshooter Fallacy Gambler’s fallacy Illusory correlation Rhyme as reason effect Hindsight bias Zero-risk bias Framing effect Sunk cost fallacy Overconfidence Outcome bias Inattentional Blindness Benjamin Franklin effect Anecdotal evidence Negativity bias Loss aversion Backfire effect
- 25. HOW COMPUTERS DECIDE FAST MACHINE LEARNING OFFERS AN ALTERNATIVE TO HUMAN COGNITIVE BIASES AND CAN BE MADE FAST THROUGH BIG DATA K-Means Clustering Markov Chain Monte Carlo Support Vector Machines Naive Bayes Affinity Propagation Decision Trees Nearest Neighbors Least Angle Regression Logistic Regression Spectral clustering Restricted Bolzmann Machines
- 26. Better decisions through predictive applications
- 27. HOW PREDICTIVE APPLICATIONS WORK COLLECT & STORE ANALYZE CORRELATIONS BUILD DECISION MODEL DECIDE & TEST OPTIMIZE
- 28. WHY TEST? “Correlation doesn’t imply causation, but it does waggle its eyebrows suggestively and gesture furtively while mouthing ‘look over there’” –RANDALL MUNROE
- 29. STORY TIME (Not safe for vegetarians)
- 30. THE GROUND BEEF DILEMMA
- 31. THE GROUND BEEF DILEMMA Yesterday Today Tomorrow Next Delivery Next Day In Stock Demand
- 32. THE GROUND BEEF DILEMMA • Order too much and you will have to throw meat away when it goes bad. You lose money and cows die in vain • Order too little and you won’t serve all your potential customers. You lose money and customers stay hungry.
- 33. CHALLENGE #1 ACCURATELY PREDICT DEMAND
- 34. CHALLENGE #2 AUTOMATE REPLENISHMENT COLLECT STOCK AND SALES PREDICT DEMAND TRADE OFF COSTS CREATE ORDERS IN ERP SYSTEM OPTIMIZE & REPEAT