From Data to AI with the ML Canvas
Download as PPTX, PDF1 like333 views
The document discusses the importance of having a substantial dataset for machine learning (ML) projects and outlines the AI project selection process. It emphasizes the need for technical and business diligence, defining success metrics, and collaboration between AI and IT teams. Additionally, it introduces the ML canvas as a framework to guide the workflow of ML projects and highlights pitfalls of unrealistic expectations based on limited success stories.
From Data to AI with the ML Canvas
- 1. From Data to AI with the ML Canvas Alexandra Petrus, Belgrade AI #1 - March, 2019
- 2. @AlxPetrus /in/alexandrapetrus/ alexandra@bucharest.ai Alexandra Petrus Products | Strategy | New Tech | AI
- 3. TERMINOLOGY AND WORKFLOWS SELECTING AN AI PROJECT THE ML CANVAS
- 4. Unless you have a huge dataset (‘Big Data’), it’s not worth attempting ML or data science projects on your problem. TRUE FALSE
- 5. Unless you have a huge dataset (‘Big Data’), it’s not worth attempting ML or data science projects on your problem. TRUE FALSE
- 6. “The future is independent of the past given the present.” 𝑃(𝑋𝑡|𝑋𝑡−1,…,𝑋0)=𝑃(𝑋𝑡|𝑋𝑡−1)P(Xt|Xt−1,…,X0)=P(Xt|Xt−1).
- 8. WHAT’S AN AI-FIRST COMPANY? Unified data warehouse2 Strategic data acquisition 1 New roles (MLE) & division of labor 4 Pervasive automation 3
- 9. WHAT’S AN AI-FIRST COMPANY? Any company + #<deep learning> AI-first company
- 10. Workflow of a ML project Collect data Train model *iterate many times until it’s good enough Deploy model *get back + Maintain/update model
- 11. Workflow of a ML project Collect data Train model *iterate many times until it’s good enough Deploy model *get back + Maintain/update model Workflow of a data science project Collect data Analyze data *iterate many times until good insights Suggest hypotheses/actions *deploy changes + re-analyze data periodically
- 12. SELECTING AN AI PROJECT Valuable for your business What AI can do
- 13. SELECTING AN AI PROJECT Valuable for your business What AI can do AI Experts Domain Experts
- 14. SELECTING AN AI PROJECT - due diligence TECHNICAL DILIGENCE BUSINESS DILIGENCE Can AI system meet desired performance? Lower costs How much data is needed? Increase revenue Engineering timeline Launch a new product or business
- 15. SELECTING AN AI PROJECT - due diligence TECHNICAL DILIGENCE BUSINESS DILIGENCE Can AI system meet desired performance? Lower costs How much data is needed? Increase revenue Engineering timeline Launch a new product or business BONUS: Brainstorming framework 1. Think automating tasks rather than automating jobs. 1. What are the main drivers of business value? 1. What are the main pain points in your business? *Having more data almost never hurts, but you can make progress w/o big data. Small datasets can be enough to get started on some projects.
- 16. Working with an AI Team ● Specify your acceptance criteria (eg: detect defects with 95% accuracy) ● Write good specifications by providing your AI team a dataset on which to measure performance (eg. test set ~1000 pics) ● Involve the AI Team with the IT Team in calibrating your data collection
- 17. AI Canvas was built by: Ajay Agrawal, Joshua Gans and Avi Goldfarb, Rotman School of Management, University of Toronto. ML CANVAS Prediction Judgement Action Outcome what do you need to know to make the decision? Ie. 95% accuracy how do you value different outcomes and errors? Ie: retrain, keep it actual. what are you trying to do? Ie. Yes/No or more nuanced what are your metrics for task success? Ie. action always leads to outcome Input Training Feedback What data do you need to run the predictive algorithm? Ie. what’s going on at the time a decision needs to be made What do you need to train the predictive algorithm? Ie. the richer and varied the dataset, the better. Have you used the outcomes to improve the algorithm? Ie. data in real situations, form a richer set of environments that training data.
- 18. “If you train a learning algorithm and every single picture you show it is a picture of a cat, then it thinks every single thing in the world is a cat, because it’s never seen anything that’s not a cat before. And I think in a similar way, if you show Executives a sequence of success stories but no failure stories, then it creates an impression that AI can do anything, and that’s just not true.” Andrew Ng - Google Brain CoFounder, ex-Baidu Chief AI Scientist, Coursera, Deeplearning.ai
Editor's Notes
- #7: Markov Chains help build the concept of decision making in a very simplistic way. It’s also one of the most important deterministic processes used in AI. How could we bring Markov Chains’ simplicity to business? Connecting the dots between data, AI, and value creation are essential for working on the right probleM.
- #8: Pic 3 main points: 1. 1950 alan turing propose the test for machine intelligence ; 2. 2011 Siri - intelligent virtual assistant making AI for everyone; 3. 2016 - Microsoft’s Tay chatbot builds its own identity on social media and goes rogue.
- #9: Internet-first companies had 3 main things in common: A/B testing; short iteration time and decision making being pushed down to engineers and PMs or other roles.
- #10: Replace DL with any AI tech
- #15: Ethical diligence: will it make society/humanity better?
- #18: Prediction - a machine can potentially tell you this Judgement - no prediction is 100% accurate so in order to determine the value of investing in better prediction you need to know the cost of different outcomes. It depends on the situation and requires human judgement. The value of human judgement can change the nature of the prediction machine. To give you a feel, Prediction within 5% accuracy - as soon as you put it in the wild, you have to retrain it, keep it actual, stick on the right path. These have to be considered Action - what is the action dependant on the predictions generated? It may be a simple yes/no or more nuanced. Outcome - an action always leads to an outcome. If you do not know what outcome you want, improvement is difficult, if not impossible Last 3 relate to data: Input - To generate a useful prediction, you need to know what is going on at the time a decision needs to be made. Training - to develop the prediction machine in the first place, you need to train a machine learning model. The richer and more varied that training data, the better your predictions will be out of the gate. If that data is not available, then you might have to deploy a mediocre prediction machine and wait for it to improve over time. Feedback - This is data that you collect when the prediction machine is operating in real situations. Feedback data is often generated from a richer set of environments than training data.