Challenges of Executing AI
1 like331 views
The document discusses the challenges and best practices in executing AI and machine learning projects, highlighting that nearly 50% of such projects fail to reach production. It emphasizes the importance of data quality, proper planning, and clear stakeholder roles to improve project success rates. Best practices include understanding data features, ensuring reliable training datasets, and maintaining transparency throughout the project lifecycle.
Challenges of Executing AI
- 1. N-U Sigma U2 Analytics Lab web: www.businessanalyticsr.com email: umesh@mytechnospeak.com Ph: +1 408 757 0093 Umesh Rao Hodeghatta, Ph.D Chief Data Scientist Challenges of Executing AI & Machine Learning Projects 10/18/2020
- 2. Outline AI Project Success Reports AI and ML Project Execution Challenges AI Project Vs Software Projects Best Practices For Executing AI Projects Q & A 2
- 3. Brief Introduction about me I have more than 20 years of experience I have worked for AT&T Bell Labs, Cisco Systems, McAfee, Wipro I have provided AI and ML solutions to some of the leading retail companies, Human Resource company and healthcare company. I have authored Books titled “Business Analytics Using R and “The InfoSec Handbook: An Information Security”, published by Springer Apress. I have taught at Walden University, Kent State University, Xavier University (XIMB). 3
- 4. AI Project Statistics AI project failure rates near 50%, More than 53% terminates at proof of concept level and does not make it to production Gartner report says that nearly 80% of the analytics projects are not delivering any business value 4
- 5. AI Project Execution Challenges Business Expectations Data Tools and Technology Resources Results and Outcome Project Planning and the Process 5
- 7. Leverage AI Deployment Timeline Quick deployment Smooth Integration Cybersecurity Value Add % development efforts Risks Agile 7 “What percentage of business decisions are we making with help from AI?”
- 8. Data is The KEY! 8
- 9. AI/ML Project Depends on DATA 9
- 10. Tools and Technology Simple Analytics and Data Visualization Data Plumbing Data cleaning Data cleaning utilities Data Preparation Reading pdf files and search certain key words? Data scaling utilities Feature selection AI and ML Predictive Analytics Image recognition Recommendation Engine Sentiment Analysis 10
- 11. Resources Important roles (not in the order of importance) Data Engineer Data Analyst Machine Learning Data Scientist Software Programer/Coder Programs the solution as per the direction of the Data Scientist Data Analyst analyses the data using data analysis tools and techniques like MSBI, Tableau, etc. and finds out patterns and what the data broadly suggests Data Plumbing Engineer: Capturing the data, collating the data, cleaning up data, etc. 11
- 12. Output Results Wrong interpretation of the results • Misguided Decisions • Few false positives or few false negatives can seriously undermine the use of the model in spite of high accuracy Selecting Metrics • Different algorithms have different but relevant metrics Overfit models may not be relied upon as they throw up different results on the unseen / new data. Trust and Ethics 12
- 13. AI Project Vs Software Project 13
- 14. Typical Software Project Process 1. Initiation 4. Deployment 2. Planning 3. Execution 1. Requirements 2. Hardware/Software 3. Aligning Business 1. Resources 2. Scheduling 1. Development 2. Testing 3. Deployment 3. Risks Planning 1. Monitoring, Tracking 2. Reassessing/Calibration 1. Beta Testing 2. Closure 14
- 15. AI Machine Learning Project Process ASSESS DATA MODEL Deploy Science Initiation • Requirements • Aligning to Business • Data Availability • Quality Of Data • Data vis-à-vis Business Requirement • Data Collection/Collation & Cleaning • Descriptive • Predictive • Prescriptive • NLP, Deep Learning • Ph.Ds and Coders • Experiments • Measure & Validate • Vary Parameters • Re-validate • Errors • Deploy • Verify • Calibrate 15
- 16. E X E C U T I O N O F A I A N D M L P R O J E C T S Best Practices
- 17. AI is Science!! Clarity on the data Label Data Distinguish supervised or unsupervised ML Composition of the classes/categories Do Not Directly jump into the model building Coding without science Understand the features and their contribution to the model Business Objectives, Requirements, Features and Data 17
- 18. Developing AI Solutions Sufficient and reliable training data Sample size or Population? Training data cover all the relevant and possible results? Is the training data in tune with the basic underlying fundamentals of the field under study? Validating Algorithms validations carried out using sound principles of validation? Results available? Results are sufficiently challenged to bring out the weaknesses if any? Transparent process of generating the model? The process worked in tune with the requirements to get the requisite confidence in the results? No Secrets 18
- 19. Project Planning Agile Stories Scrum Meetings Sprints Weekly discussions Weekly updates Code reviews or process Demos Discussion with Business Guide Business and Team 19
- 20. Summary W.r.t. Data science project, you don’t know if it’s going to work Data science requires an experimental process that allows for uncertainty Businesses and all the resources involved need to clearly understand the roles and responsibilities of various stakeholders Transparency is critical for the success 20
- 21. References 21 Hodeghatta, U. R., & Nayak, U. (2016). Business analytics using R-a practical approach. Apress. https://www.wsj.com/articles/ai-project-failure-rates-near-50-but-it-doesnt- have-to-be-that-way-say-experts-11596810601 https://www.forbes.com/sites/gilpress/2019/07/19/this-week-in-ai-stats-up-to-50- failure-rate-in-25-of-enterprises-deploying-ai/#34ea003272ce; https://www.forbes.com/sites/gilpress/2020/01/13/ai-stats-news-only-146-of-firms- have-deployed-ai-capabilities-in-production/#4da0ee526500
- 22. THANK YOU E M A I L : U M E S H @ B U S I N E S S A N A L Y T I C S R . C O M : U M E S H @ M Y T E C H N O S P E A K . C O M W E B : H T T P : / / W W W . M Y T E C H N O S P E A K . C O M P H : + 1 4 0 8 7 5 7 0 0 9 3 Q & A