Intro to Learning Tech and Learning Organisations
1 like148 views
The document outlines the introduction of AI in enterprises, focusing on learning technologies, AI algorithms, and their applications in various industries. It discusses the challenges of AI adoption, emphasizing the importance of strategy, investment, and explainability for successful implementation. Additionally, it highlights the future of learning organizations and the evolving role of AI in enhancing user experience and decision-making processes.
Intro to Learning Tech and Learning Organisations
- 1. INTRO TO AI APPLYING LEARNING TECHNOLOGIES TO THE ENTERPRISE LUIS F GONZALEZ GUEST LECTURER MD ELEMENTAI ASIA PACIFIC
- 2. SLIDO – Polling the class: 1. In which area AI going to impact your life the most in the next 20 years? 2.What topic of AI are you most interested to learn? 3. What worries you the most about AI in society?
- 3. Agenda Intro to Learning Algorithms (AI) The Learning Enterprise XAI – Human Experience and FEAT Group Discussions
- 4. What are Learning Technologies? The theory and development of computer systems able to perform tasks that traditionally required human intelligence like learning. Artificial Intelligence is somewhat inaccurate as systems are not intelligent alas, they learn. Video must see to understand deep learning https://youtu.be/5tvmMX8r_OM
- 5. Let’s simplify the progression Input Input Input Hand-designed rules Output Learned simple features Learned complex features Mapping from features Output Hand-designed features Mapping from features Output Rules-based systems 1960’s Classic machine learning 2000’s Deep learning 2020’s Input Random Probability Distribution Output Stochastic systems 1980’s
- 6. Types of Learning Possible
- 8. What are Learning Technologies Components? Sensing the world Perception Learning from every interaction Communication Optimizing to specific outcomes Decision making Understanding concepts & relations Reasoning Taking actions in the world to achieve goals Interaction Computer Vision Natural Language Understanding & Generation Forecasting and Operations Research Knowledge Graphs and Representations Reinforcement Learning Answer questions about a scene Determine if a growth is cancerous or not Infer what happened to characters in a story Drive on city streets and highways Identify objects in a scene
- 9. Perception: Vision and its applications
- 12. Applications today For NLP/G/U Abstract Summarization helps us understand your Context:
- 13. Decision: Time Series and its applications
- 14. Interaction: Reinforcement Learning Reward Functions
- 15. Agenda Intro to Learning Algorithms (AI) The Learning Enterprise XAI – Human Experience and FEAT Group Discussions
- 16. Normal Components of AI in Enterprises 5 Object Detection & Monitoring Robust detection, counting and tracking of objects and people in a wide variety of environments, enabling valuable workflows in many real-world situations. Image Classification Video Alteration Object Tracking and Counting 6 Optimization AI-powered optimization boosts the efficiency of business processes and tasks, maximizing lift and ROI compared to traditional optimization methods. Routing Disruption management Re-optimization 7 Explainability Making your AI explainable to users helps drive adoption and lowers the barrier of entry for users. Technical Explanations Bias Evaluation and Tracking Sample-based Explanations And more including... ● Recommender systems ● Assignment with constraints ● Association rule learning ● Human-AI interaction ● Image segmentation ● Image clustering ● Routing with constraints ● etc.. 4 Time-Series Forecasting Best-in-class AI-powered forecasting provides better accuracy and can deliver lift for a wide range of forecasting scenarios. Hybrid Forecasting Models Statistical Forecasting Deep-learning Forecasting Models 3 Anomaly Detection Detect anomalies on objects in natural environments and in various types of data, allowing for near real-time reaction. Visual Anomaly Detection Event-based Anomaly Detection Anomalies in Forecasting Data 1 Text Extraction & Analysis Accelerate the extraction of insights from multiple forms of text, catching signals that are easily overlooked by humans. Text Summarization Sentiment Analysis Text Classification 2 Optical Character Recognition Instantly transcribe text from natural environments or digital documents, reducing manual work and enabling automation. Documents Handwritten Notes Live Scenes and Video
- 18. Learn user’s behaviour based on voice commands and can adjust settings automatically in subsequent interactions Target user with personalized products and services ads based on their demographic profile, search history, visited sites, liked social media posts, etc. Improve efficiency/quality in servicing customers by using AI assistants (e.g., chatbots, robo-greeters in bank branches and cardless ATM machines via facial recognition) Detect suspicious/fraudulent activities in network and/or transactions using predictive analytics Recommend music or videos based on user’s historical consumption and preferences Provide best driving routes, ETA, and/or match drivers with riders based on historical and real- time data Smart Home Devices Media & Entertainment Navigation and Transportation E-commerce & Targeted Ads Customer Service Assistants Security and Fraud Detection Examples - Applications by Industry
- 19. Element AI | Copyright © 2019 | Strictly Confidential Typical patterns of AI adoption in enterprise & associated challenges Major challenges in enterprise AI adoption stem from lack of well-designed AI strategy and enablement support REPRESENTATIVE Strategy & Enablement Experimentation Production deployment No effort in AI strategy/roadmap Algorithm-focused tests in silo Lack of basic capabilities to run enterprise AI solutions Bottom-up use case collection & arbitrary prioritization AI roadmap with traditional methods, part of digital strategy Various experimentation themes (technical + non-technical) Lack of investment in integrated AI architecture for scale Lack of investment in enterprise deployment infrastructure Algorithm-focused tests in silo • No buy-in from leadership • No strategy & capabilities definition • Limited use cases • No end-to-end consideration for real application environment • Prone to resource overuse w/o result • Failure in deployment of tested models (technical & business reasons) • Short-sighted, low hanging fruit only - difficulties in proving value of AI • Misinformed decision on AI strategy & capabilities acquisition • Lack of architectural consideration • Misled investment priorities in experimentation • Difficulty in proving ROI of AI solution • Lack of productivity in architecture • High exposure in risk, bias • Difficulty in proving ROI of AI solution • No enterprise-grade infrastructure • High exposure in risk, bias • No experimentation strategy • Missing experimentation other than algorithm performance Pattern 1. Experiments w/o plan Pattern 2. Bottom-up planning Pattern 3. Incapable of ‘scalable AI adoption’
- 20. Personas in the Learning Organisation AI Strategy Strategy Lead Learning Processes Owner Data & Learning Strategist Intelligent Business Strategy AI Build &Deploy Data & Learnings Engineer Applied Research Engineer AI Developer AI Discovery Business Analyst Data & learnings Analyst Applied Research Science HCI Designer Data Stewart AI Integration Learning Algorithms Tester ML/DL Trainer AI Operations Learning Algorithms Application Support Performance Monitor Learnings Stewart AI Management Procurement Lead Learning Tech Governance Lead AI Legal Specialist Ecosystem Relations AI Infrastructure. Platform Lead Microservices & Learnings Architect
- 21. What is the most important skillsets in an Learning (AI First)Enterprise?
- 22. Learning Technologies Process Framework AI Strategy AI StrategyFormulation AI Governance AI Research & InnovationAI Architecture & Design Business Demand &Impact Mgmt AI Use-case portfolio planning andscheduling AI Build &Deploy SolutionPlanning Solution Build Solution Test AI Discovery AI ProblemDefinition Data Analysis Create Test Learn AI Integration AI ChangeIntroduction AI ChangeAdoption AI ChangeManagement AI ChangeExecution AI Operations AI Service Operations AI Continuous Learning AI Upgrades AI Budget AI ContractsManagement AI Management Procurement AI Governance and RiskManagement Model Performance Management AI TalentManagement AI Infrastructure
- 23. Element AI | Copyright © 2020 | Strictly Confidential
- 24. Learning Organization in a page RaaS Ready made products Bespoke solutions LEARNING TECHNOLOGIES ENGINEERING & INTEGRATION LT MODEL TRAINING AND DEVELOPMENT Build Deploy Run Improve ECOSYSTEM & PARTNERSHIPS ACADEMIC RELATIONS Research Led Innovation & Though Leadership AI PLATFORMS LEARNING TECHNOLOGIES RESEARCH INNOVATION FUNDAMENTAL RESEARCH Publications Innovation Fundamentals APPLIED RESEARCH SCIENCE Model Applications Industry Innovations Startup Incubation & M&A LT VENTURES Consultation and Advisement REGULATION LEARNING TECHNOLOGIES GOVERNANCE FRAMEWORK LEARNING TECHNOLOGY SUPPORT LT Integration and Geographical support LT Sustainment & Continuous Improvement LT SOLUTIONS ENABLEMENT Use Case on New Revenue Stream Gen Use Case on Cost Reduction LEARNING TECHNOLOGIES OPERATIONS & CONTINUOUS IMPROVEMENT Learning Technologies Outsourcing Solutions Agile, Rapid prototyping and business model adaptation LT Ideation and Design thinking LT METHODOLOGY OUTSORCING OPERATIONS OS Hypothesize Design Collect Assess Data Model Prototype Deploy Operate Maintain Improve DATA SCIENCE & EGINEERING DATA MODELLING & MANAGEMENT CO-Creation and Solution Synergies CO-DEVELOPMENT Talent Pipeline & Education RESOURCES Hypothesize HUMAN RESEARCH COGNITIVE, BEHAVIOURAL AND CULTURE Use Case on Improvement ML DEV, MULTICLOUD, EDGE & DATA PLATFORMS Research Engineering Advisory/Enablement
- 25. Structured Data Unstructured Data Semi-Structured Data Description • Formatted data set with defined fields such as (numeric, currency, alphabetic, name, date, address, etc.,) • Generally, stored in relational database format, tables, excel, CSV, etc., • Often managed using Structured Query Language (SQL) • Sources of information that cannot easily be classified into relevant data fields • Generally, sources of information are classified between Textual vs. Non-Textual data • Textual: PDFs, Books, etc., • Non-Textual: Images, Audio, Video • A cross between Structured and Unstructured Data sources • Generally, tags or other types of markers are used to identify specified elements within the information sources, but lacks a rigid structure Examples • CRM purchase history • Historical stock price data • Cellphone Metadata • Financial Reports • Text from a PDF • Satellite Images • Music audio file • Security camera video file • Database of images with tagged data fields such as data, creator, location, keywords describing image Data Realities
- 26. 23 AI Infrastructure – Flexibility introduced to Enterprise Architecture High level considerations ILLUSTRATIVE
- 27. Element AI | Copyright © 2019 | Strictly Confidential Plan for enterprise-grade AI architecture & processes AI adoption at the core of enterprises requires ‘industrialized’ architecture/processes for productivity and sustainability Use case design Data collection & preprocessing Model building /selection Testing & deployment Finding insights & communications 45~60 % 20~25 % 10% 3% Integrated process management 10% Effort Distribution in Typical Data Science Projects (% of time) • Manual data handling • Lack of active-learning in data handling tools • Lack of toolchain integration to automate data-model-deployment procedures, hindering productive ML application development • Mere collection of open-source frameworks without designing reusable, integrated, and scalable ML application development architecture • No use case repository • Siloed use case management REPRESENTATIVE • Manual, ad-hoc collection/sharing of learnings • Lack of architectural, process embedding for AI risk & explainability • No application framework for model reuse, cross-learning Hypothesize Design Collect Assess Data Model Prototype Deploy Operate Maintain Improve Hypothesize Research Engineering Advisory/Enablement
- 28. The Learning Organisation Kickstart
- 29. Build a 3 minute pitch on how you propose the company you work in build a Learning Technologies capability (COE, SideCar, M&A, ETC.)
- 30. Agenda Intro to Learning Algorithms (AI) The Learning Enterprise XAI – Human Experience and FEAT Group Discussions
- 31. Agenda Intro to Learning Algorithms (AI) The Learning Enterprise XAI – Human Experience / Fairness and Transparency Group Discussions
- 32. Fairness Ethics Accountability Transparency FEAT principles were created to guide better AI development Justifiability Accuracy & Bias Internal & External Explainability Interpretability
- 34. Fairness Ethics Accountability Transparency Sources: Bill Rankin, “Radical Cartography” http://www.radicalcartography.net/index.html?chicagodots; David Ingold and Spencer, “Amazon Doesn’t Consider the Race of Its Customers. Should It?”; Calvano, E, G Calzolari, V Denicolò and S Pastorello (2018a), “Artificial intelligence, algorithmic pricing and collusion,” CEPR Discussion Paper 13405; Bias Unintended Consequences Systemic Issues 1 2 3
- 35. XAI for Trust ● How do I know the model is doing something reasonable? ● High-stakes decisions ● Counter-intuitive outputs End users who are responsible for high-stakes decisions will not blindly trust an algorithm without what they believe to be strong evidence that its recommendations or outputs are accurate. One powerful way to build this trust is if the algorithm provides an explanation along with its output. Strong evidence on the accuracy of the output for high- stake decisions. 1. Need to know why model is recommending this potentially risky/costly decision? 1. Why model output is counter-intuitive?
- 36. XAI for Compliance Right to an explanation if receiving an adverse decision on a loan or insurance policy
- 37. XAI for Bias Ensure that people are not being unfairly excluded Ensure you are not missing a key part of the market when identifying potential customers
- 38. XAI for Generalization Validate the reasoning continuously matches the business objectives, prevent concept drift (objectives, data, how close you are test time from your training distribution)
- 39. Future of the Learning Organizations 39 TIME VALUE Current Tech Org Future Learning Organizations New business model Incumbent’s business model Focus on Adoption (advertisement) Exploitation of Data Humans as Research Subjects Harvesting of existing Data "Context Collapse" Results > Matrix Focus on Value Co-creation (Economic Communities) Empowerment of Humans Predictions with you and Transfer/Active Learning F.E.A.T. = Trust Results > Star Trek Learning Tech Enablement Prediction > Personalisation > Humanisation
- 40. Examples of Fairness High- frequency trading Accuracy needed Trust needed Auditing of financials and controls Diagnostic medicine Marketing spend optimization Document search for litigation Maintenance recommendati on Examples of Explainability Explainability is about communication with a human and so has much in common with design and education. In particular there is no established metric to evaluate the quality of an explanation, so tracking progress and comparing methods is hard. In some cases there may not be a metric. The field is very immature. There are not yet standard approaches, benchmarks, or indeed definitions of the field. Interpretability #1 This is why I refused the application “I refused the application because variable X_2 was too low and this explained 60% of the variance considering that X_4 was high and X_7 account for more than 15% of the decision” #2 This is why I believe the image is a truck Many published methods have not actually been tested with human subjects to validate how math functions translate to user outcomes Source: https://arxiv.org/abs/1907.07374
- 41. User roles spectrum ● End-users: input-output relationship ● Developers: inner working of the model ● In practice, the desired explanations are a combination of these two extremes Application domain ● Some industries might already have specific requirements, e.g. banking and insurance Explainable AI needs to be designed with the user role and the application domain in mind
- 42. Explainability is about communication with a human and so has much in common with design and education. In particular there is no established metric to evaluate the quality of an explanation, so tracking progress and comparing methods is hard. In some cases there may not be a metric. The field is very immature. There are not yet standard approaches, benchmarks, or indeed definitions of the field. An example metric How much does the explanation improve a user’s ability to predict future model outputs? Why is explainability hard?
- 43. Going beyond model debugging What kind of action can we drive? Actionable XAI Explanations that drive actions “I need to understand why a model made a decision so I can complete a regulatory audit report” Human-AI Interaction for XAI AI can interact and also learn from human insight “I want to easily see the reasoning so I can correct errors and give better feedback on what I want the output to be” End to End XAI Moving away explaining a model to explaining a business process. “If I am assembling a car, understanding every piece separately is not sufficient. I must be able to audit the assembly process.” 1 2 3 44
- 44. Case Study at a Aisin Seiki by explaining model errors CASE STUDY The manufacturer needed to explain accuracy, confidence in predictions, and in- and out-of distribution operating range for plant operators and customers to trust the results. We implemented explainability components on top of the client’s existing model to identify negative (erroneous) influences in the training data and re-calibrate its confidence estimate to minimize misclassifications. Early results showed that false positives could be reduced by a factor of 10X while maintaining nearly 0 false negatives with the more explainable workflow. A leading auto parts manufacturer wanted to automate quality inspection with machine vision. A multi-XAI component workflow helped identify and reduce errors. 45 Decision uncertainty Feature attribution Sample-based explanation
- 45. We believe the way forward for explainability is an end-to-end approach that requires combining many techniques ELEMENT AI PERSPECTIVE Data collection What data points matter most? Where might there be gaps or issues in representativeness? Model selection Can complex models be constrained or simplified to be more interpretable? EXPLAINABILITY IN THE AI APPLICATION DEVELOPMENT LIFECYCLE Training & evaluation What is the model actually learning? How well will it perform for different groups and scenarios? Interface design What do different user personas understand? What information do they need to trust it? Most open-source tools and platforms focus narrowly on explaining models (such as which features are important) Performance and trust are affected by the whole application development lifecycle. Like supply chain quality assurance, AI applications need to trace the impact of errors forward to affected clients and backwards to the source.
- 46. Learning Technologies Future is XAI enabled Level 1 Efficiency Level 2 Personalization Level 3. Reasoning Level 4 Exploration Definition Learning Technologies facilitates specific functions with systems and devices, making user interactions more efficient and effective AI uses pattern learning to recognize, optimize and personalize functions in order to improve and simplify interactions for users AI uses causality learning to understand the cause of certain patterns and behaviors, this information is used to predict and promote positive outcomes for users AI uses experimental learning to continuously improve, by forming and testing hypotheses it uncovers new inferences, seamlessly adding value to users’ lives and enabling a deeper affinity Presence in our lives Familiar: Systems and devices that utilize AI are appearing in user’s everyday lives Common: AI is optimizing most devices at the edge and most systems through the cloud Universal: AI is everywhere and interconnected for the benefit of all devices and systems Foundational AI forms a core component of the infrastructure for all devices and systems in society which share and learn collectively Human Experience Task Centric: Can execute specific commands within specific parameters to achieve a specific task. Humans need an understanding of how models operate in relation to the task to achieve proficiency in their use and manage their limitations. Goal-oriented: Multiple actions Works out various options for achieving a given goal and presents them to the user for selection or is pre- programmed to efficiently meet the desired goal. User needs an understanding of the tactical outcomes to better perform and trust the models interacting. Mission-focused Long-term actions Understands users and its environment in order to predict, recommend and execute solutions to assigned missions. User needs expertise in the strategic direction and involves in the coordination of efforts to best use these models. Purpose-driven Exploratory actions Using local context and external sources of knowledge, it balances users’ competing needs and interests and is able to take creative approaches to influence user behaviors, whilst in service of the user’s higher purpose. User helps orchestrate, prioritize and participate at a peer to peer level in a huma and machine teams.
- 47. Appendix: Follow UpMaterial 1.Data Trusts New tool ForGovernance. https://hello.elementai.com/rs/024-OAQ-547/images/Data_Trusts_EN_201914.pdf 2. Applying AI Building and Organization for scalingAI: https://techwireasia.com/2021/02/why-scaling-ai-is-an-enterprise-grade-challenge/ 4. Building the AI-Powered Organization Harvard Business Review https://hbr.org/2019/07/building-the-ai-powered-organization 3.Enterprise Software The case for Services in Enterprise Software Startups. https://a16z.com/2018/03/12/services-enterprise-software- products-startups/ 5. Michael I Jordan Lex Fridman Podcast. https://youtu.be/EYIKy_FM9x0