Deciphering AI: Human Expertise in the Age of Evolving AI
0 likes53 views
1) The document discusses how human expertise remains important in the age of evolving AI, especially as AI systems transition from narrow, rule-based approaches to more general and autonomous capabilities like deep learning and generative AI. 2) It provides examples of how human expertise can guide different AI approaches, from feature engineering for machine learning to providing feedback to help validate or invalidate systems. 3) The document also covers challenges around the business use of advanced AI, including how to ensure systems are explainable, accountable, and developed responsibly according to principles like fairness, privacy and reliability.
Deciphering AI: Human Expertise in the Age of Evolving AI
- 1. Australia’s National Science Agency Deciphering AI: Human Expertise in the Age of Evolving AI Dr/Prof Liming Zhu Research Director, CSIRO’ Data61 • Expert, OECD.AI – AI Risks and Accountability • Expert, ISO/SC42/WG3 – AI Trustworthiness • Member, National AI Centre (NAIC) Think Tank All pencil drawings in this presentation are created by AI
- 2. • Australian government • “An engineered system that generates predictive outputs such as content, forecasts, recommendations or decisions for a given set of human-defined objectives or parameters without explicit programming. AI systems are designed to operate with varying levels of automation.” • EU AI Act • “Software that is developed with one or more of the techniques and approaches listed in Annex I and can, for a given set of human-defined objectives, generate outputs such as content, predictions, recommendations, or decisions influencing the environments they interact with.” AI Definition – Examples 2 |
- 3. • Purposes & Requirements • AI governance/regulation: under/over-inclusiveness, flexibility, practicality… • Business transformation: applicability, measurability, clarity… – Executive Education: skills, culture, governance, ethics... • R&D, Public understanding… • Definition types • Capabilities: human-like; reasoning, learning, perception, communication.. • Application: generate contents, recommendations, decisions… • Approaches: rule/logic-based, (un)supervised machine learning… • … AI Definitions – Fit for Purpose 3 |
- 4. • Example - Fraud detection • Data->Features: transaction time/amount/frequency, account age, geolocation… • Rule/logic-based • data, feature, data -> rules, feedback, + AI helps manage/derive complex rules • Machine learning (learned model: Y=weightsi*Xi+ b & human-designed learning algorithm) • Supervised: labelled data, features, AI learns rules, feedback • Unsupervised: no labelled data, features, AI learns rules, feedback Approaches & Role of Human Expertise 4 |
- 5. • Deep learning/neural networks (billions of weights/features) • No feature engineering, ”dumb” algorithm + big data, emergent/alien capabilities • Non-domain experts improve learning efficiency; domain expert feedback Approaches & Role of Human Expertise 5 | Encoding human expertise -> Learning human-understandable expertise from human expertise and data -> Invalidating human expertise Explaining alien intelligence in human-understandable terms
- 6. Deep Neural Network -> ChatGPT 6 | Reinforcement Learning AI learns to make decisions by interacting with an environment to maximize cumulative reward through trial & error. https://www.understandingai.org/p/large-language-models-explained-with https://huyenchip.com/2023/05/02/rlhf.html
- 7. Foundation Models – Generality is Free? Problem-specific training + generalization --> general capability training + adaptation Value of unique data & human expertise in training vs predicting? Bommasani, R. et.al , 2022. On the Opportunities and Risks of Foundation Models. 7 |
- 8. Generative AI •Text •Image/video •Code/Scripts •Data Predictive Diagnostic Generative AI – Generate Anything? Prescriptive 8 |
- 9. Business Transformation with AI 9 | • General Capability • human resources or tools/functionality • Ease of Access • Cost-benefit analysis/plan ->low-cost exp. • Changing nature/role of human expertise • Explanation & understanding • Changing org structure & collaboration mode • Reverse Conway’s law
- 10. 10 | https://a16z.com/2023/06/20/emerging-architectures-for-llm-applications/ Example: LLM App Architecture Zero-gradient Infrastructure
- 11. Responsible AI – Regulation & Ethics 11 | Australia’s AI Ethics Principles (developed by Data61) 1) Human, societal and environmental wellbeing 2) Human-centred values 3) Fairness 4) Privacy protection and security 5) Reliability and safety 6) Transparency and explainability 7) Contestability 8) Accountability Australia’s Responsible AI Network (RAIN) Minister Husic: “I'm determined that we go further than ethics principles. I want Australia to become the world leader in responsible AI.”
- 12. Best Practices for Responsible (Generative) AI 12 | Lu, Q., Zhu, L., Xu, X., Xing, Z., Whittle, J., 2023. Towards Responsible AI in the Era of ChatGPT: A Reference Architecture for Designing Foundation Model-based AI Systems. http://arxiv.org/abs/2304.11090 CSIRO Responsible AI (RAI) Pattern Catalogue • RAI-by-Design Products • Development Processes • Governance https://research.csiro.au/ss/science/projects/responsible-ai-pattern-catalogue/
- 13. Summary & Questions • Business transformation with this new wave of AI • General capabilities/”interns” vs specific tools • Low-cost experimentation vs problem-driven planning • Value of unique data & human knowledge • Universities Accord • Skills: What human expertise/knowledge? • VET->HighEd->Biz: “the revenge of the generalist” • Access/Investment: FM/GenAI • Governance/Accountability: Responsible AI More info & Contact https://research.csiro.au/ss/ Liming.Zhu@data61.csiro.au Brendan.Omalley@data61.csiro.au Coming out late 2023 For the latest, follow me on Twitter: @limingz LinkedIn: Liming Zhu 13 |