AI in Testing - Hugo Pinto
Download as PPTX, PDF1 like1,046 views
The document discusses the role of artificial intelligence (AI) in software testing and deployment, highlighting its potential to improve efficiency and automation while addressing the challenges involved. It contrasts various companies' approaches to testing, such as Google, Facebook, Amazon, and Spotify, pointing to the ongoing necessity of human testers despite automation capabilities. The text also raises concerns regarding biases and the future implications of AI in coding and testing processes.
AI in Testing - Hugo Pinto
- 2. AI IN TESTINGCode, test, deploy, repeat…or maybe not! London, 2017 @hugomcpinto
- 5. Example 2 – AI or not AI @hugomcpinto
- 7. @hugomcpinto WHY NOW - WHAT IS DRIVING THE AI FRENZY?
- 8. what you do around you now emotions preferences @hugomcpinto
- 9. air quality data traffic data behavioral data energy consumption & production data @hugomcpinto
- 10. What is Artificial Intelligence? SENSE Perceive the world by acquiring and processing images, sounds and speech. COMPREHEND Analyze and understand the information collected by adding meaning and insights. ACT Take action in the physical world based on comprehension and understanding. LEARN Improve performance (quality, consistency, and accuracy) based on real world experiences. Expert Systems Computer Vision Inference Engines Machine Learning RPA Deep Learning Sensor Processing Knowledge Repre- sentation Mini Bots Emotion Recognition Gesture Recognition Ontologies Neural Networks BiometricsNLP Video Analytics @hugomcpinto
- 11. @hugomcpinto
- 12. Software development life cycle @hugomcpinto
- 13. It needs to be fast @hugomcpinto
- 14. It needs to be scalable @hugomcpinto
- 15. It needs to be cheaper @hugomcpinto 30%QA costs
- 16. @hugomcpinto More people is no longer enough
- 17. @hugomcpinto New services, new partners
- 18. Continuous multi-channel deployment @hugomcpinto
- 20. Key opportunities: Testing ALL code Detecting anomalies without testing Creating new tests automatically Self-healing RPA @hugomcpinto 80%Possible automation
- 21. @hugomcpinto Google 1. Internal testing 2. Crowdtesting platform 3. "Dogfooding” 4. Beta testing Facebook Tends to use "canary" releases An incremental rollout strategy to test fixes, updates, and new features in production Amazon Development > testing (the company famously deploys software every 11.6 seconds!) Making software easy to deploy Easy to roll back in case of a failure. Spotify Just like programming, testing is considered a creative process. It cannot be (fully) automated. Spotify heavily relies on dedicated testers that explore and evaluate the product, instead of trying to automate as much as possible. Different Strategies
- 22. What about testing AI?
- 23. failure = death
- 28. Coding, testing, learning on device Internet of things Quantum computing @hugomcpinto
- 29. @hugomcpinto
Editor's Notes
- #4: Let’s begin with a quiz to test your knowledge of AI
- #5: Not AI – just incremental processing capability
- #6: Various components of AI Speech to text, text to speech, Natural Language processing, entity resolution, to name a few
- #7: Defnitely AI Image recognition and entity resolution - currently being applied to live video and with machine learning algorithms that pick up on any “anomalies” for instance in train stations
- #8: And Why Now? Decreasing Cost of Storage - Over the past 30 years, the cost per GB of hard disk storage has halved every 14 months, from $3,488,630 in 1980 to $0.03 in 2015. Exponential Growth of Data - IDC predicts 50-fold growth from the beginning of 2010 to the end of 2020 Rapid Rise in AI Investment - AI startups in the US alone have increased 20-fold in the past four years. Virtually Unlimited Computing Power - Public cloud computing has reached almost $70 billion in 2015 worldwide All of this leads to a declining error rate in visual recognition
- #9: This explosion of personal information – structured and non-structured
- #10: Along with an explosion of contextual data of the world around us and the services that power our lives
- #11: We now have the capability to begin putting together components that combined provide machines and systems to perform human-like activities…even better than humans. Seeing, Hearing, Speaking, reasoning, finding patterns, understanding text
- #12: But this is bringing challenges to companies - they need an AI strategy to understand how they can start building a consistent approach to AI
- #13: And on another side we have the evolution of the software development cycle which is getting more and more automated Low code or no code platforms AI that codes Distributed agile teams Devops / Continuous delivery It’s being seriously challenged by how key companies do things, as well as the fact all large corporations want (and need) to become software companies
- #14: Back in San Francisco, Tesla sensors started getting damaged on the bottom of the car. They ran some data science to discover the damage was happening when the vehicles went over speedbumps They discovered speedbumps in San Francisco that were 2 inches taller They code a geo-fencing trigger around those speedbumps that rose the suspension of all teslas when they approached that speedbump And dropped it back to normal levels when they left the geo-fenced area I wonder if they spent months doing testing of this? Deployed with devops 2 weeks after the initial problem was identified.
- #15: Facebook needs to enable multiple teams globally to commit and still ensure the performance of the platform, no matter where it’s being developed
- #16: QA costs represent in average 30% of development costs
- #17: If you deploy twice as much code you can’t double the size of the team Throwing bodies at the problem is no longer a solution And talent is scarce
- #18: For instance in mobile apps, there tends to be a large amount of 3rd parties committing services to your environment When you install a new key functionality how do you run QA
- #19: The transition from traditional waterfall models is pushing and putting pressure on standard testing capabilities, since the amount of code commits is increasing rapidly, and with the adoption of Agile and DevOps it’s only going to keep accelerating and growing – and the number of channels and operating systems as well
- #24: How do you test AI to ascertain the social impact of AI, an algorithm, a dialogue?
- #25: How can you avoid the subtle cognitive biases that AI has begun to surface – ethnic and gender discrimination
- #26: How do you test AI to ascertain the social impact of AI, an algorithm, a dialogue?
- #27: How can you prevent a single entity from becoming unevenly equipped to crush the competition
- #28: How can you prevent a single entity from becoming unevenly equipped to crush the competition
- #30: A marketplace for different testing solutions and libraries why does everyone need to go through the same steps why should you be limited to the testing knowledge you have physically available