Combining Machine Learning with Physical Computing - June 2023
Download as PPTX, PDF0 likes181 views
Machine learning and physical computing can be combined for educational purposes. Several platforms and tools were described that allow students to collect sensor data, train machine learning models, and deploy models to physical devices like microcontrollers to perform tasks. Examples included using micro:bit, Arduino, Raspberry Pi, and smartphones for collecting data from sensors to classify images, sounds, and motions using pre-trained and custom models. Open-source platforms and curricula were provided to help educators incorporate machine learning and physical computing into their lessons.
Combining Machine Learning with Physical Computing - June 2023
- 1. COMBINING MACHINE LEARNING WITH PHYSICAL COMPUTING Hal Speed 2023 June 16 https://www.slideshare.net/Hal_Speed
- 3. Breaking News WEF Future of Jobs weforum.org/reports page 39 https://teachai.org/
- 4. Five Big Ideas in AI ● Framework to guide the development of standards and learning materials ● For use by governmental organizations, curricula developers, CSTA, educators, etc.
- 6. What is Machine Learning? ● Machine Learning is a subfield of Artificial Intelligence focused on developing algorithms that learn to solve problems by analyzing data for patterns. ● Deep Learning is a type of Machine Learning that leverages Neural Networks and Big Data Artificial Intelligence Deep Learning Machine Learning Source: EdX tinyML course
- 7. What is Machine Learning? Computer Computer Human Programming Machine Learning Input Data (ingredients) Human Program (recipe) Input Data (ingredients) Output (cake) Desired Output (cake) Program (recipe) or Model
- 8. What is Machine Learning? Computer Computer Human Programming Machine Learning Input Data (2,2) Human Program (x+y) Input Data (2,2) Output (4) Desired Output (4) Program (x+y, x*y, x2, y2, ?) or Model
- 11. Once We Have A Trained Model Trained Model Unlabeled or Unclassified Data Inferences or Classifications (guesses with a degree of confidence)
- 12. How Do ML Models “See” and “Hear”? spectrogram
- 13. Large Language Models ● Examples ○ GPT-3 and GPT-4 from OpenAI ○ BERT, LaMDA, PaLM 2 from Google ○ LLaMA from Meta ● Uses massively large datasets to train a language model using deep learning ○ GPT-3 was trained on about 45TB of text data ○ For comparison, the English version of Wikipedia is estimated to be around 50GB ○ Wikipedia makes up 3% of GPT-3 training data ● Size of neural network parameters increasing at an exponential rate 13
- 14. https://code.org/ai https://tinymledu.org/4K12 AI Resource Lists https://raise.mit.edu/resources.html https://www.actua.ca/ai/ https://ai4k12.org/ https://aiforteachers.org/
- 15. Machine Learning Workflow Collect Data Preprocess Data Design a Model Train a Model Evaluate Optimize Convert Model Deploy Model Make Inferences Data Engineering or Data Science Model Engineering Model Deployment Source: EdX tinyML and Google TensorFlow Usage
- 16. Jupyter Notebook/Colab Edge Impulse Seeed Studio Codecraft ML Machine & PlushPal Google Teachable Machine Microsoft Lobe ML for Kids Scratch + Teachable Machine smartphone Wio Terminal Pico4ML micro:bit others Arduino IDE Microsoft MakeCode MicroPython/ CircuitPython Code.org Building a tinyML Pipeline for K-12 Collect Data Preprocess Data Design a Model Train a Model Evaluate Optimize Convert Model Deploy Model Make Inferences Nano 33
- 17. ML Machine for micro:bit ● Easy to use gesture model ● Web based, no login ● See the data in real time ● Collect, train, test ● Have another micro:bit respond to model predictions https://ml-machine.org/
- 19. EdX Tiny Machine Learning https://www.edx.org/professional-certificate/harvardx-tiny-machine-learning
- 20. Arduino Tiny Machine Learning Kit ● Supported hardware kit for the EdX courses ● Arduino Nano 33 BLE Sense ○ IMU (motion sensor) ○ microphone ● Plus OV7675 camera and breakout board
- 21. ● No-code ML platform ● Broad hardware support including: ○ Smartphones ○ Arduino ○ micro:bit ○ Pico4ML ○ Wio Terminal ○ …and many more ● Coursera courses ● micro:bit V2 tutorials ○ Voice activated ○ Dance move detector https://www.edgeimpulse.com/
- 23. CodeCraft ● The power of Edge Impulse with the simplicity to Scratch ● Learning resources ● Get Started with TinyML ebook ● Hackster project by Marcelo Rovai https://ide.tinkergen.com/
- 24. Other Hardware
- 25. Pico4ML ● Includes camera, IMU (motion), and microphone ● Bluetooth optional ● Three pre-trained models and magic wand tutorial available https://www.arducam.com/pico4ml-an-rp2040-based-platform-for-tiny-machine-learning/
- 26. BrainCraft HAT + RPi with Lobe Three different step-by-step tutorials for image detection: ● Getting started ● Rock, paper, scissors game ● Package detector https://learn.adafruit.com/machine-learning-101-lobe-braincraft
- 27. Microsoft Farm Beats for Students The easy-to-use FarmBeats kit includes ● preconfigured Microsoft Azure cloud services ● A Raspberry Pi with soil moisture, light, ambient temperature, and humidity sensors to collect data. ● The data is then visualized in an online dashboard that provides insights to help students. Partnership Future Farmers of America and Microsoft are working together to create activity guides and resources to help chapters get started with using the technology. https://aka.ms/farmbeatsforstudents
- 29. Calypso for Cozmo ● A robot intelligence framework that Incorporates multiple AI technologies: ○ Computer vision; face recognition ○ Speech recognition and generation ○ Landmark-based navigation ○ Path planning ○ Object manipulation ● Rule-based pattern matching language inspired by Microsoft’s Kodu Game Lab ● Teaches computational thinking: “Laws of Calypso”, idioms, etc. world map perception rules speech recognition https://calypso.software/
- 30. DeepRacer ● Intended for professionals ● Support for advanced sensors https://aws.amazon.com/deepracer/
- 31. OpenBot ● Turning smartphone into robots ● Software stack for Android ● Real-time autonomous navigation https://www.openbot.org/
- 32. HUSKEYLENS ● AI vision sensor ● Built-in training algorithms ● Connect to micro:bit and use MakeCode or Mind+ ● User guide and tutorials wiki https://www.dfrobot.com/product-1922.html
- 33. KOI Camera Module ● Works with or without micro:bit ● Progammable with MakeCode, KittenBlock, and MicroPython ● Quick start guide and tutorials https://www.kittenbot.cc/products/kittenbot-koi-artificaial-intelligence-module
- 34. Experiments with TF Lite for Microcontrollers ● Well-documented projects with Tensorflow Lite for Microcontrollers ● Includes code & instructions ● Projects use the Arduino Nano 33 BLE Sense https://experiments.withgoogle.com/collection/tfliteformicrocontrollers
- 35. AIY - With Google https://aiyprojects.withgoogle.com/
- 36. EdgeBadge ● Step-by-step tutorials for gesture and speech detection https://learn.adafruit.com/tensorflow-lite-for-edgebadge-kit-quickstart
- 37. ● Introduction to ML: Image Classification ● Personal Image Classifier: PICaboo ● Personal Audio Classifier ● Voice Calculator Tutorial ● Therapist Bot Tutorial ● Awesome Dancing with AI Tutorial ● Facemesh Filter Camera ● Rock Paper Scissors Tutorial https://appinventor.mit.edu/explore/ai-with-mit-app-inventor
- 41. Machine Learning for Kids https://machinelearningforkids.co.uk
- 42. http://cognimates.me Cognimates offers AI extensions for Scratch, such as: ● speech recognition ● sentiment analysis ● visual pattern detection ● robot control
- 44. Other Scratch-based Editors w/ML Extensions https://scratch.techpark.jp/ https://stretch3.github.io/ https://mblock.makeblock.com/ https://www.kittenbot.cc/pages/software
- 47. ● Data collection and analysis tool https://codap.concord.org/
- 48. Code.org Resources AI and Machine Learning Module ● ~ 5 week curriculum ● Standalone or optional unit in CS Discoveries https://code.org/ai AI for Oceans Classifier How AI Works Videos AI and Ethics
- 49. High School Curriculum Unit http://www.exploringcs.org/for-teachers-districts/artificial-intelligence Intended to be an alternative unit to either unit 5 or 6 of the ECS course
- 51. AI4All: Online Learning http://ai-4-all.org/open-learning AI4ALL Open Learning empowers high school teachers of all subjects to bring AI education to their classrooms through a free, adaptable AI curriculum and teacher resources. Interdisciplinary, Approachable AI Curriculum
- 52. ReadyAI Resources https://www.readyai.org/ AI Picture Books AI Teaching & Learning Kits Self-paced Courses Lesson Plans Unplugged Lessons Teacher Training
- 54. Other Books for Young Students https://tinkertoddlers.com/
- 56. https://www.waicy.org/ World AI Competition for Youth ● For students ages 6-18 ● 2023 details coming soon