Edge AI Smart Manufacturing - Defect Detection and Beyond (GTC 2019)
6 likes1,658 views
The document discusses Inventec's innovations in AI-driven smart manufacturing, particularly focusing on laptop testing robots and automated optical inspection (AOI) technology. It emphasizes the benefits of edge AI for enhancing privacy, reliability, and network efficiency in production processes. The collaboration between Inventec and Skywatch aims to streamline manufacturing by utilizing AI for process automation and predictive analysis, ultimately improving testing efficiency and reducing labor costs.
Edge AI Smart Manufacturing - Defect Detection and Beyond (GTC 2019)
- 1. Trista Chen, Chief Scientist of Machine Learning of Inventec Wei-Chao Chen, Co-founder of Skywatch & Head of Inventec AI Center {chen.trista, chen.wei-chao} @ inventec.com S9617
- 2. About Inventec • Public company (Since 1975; Taipei, Taiwan) • Tier 1 electronics manufacturer • Annual revenue USD $16B+ (2018) • Factories in Taiwan, Shanghai and Chongqing GTC 2019 Personal Computers Laptops Enterprise Computers Servers Solar Energy Smart Devices Medical Devices
- 3. AI for Smart Manufacturing • Process Automation – Automatic optical inspection – Production scheduling – Automatic testing • Predictive Analysis – Order forecast – Production / yield prediction – Predictive maintenance GTC 2019
- 4. Smart Manufacturing Automatic optical inspection Order forecast Production scheduling Test automation … Future Product AIOT / Smart City applications Medical devices Robotics Computation Platform …. Inventec AI Center
- 5. Why Edge AI? • Privacy • Reliability • Network Efficiency GTC 2019 JETSON TX2 MODULE •NVIDIA Pascal™ Architecture GPU •2 Denver 64-bit CPUs + Quad-Core A57 Complex •8GB L128 bit DDR4 Memory •32GB eMMC 5.1 Flash Storage •Connectivity to 802.11ac Wi-Fi and Bluetooth- Enabled Devices •10/100/1000BASE-T Ethernet
- 6. GTC 2019 Laptop Testing Robot A0I Re-inspection Laptop Surface AOI Re-inspectionAOI for SMT lines and PCBs
- 7. GTC 2019 Laptop Testing Robot A0I Re-inspection Laptop Surface AOI Re-inspectionAOI for SMT lines and PCBs
- 8. 1. Laptop Testing Robot Background • Millions of laptops manufactured every month – Similar basic design, many different configurations – Production tests are mostly automatic GTC 2019
- 9. 1. Laptop Testing Robot Background • Product approval is labor intensive – Takes weeks to certify for mass production – ~1M test assets – Hundreds of people running the tests GTC 2019
- 10. 1. Laptop Testing Robot Motivation (least -> most important) • Labor cost and management – Skilled and trustworthy testers are difficult to find • Time-to-market – Regression tests take weeks / months to complete • Increase confidence – The management confidence level on manual testing is only around 70% GTC 2019
- 11. 0. 3 3 0 0 . / 0 0 0 0 3 GTC 2019 By: Jimmy Ou, Wei-Chao Chen, et al. of Skywatch Collaboration with Joseph Shi, Jack Hung & Inventec QA Engineers 1. Laptop Testing Robot
- 12. GTC 2019 1. Laptop Testing Robot Pneumatic actuators Mechanical / electrical keyboard and mouse
- 13. GTC 2019 1. Laptop Testing Robot Camera with computer vision capability Asideoflaptop Industrial Camera
- 14. 1. Laptop Testing Robot • Computer Vision (OpenCV) – Perspective correction – Edge detection & denoising – Template matching • Tesseract OCR – LSTM-based NN OCR engine GTC 2019
- 15. 1. Laptop Testing Robot GTC 2019 Test subject Controller 4X real-time demo
- 16. 1. Laptop Testing Robot Future Opportunities • Automatic testing script generation – from human-readable to machine-readable test cases • Scheduling optimization – Reshuffle test levels and orders to speed up process • Smart random testing – Aka monkey testing GTC 2019
- 17. GTC 2019 Laptop Testing Robot A0I Re-inspection Laptop Surface AOI Re-inspectionAOI for SMT lines and PCBs
- 18. 2: AOI (Automated Optical Inspection) Re-inspection GTC 2019 Common SMT defects
- 19. 2: AOI Re-inspection AOI Machine Fail Pass GTC 2019 High false positive Low first-pass yield Re-inspection by human
- 20. 2: AOI Re-inspection AOI Machine Pass Fail AOI Re-inspection Pass Fail GTC 2019 By: , , of NTU & advised by Wei-Chao Chen of Skywatch & NTU & Trista Chen of Inventec Data & domain knowledge provided by Jessie Huang, Peter Chu @ TAO, among others High false positive Low first-pass yield
- 21. 2: AOI Re-inspection GTC 2019 Traditional Computer Vision defects w/ well-defined rules Deep Learning defects w/ intuitive descriptions
- 22. 2: AOI Re-inspection GTC 2019 0. Excess Solder (true) 1. Excess Solder (false positive à pass ) 2. Insuff. Solder (true) 2. Insuff. Solder (false positiveàpass)
- 23. GTC 2019 Laptop Testing Robot A0I Re-inspection Laptop Surface AOI Re-inspectionAOI for SMT lines and PCBs
- 24. 3. Laptop Surface AOI GTC 2019 Brand Logo Customer Info
- 26. Brand Logo GTC 2019 Customer Info Type #2 Dot Type #1 Scratch Type #3: Area Type #4: Abrasion Type #5: Don’t care
- 27. 3. Laptop Surface AOI GTC 2019 Defect type Class A spec Class B spec Scratch Length: 12mm Acceptable: 2 lines Length: 20mm, Acceptable: 2 lines Dent 0.5 mm2 < size < 0.7mm2 Acceptable: 2 points 0.5 mm2 < size < 1mm2 Acceptable: 3 points … Fail Pass Human readable rules Labor intensive
- 28. 3. Laptop Surface AOI GTC 2019 Defect type Class A spec Class B spec Scratch Length: 12mm Acceptable: 2 lines Length: 20mm, Acceptable: 2 lines Dent 0.5 mm2 < size < 0.7mm2 Acceptable: 2 points 0.5 mm2 < size < 1mm2 Acceptable: 3 points … Fail Pass Labor intensive Human readable rules AI
- 29. 3. Laptop Surface AOI GTC 2019 Fail Pass AI Training data
- 30. 3. Laptop Surface AOI Pass Fail AIMobile M1 (Nvidia TX2) GTC 2019 By: Benson Lin of Skywatch, Trista Chen and Irene Chen of Inventec, Wei-Chao Chen of Skywatch & Inventec In collaboration with Steven Wang, Sing-Wang Chen, Alfa Shih & Tim Zhang et al @ ICC Machine manufactured by Jerry Tseng @ Leh-Yeh; Edge machine provided by Mark Lu @ AIMobile Surface defect classification
- 31. 3. Laptop Surface AOI Pass/Fail Classification DL detection Pass Fail AIMobile M1 (Nvidia TX2) GTC 2019
- 32. 3. Laptop Surface AOI Pass/Fail Classification DL detection Pass Fail GTC 2019 Stage 1: Explainable Stage 2: Adjustable
- 33. 3. Laptop Surface AOI Pass/Fail Classification DL detection Pass Fail GTC 2019 Stage 1: Explainable Stage 2: Adjustable
- 34. 3. Laptop Surface AOI – Explainable GTC 2019 Why did it pass/fail? • Defect type • Defect count • Defect size Defect type Class A spec Class B spec Scratch Length: 12mm Acceptable: 2 lines Length: 20mm, Acceptable: 2 lines Dent 0.5 mm2 < size < 0.7mm2 Acceptable: 2 points 0.5 mm2 < size < 1mm2 Acceptable: 3 points …
- 35. Defect class cluster analysis Defect types: s1 s2 s3 s4 s5 5 classes a b c a b c a b c a b c a b c 15 classes GTC 2019 Stage-1 multi-class detector 5 classes ? ?
- 36. GTC 2019 a b c a b c a b c a b c a b c Defect types: s1 s2 s3 s4 s5 15 classes Laptop categories: a, b, c Defect types: s1 s2 s3 s4 s5 Output Input s5 s4 s1 s3 s2 5 classes Defect class cluster analysis
- 37. 3. Laptop Surface AOI Pass/Fail Classification DL detection Pass Fail GTC 2019 Stage 1: Explainable Stage 2: Adjustable
- 38. 3. Laptop Surface AOI – Adjustable GTC 2019 Class A 2 lines 2 dots
- 39. 3. Laptop Surface AOI – Adjustable GTC 2019 Class B
- 40. 3. Laptop Surface AOI – Adjustable GTC 2019 Class C
- 41. 3. Laptop Surface AOI – Adjustable GTC 2019 Class A
- 42. 3. Laptop Surface AOI – Adjustable GTC 2019 Class A+
- 43. 3. Laptop Surface AOI – Adjustable GTC 2019 Class A++
- 44. 3. Laptop Surface AOI Pass/Fail Classification DL detection Pass Fail GTC 2019 Stage 1: Explainable Stage 2: Adjustable
- 45. 3. Laptop Surface AOI – Factory Prototype GTC 2019 CCD NB
- 46. Benefits of AI for Manufacturing GTC 2019 • The Big Scope – Consistency – Faithful digital record – Industry 4.0 OEE SECS MES SECS 1. 2. 3. 4. MES / / / /… / / & Idle 1. 2. 3. 4. 1. 2. MES/ 3. 4. / Repair Source: Taiwan’s Institute for Information Industry (III) 2017 • The Obvious – Accuracy – Labor Saving ROI
- 47. Benefits of AI for Manufacturing – I4.0 GTC 2019 95.00 96.00 97.00 98.00 99.00 100.00 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 Production Efficiency Peak Productivity Identified Problem Image source: Hitachi “Factories of the Future”, NEXT 2019 time Process chain
- 48. GTC 2019 Laptop Testing Robot A0I Re-inspection Laptop Surface AOI Re-inspectionAOI for SMT lines and PCBs
- 49. Unexpected Bonus: to do well while building your AI engine We’re on a mission to connect 60,000 residential disabled in Taiwan to join AI work by providing high-quality data fuels to your AI engines.
- 50. GTC 2019 Source: Christopher Porter’s Roomba and Cat from Flickr
- 51. GTC 2019
- 52. Trista Chen, Chief Scientist of Machine Learning of Inventec Wei-Chao Chen, Co-founder of Skywatch & Head of Inventec AI Center {chen.trista, chen.wei-chao} @ inventec.com S9617