NUS-ISS Learning Day 2018- Intelligent visual data analytics for smart urban solutions
- 1. Intelligent Visual Data Analytics for Smart Urban Solutions A picture is worth a thousand words Dr TIAN Jing, NUS-ISS Email: tianjing@nus.edu.sg 13 July 2018 1 Agenda • Introduction to visual data analytics • Machine learning for visual data analytics • Design and build visual data analytics solutions • Demo 2
- 2. Introduction to visual data analytics 3 Visual data (image and video) 4
- 3. Global computer vision market revenue Source: https://www.tractica.com/research/computer-vision-technologies-and-markets/ 5 China computer vision market revenue Source: Computer vision market in China, iResearch Inc 6
- 4. Source: https://www.straitstimes.com/singapore/new-technology-on-trial-at-changi-prison-can- detect-cell-fights-through-video-analytics 7 Use case: Security and safety • Objective: Use CCTV video analytics to provide real-time monitoring and alert relevant parties, for example by sending automatic alerts to command centres and security guards or to the public. Source: https://www.channelnewsasia.com/news/business/new-tech-tested-at-sentosa-can-spot-unattended-objects-intruders-9135128 Intruder detection in the sea Unattended object detection in public place Track the person- of-interest in multiple cameras 8
- 5. Use case: Airport Source: https://www.businesstimes.co m.sg/transport/changi-airport- testing-facial-recognition- systems-to-find-late-passengers 9 Use case: Construction • Objective: PUB is using video analytics to detect silt discharge at construction sites, automatically sends alerts to various parties, such as the contractor, CCTV vendor and PUB upon detection of silt discharge or image problems. 10 Source: https://www.pub.gov.sg/drainage/earthcontrolmeasures/SIDS
- 6. Use case: Flood monitoring • 208 water level sensors used for monitoring of the drainage system. • 49 CCTV at selected locations used for monitoring the road. 11 Source: https://app.pub.gov.sg/waterlevel/pages/WaterLevelSensors.aspx Use case: Public transportation Public transportation analytics: Various sensor data, such as fare cards, Wi-Fi, CCTV systems and cellular networks are used to better model commuter flows, improve planning. LTA Datamall, https://www.mytransport.sg/content/mytransport/home/dataMall.html 12
- 7. Multi-view Geometry Face Detection/Recognition Object Tracking Object Detection Human Pose 3D Scene Vision for Robots Autonomous vehicle Sports Visual data analytics tasks (1) 13 Visual data analytics tasks (2) Low Level Task Input: Image Output: Image Examples: Noise removal, image sharpening Mid Level Task Input: Image Output: Attributes Examples: Object recognition, segmentation High Level Task Input: Attributes/Image Output: Understanding Examples: Scene understanding, autonomous navigation Low computational cost High computational cost 14
- 8. History • In 1966, a MIT professor Marvin Minsky asked his student Gerald Jay Sussman to “spend the summer linking a camera to a computer and getting the computer to describe what it saw”. 15 Visual data analytics pipeline (1) Image acquisition Image restoration Image processing and analysis Feature representation & description Image enhancement Object detection and recognition Application domain Scene understanding 16
- 9. Visual data analytics pipeline (2) Image acquisition Image restoration Image processing and analysis Feature representation & description Image enhancement Object detection and recognition Application domain Scene understanding 17 Visual data analytics pipeline (3) Image acquisition Image restoration Image processing and analysis Feature representation & description Image enhancement Object detection and recognition Application domain Scene understanding 18
- 10. Visual data analytics pipeline (4) Image acquisition Image restoration Image processing and analysis Feature representation & description Image enhancement Object detection and recognition Application domain Scene understanding 19 Visual data analytics pipeline (5) Image acquisition Image restoration Image processing and analysis Feature representation & description Image enhancement Object detection and recognition Application domain Scene understanding 20
- 11. Visual data analytics pipeline (6) Image acquisition Image restoration Image processing and analysis Feature representation & description Scene understanding Image enhancement Object detection and recognition Application domain 21 Visual data analytics pipeline (7) Image acquisition Image restoration Image processing and analysis Feature representation & description Scene understanding Image enhancement Object detection and recognition Application domain 22
- 12. Machine learning for visual data analytics 23 Machine learning • Speech Recognition • Image Recognition • Playing Go • Chatbot f f f f “Cat” “How are you” “5-5” “Hello”“Hi” (what the user said) (system response) (next move) Objective: Looking for a function! 24
- 13. Machine learning tasks in vision 25 Image classification Training Labels Training Images Classifier Training Training stage Image Features Image Features Testing stage Test Image Trained Classifier Trained Classifier Outdoor 26
- 14. Image classification 27 Classical machine learning pipeline 1. Select / develop features: SURF, HoG, SIFT, … 2. Add on top of this Machine Learning for multi-class recognition and train classifier 28 Feature Extraction: SIFT, HoG... Detection, Classification Recognition Classical computer vision feature definition is domain-specific and time-consuming
- 15. Deep learning pipeline “having had countless ConvNet papers rejected, published and ignored, and occasionally paid attention to, for over 15 years” -- Yann Lecun 29 Deep learning pipeline • Build features automatically based on training data • Combine feature extraction and classification 30 Deep neural network DL experts: Define neural network topology and train the neural network model
- 16. Beyond image classification 31 Image regions with CNN features 32
- 17. Design and build visual data analytics solutions 33 Architecture design considerations • Volume • Security and Privacy • Transmission Bandwidth • Latency and Response time • Data storage, retrieval, and management 34
- 18. Cloud computing Edge computing (Raspberry Pi) Vision intelligence: Cloud and Edge Edge Analytics MessageBroker Machine Learning / Artificial Intelligence Sensors/DeviceControllers Devices Cloud Analytics 35 Demo 36
- 19. Demos • Face detection and recognition • Generic object detection and image classification Page 37 Thank You! Dr TIAN Jing Email: tianjing@nus.edu.sg 38