SMART Seminar Series: "Using AI and edge computing devices for traffic flow monitoring". Presented by Dr Johan Barthelemy
1 like786 views
The document discusses the use of edge computing and IoT for traffic flow monitoring in urban environments, specifically in Liverpool's central business district. It highlights the deployment of an edge computing device utilizing NVIDIA Jetson TX2 and LoRaWAN for real-time data gathering and analytics on pedestrian and vehicle movement while ensuring privacy through on-device processing. The project aims to leverage this data for improving urban planning, traffic management, and citizen engagement through a public dashboard and applications.
SMART Seminar Series: "Using AI and edge computing devices for traffic flow monitoring". Presented by Dr Johan Barthelemy
- 1. UsingAI and edge computing devices for traffic flow monitoring Dr J. Barthélemy, Dr N. Verstaevel, Dr H. Forehead, Senior Prof. P. Perez, Bilal Arshad, Dinh (Andrew) Che
- 2. Edge Computing with Jetson TX2 for Monitoring Flows of Pedestrian and Vehicles At SMART, we believe that People with good information and good tools will make good Decisions and change their world 2
- 3. The case of Liverpool CBD is growing fast: new UoW campus, airport development,… What does it mean for the city and its community? What are the problems? Smart Cities and Suburbs Program: How can we use IoT to solve the problems? 3
- 4. Which problems? Let’s ask the people! Pedestrians - Where are they going? - What are the most popular routes? - What are the most congested locations? - Impact of city activity? Cyclists - Which route are they taking? - How can we improve bike usage? Cars - Live traffic? 4
- 5. Sensors locations 20 sensors Image credit: OpenStreetMap 5
- 6. How about using CCTV? 6 Privacy!
- 7. Sensor requirements • Mobile units + leveraging CCTV infrastructure • Privacy is important! – On board video analytics – Only indicators transmitted (no raw data!) • Real-time image processing • LoRaWAN network 7
- 8. LoRaWAN: An IoT Network Image credit: https://ttnmapper.org • Long range • Low power • Low bandwidth (max 200 bytes/message) • Open source • Free for use by the community 8
- 9. History of the prototypes 9
- 10. The sensor An edge computing device Components — NVIDIA Jetson TX2 for onboard processing — Pycom LoPy 4 for data transmission on The Things Network — Camera (USB webcam / existing CCTV) 10
- 12. From the input to the dashboard Image Acquisition • CCTV/Webcam Detection • Deep Convolutional Neural Network Tracking • Kalman Filtering Data transmission • LoRaWAN/OneM2M Dashboard/Database Image credit: NVIDIA Corporation Image credit: Pycom 12
- 13. Detection:YOLO v3 Inference: Jetson TX2Training: Titan Xp • Fully convolutional DNN • 106 hidden layers • Detections at 3 scales • 3 classes: person, bicycle, vehicles • Pascal VOC and COCO datasets cuDNN FP16+ + 13
- 14. Detection:YOLO v3 Current training dataset • 80.000 images from Pascal VOC, COCO • 6 data augmentation techniques 480.000 images 14
- 16. Tracking: Kalman Filtering 16 Prediction of O in current frame Position of O in previous frame Detection D in current frame 𝐼𝑜𝑈 ≔ ≥ 𝛿 𝑥 𝑦 if D = O
- 17. Detection:YOLO v3 • Detecting locations of pedestrians and vehicles • Number of objects of each type 17 VIRAT Video Dataset
- 18. Tracking: Kalman Filtering • Associating IDs with the detections • Trajectories 18 VIRAT Video Dataset
- 19. Final output No image Privacy OK! 19
- 20. Liverpool Smart Pedestrian architecture Fixed counters Air quality x20 x5 x15 Mobile counters Sensors IoT Core ApplicationsTransport + Forecasting Analytics Public Dashboard Liverpool IoT Core Citizens app 20
- 21. ● Open source service platform for M2M interoperability based on the oneM2M standard. ● Follows a RESTful approach GET/POST/DELETE/SUBSCRIBE ● Access control policies ● Plugins ● Can be decentralised with multiple nodes ● Support different technologies: ○ Lora ○ Zigbee ○ Sigfox ○ Bluetooth Low Energy ○ Z-wave OM2M Eclipse 21
- 22. Liverpool IoT Core MQTT TTN AE - TheThingsNetwork App Id … DATA IN_Node METADATA MN_Node1 MN_... Subscriptions HTTP Rest API Database Inject HTTP GET Publish data Collect data Manage and secure data Store data POA 22
- 23. The Things Network application AE-TheThingsNetwork CNT- App Id CNT - DATA CIN – DATA_... CNT - METADATA CIN – METADATA_... 23
- 24. Visual Sensor application MN- SensorName CNT- DATA CIN –... CIN –... CNT- DESCRIPTOR CNT- CONFIGURATION CIN –... CNT- SYSTEM CIN –... CNT- COMMAND CIN –... 24
- 25. Dashboard 25
- 26. Dashboard Trajectories of the detections (inside a building) 26
- 27. Dashboard Heatmap of the maximum number of the detections (inside a building) 27
- 28. Dashboard 28
- 29. Next steps: Inferring network dynamics Image credit: OpenStreetMap Image credit: Google Maps 29
- 30. Next steps:Algorithm transferability Testing in Vietnam 30
- 31. Conclusions It’s only the beginning! • VS able to perform real time object detection and tracking • Privacy compliant… but meaningful information • Open data for people centric approach – citizen applications – city and traffic planners • IoT core scalable and interoperable – adding new sensors: air quality, noise, … Traffic modelling AnalyticsDashboardCitizens app 31
- 32. But it’s already successful! Best Local Government Initiative Document title32
- 34. Image credit: https://towardsdatascience.com YOLO v3 architecture 34
- 35. 35 Image credit: Redmon, J. and Farhadi, A. (2018). YOLOv3: An Incremental Improvement YOLO v3 performances
- 36. GPU and Edge computing Image credit: NVIDIA Corporation Dataset Deep Neural Net Trained model deployment Inference: Jetson Image credit: NVIDIA Corporation Training: Titan/DGX/Tesla Image Acquisition Detection Tracking Data transmission Dashboard/Database 36
- 37. Architecture of the solution Fixed counters Air quality x20 Noise level x20 x5 x15 Mobile counters Sensors IoT Core ApplicationsTransport + Private and Public APIs Traffic modelling Analytics Dashboard Citizens app 37
- 38. Dashboard Heatmap of the maximum number of detections 38
- 39. Dashboard Trajectories of the detections 39