Disaster Monitoring using Unmanned Aerial Vehicles and Deep Learning
Download as PPTX, PDF3 likes979 views
The document discusses a workshop on disaster management using unmanned aerial vehicles (UAVs) and deep learning techniques for real-time monitoring and identification of disasters. It highlights the advantages of drones, modern computer vision techniques, and the effectiveness of convolutional neural networks in disaster image analysis. Future work includes enhancing the dataset and improving precision for practical applications in disaster management.
![Motivation
6
Advantages of Deep learning:
• Superior performance in terms of precision
• Perform classification and predictions particularly
well due to their structure.
• Flexible and adaptable
• No need for hand-engineered features
• Generalizes well
• Robust in low-resolution and -quality images.
Andreas Kamilaris and Francesc X. Prenafeta-Boldú, Deep Learning in Agriculture: A
Survey, Computers and Electronics in Agriculture Journal, 2017. [Under review]](https://image.slidesharecdn.com/201709-enviroinfodisastermanworkshop-170917101749/85/Disaster-Monitoring-using-Unmanned-Aerial-Vehicles-and-Deep-Learning-6-320.jpg)
![Methodology
12
CNN Model: VGG architecture, pre-trained with the
ImageNet dataset of images.
Dataset: 544 aerial photos from Google images (min.
256x256 pixels), acquired using the query:
[Disaster]: earthquake, hurricanes, flood and fire.
[Landscape]: aerial views of cities, villages, forests and
rivers
[Disaster | Landscape] + "aerial view" + "drone"](https://image.slidesharecdn.com/201709-enviroinfodisastermanworkshop-170917101749/85/Disaster-Monitoring-using-Unmanned-Aerial-Vehicles-and-Deep-Learning-12-320.jpg)
Disaster Monitoring using Unmanned Aerial Vehicles and Deep Learning
- 1. 1 EnviroInfo Conference 2017 Disaster Management for Resilience and Public Safety Workshop Disaster Monitoring using UAV and Deep Learning Andreas Kamilaris 13th September, 2017 Luxembourg
- 2. Problem 2 Monitoring and identification of disasters are crucial for mitigating their effects on the environment and on human population.
- 3. Motivation 3 Disaster monitoring can be facilitated by the use of unmanned aerial vehicles (UAV), equipped with camera sensors which can produce frequent aerial photos of the areas of interest.
- 4. Motivation 4 Advantages of Drones: • Small size • Low cost of operation • Exposure to dangerous environments • High probability of mission success • No risk of loss of aircrew resource • High-resolution image sensing • High operational flexibility
- 5. Motivation 5 Modern computer vision techniques: • Artificial Neural Networks • Scalable Vector Machines • Multi-layer Perceptrons • Random Forests • Gaussian Mixture Models • K-Nearest Neighbors • Unsupervised feature learning • Feature extraction techniques: Color, shape, texture • Deep learning Machine Learning- based Approaches Probabilistic Modelling
- 6. Motivation 6 Advantages of Deep learning: • Superior performance in terms of precision • Perform classification and predictions particularly well due to their structure. • Flexible and adaptable • No need for hand-engineered features • Generalizes well • Robust in low-resolution and -quality images. Andreas Kamilaris and Francesc X. Prenafeta-Boldú, Deep Learning in Agriculture: A Survey, Computers and Electronics in Agriculture Journal, 2017. [Under review]
- 7. Research Questions 7 Can drones and aerial image sensing be used for real-time monitoring of physical areas and? accurate identification of disasters? Can deep learning be used in combination with drones and aerial images for real-time disaster monitoring/identification?
- 8. Deep Learning 8 Convolutional Neural Networks
- 9. Deep Learning 9 Convolutional Neural Networks • Can be applied to any form of data, such as audio, video, images, speech, and natural language. • Various “successful” popular architectures: AlexNet, VGG, GoogleNet, Inception-ResNet etc. • Pre-trained weights • Common datasets for pre-training CNN architectures include ImageNet and PASCAL VOC. • Many tools and platforms that allow researchers to experiment with deep learning e.g. Keras, Theano.
- 10. General Idea 10 Disaster!Nothing to worry about!
- 11. State of the Art 11 No. Disaster Image source Accuracy 1. Fire (Kim, Lee, Park, Lee, & Lee, 2016) Aerial photos Human-like judgement 2. Avalanche (Bejiga, Zeggada, Nouffidj, & Melgani, 2017) Aerial photos 72-97% 3. Car accidents and fire (Kang & Choo, 2016) CCTV cameras 96-99% 4. Landslides (Liu & Wu, 2016) Optical remote sensing 96% 5. Landslides and flood (Amit, Shiraishi, Inoshita, & Aoki, 2016) Optical remote sensing 80-90%
- 12. Methodology 12 CNN Model: VGG architecture, pre-trained with the ImageNet dataset of images. Dataset: 544 aerial photos from Google images (min. 256x256 pixels), acquired using the query: [Disaster]: earthquake, hurricanes, flood and fire. [Landscape]: aerial views of cities, villages, forests and rivers [Disaster | Landscape] + "aerial view" + "drone"
- 13. Dataset 13 No. Image Group No. of Images Relevant Possible Disaster 1. Buildings collapsed 101 Earthquakes and hurricanes 2. Flames or smoke 111 Fire 3. Flood 125 Earthquakes, hurricanes and tsunami 4. Forests and rivers 104 No Disaster 5. Cities and urban landscapes 103 No Disaster
- 14. Dataset: Disasters 14 Buildings collapsed Flames or smoke Flood
- 15. Dataset: Landscapes 15 Forests and rivers Cities and urban landscapes
- 16. Setup 16 • 82% (444 images) of our dataset as training data and 18% (100 images) as testing data. • Random assignment of images in training/testing. • Training procedure 20 minutes on a Linux machine, testing 5 minutes for the 100 images. • Learning rate: 0.001 • Used data augmentation techniques. • 30 epochs
- 17. Results: Training Vs. Testing 17 83 84 85 86 87 88 89 90 91 92 82-18 70-30 75-25 85-15 90-10 Training Vs. Testing Percentage OverallPrecision(%)
- 18. Results: Training Vs. Precision 18 0 10 20 30 40 50 60 70 80 90 100 5 10 15 20 25 30 35 OverallPrecision(%) Number of Epochs
- 19. Results: Confusion Matrix 19 91% Precision 9% Error
- 20. Results: Analysis of Error 20 9% Error Urban Vs. Buildings collapsed (4%) Urban Vs. Fire (2%) Urban Vs. Flooding (1%)Flooding Vs. Buildings collapsed (2%)
- 21. Conclusion 21 Deep learning offers good precision and many benefits. Can be successfully used in combination with UAV for disaster monitoring/identification. It has also some disadvantages: • It takes (sometimes much) longer time to train. • It requires the preparation and pre-labeling of a dataset containing at least some hundreds of images.
- 22. Future Work 22 • Publish the dataset to the research community. • Enhance the dataset with more images. • Experiment with different architectures, platforms and parameters. • Increase overall precision to more than 95%. • Perform a real-life case study with drones used for monitoring some particular disaster e.g. indication of fire.
- 23. Vision 23 Better disaster modelling, especially when combining UAV and deep learning with geo- tagging of the events identified and geospatial applications. Facilitate the integration of relevant actors (i.e. action forces/authorities, citizens/volunteers, other stakeholders) in disaster management activities with regard to communication, coordination and collaboration.
- 24. 24 Many thanks for your attention! Andreas Kamilaris andreas.kamilaris@irta.cat
Editor's Notes
- #3: If wireless objects (e.g. sensor networks) represent a future of “billions of information devices embedded in the physical world,” why should they not run the standard internetworking protocol? Thus, the Internet can penetrate into the real world of physical objects.
- #4: If wireless objects (e.g. sensor networks) represent a future of “billions of information devices embedded in the physical world,” why should they not run the standard internetworking protocol? Thus, the Internet can penetrate into the real world of physical objects.