"Challenges and Approaches for Extracting Meaning from Satellite Imagery," a Presentation from Orbital Insight
0 likes246 views
The document discusses Orbital Insight's approach to utilizing satellite imagery for data analysis, highlighting practical machine learning methods, challenges in data handling, and the importance of temporal data. It emphasizes the significance of integrating multiple data sources to enhance signal accuracy and trend analysis in geographic observations. The conclusions stress the distinct characteristics of satellite imagery and the need for specialized machine learning models to address these challenges.
"Challenges and Approaches for Extracting Meaning from Satellite Imagery," a Presentation from Orbital Insight
- 1. © 2019 Orbital Insight Challenges and Approaches for Extracting Meaning from Satellite Imagery Adam Kraft Orbital Insight May 2019
- 2. © 2019 Orbital Insight Overview • Introduction to Orbital Insight • Practical Machine Learning Methods for Satellite Imagery • Working with Multiple Data Sources • Analyzing Trends and Change Over Time 2
- 3. © 2019 Orbital Insight Introduction to Orbital Insight
- 4. © 2019 Orbital Insight Helping Clients Make Better Decisions Today Financial data Ship location (AIS) data EO/SAR Satellite Imagery Drone / Aerial Imagery Manufacturing data Identification of areas of interest Weather data Mobile Device data Business data IOT data Area of Interest (AOI) data Image ingestion and processing Normalization / pattern detection GPU-based CV / ML RESTful API Web App SaaS Orbital Insight Orbital Insight sources, processes, and transforms geospatial datasets at scale
- 5. © 2019 Orbital Insight TRUCKS LAND USE TANK SHADOWS SHIPS NEW HOUSING DEVELOPMENT AIRPLANES BUILDINGS RAILCARS Computer Vision is Critical for Large Scale Processing of Satellite Imagery
- 6. © 2019 Orbital Insight Practical Machine Learning Methods for Satellite Imagery
- 7. © 2019 Orbital Insight Common ML Pipeline Data ML Model Loss Function(s) 7 Optimization
- 8. © 2019 Orbital Insight Data Handling for Satellite Imagery • Augmentations • Full 360 degree rotations • Shear transformations • Adding artificial clouds and haze 8 Data ML Model Loss Function(s)
- 9. © 2019 Orbital Insight Data Handling for Satellite Imagery • Sampling • Class imbalance • Rare cases • Active learning 9 Data ML Model Loss Function(s) Source: http://burrsettles.com/pub/settles.activelearning.pdf
- 10. © 2019 Orbital Insight ML Model for satellite imagery • Architecture • Aim to retain full resolution • Pretraining with more/less than 3 channels • Can sample 3 channels • Add zeros for extra channels, freeze part of network for a few epochs 10 Data ML Model Loss Function(s) ImageNet car
- 11. © 2019 Orbital Insight Loss Functions for satellite imagery • Handle class imbalance • Enforce temporal consistency • Can predict “free” information • Satellite metadata • Distance or neighbor information 11 Data ML Model Loss Function(s) https://earthobservatory.nasa.gov/features/ColorImage/page2.php
- 12. © 2019 Orbital Insight Working with Multiple Data Sources
- 13. © 2019 Orbital Insight Variety of Data / Inputs from Multiple Sources 13
- 14. © 2019 Orbital Insight Variance in Observing Same Location 14
- 15. © 2019 Orbital Insight Domain Transfer and Adaptation Domains can differ across: • Seasons • Geographies • Sensors 15 M. Wulfmeier, A. Bewley, and I. Posner, “Addressing Appearance Change in Outdoor Robotics with Adversarial Domain Adaptation,” in IEEE/RSJ International Conference on Intelligent Robots and Systems, 2017.
- 16. © 2019 Orbital Insight How to Best Combine Sources? • Fuse inputs • Easier to integrate • Fuse outputs • Easier to interpret • Intermediate fusion • Difficult to integrate/interpret, yet may yield best results 16 CNN Output CNN Output
- 17. © 2019 Orbital Insight Analyzing Trends and Change Over Time
- 18. © 2019 Orbital Insight More Data is Usually Better • Signal increases as data increases • Using more sources increases signal • Errors can wash out • Less dependent on accuracy of individual measurements 18
- 19. © 2019 Orbital Insight Temporal Data • Not just analyzing snapshots • Non-uniform samples • Unlike other temporal data: audio, video • Noise from clouds and haze 19
- 20. © 2019 Orbital Insight Methods for Learning on Temporal Data • LSTMs / 3D convolutions • Better uncertainty outputs from ML models • Results in even better trend analysis 20 Kendall, Alex, and Yarin Gal. "What uncertainties do we need in bayesian deep learning for computer vision?." Advances in neural information processing systems. 2017.
- 21. © 2019 Orbital Insight Conclusions • Know your data. Satellite imagery has different characteristics than consumer camera images. We can use this to our advantage. • Satellite data contains many different sources and there are different ways to combine information from those sources • The temporal component to satellite imagery can add challenges, which you should account for in your ML models. 21
- 22. © 2019 Orbital Insight Additional Resources 22 Orbital Insight Links Company Website https://orbitalinsight.com/ New York Times Article https://www.wsj.com/articles/startups- mine-market-moving-data-from-fields- parking-lotseven-shadows-1416502993 Other Satellite Imagery Links Functional Map of the World Challenge https://www.iarpa.gov/challenges/fmow.ht ml
- 23. © 2019 Orbital Insight Back-up Slides
- 24. © 2019 Orbital Insight Orbital Insight uses computer vision and data science to turn millions of images into a big-picture understanding of the world. Port of Rotterdam. Image Source: Astrium
- 25. Image Source: PBS Image, SpaceX Falcon Heavy Launch Defining the New Geospatial Analytics Category Commercialization of Space, Artificial Intelligence, Cloud & GPUs Commercialization of Space Cloud Computing & GPUs 25 Artificial Intelligence (Computer Vision & Data Science) Launch Systems Satellite Operations Analytics © Orbital Insight
- 26. © Orbital Insight 13 Image Source: Orbital Insight Data Overlaid on a Satellite Image Consumer Traffic Parked Car Counting; Retailers & Malls UNITED STATES