Geospatial machine learning for urban development
2 likes2,256 views
This document discusses using machine learning and computer vision techniques with satellite imagery to generate addressing systems for areas that currently lack adequate street addressing. It describes a generative addressing scheme that uses hierarchical and linear descriptors, such as region names indicating orientation and distance from downtown and road names indicating distance from the center and orientation. The document outlines a pipeline that involves predicting road networks from satellite imagery, partitioning regions based on road connectivity, and assigning addressing cells along roads with distance and block offsets. Results show this approach can improve street address coverage to 80% for unmapped developing areas by discovering road networks in non-urban settings and defining regions according to natural boundaries.
![39Geometric Shape Processing: Satellite Images
[*] I. Demir et al., 2018. “Generative Street Addresses from Satellite Imagery”.
International Journal on Geo-Information (IJGI).](https://image.slidesharecdn.com/geospatialmachinelearningforurbandevelopment-181114165928/85/Geospatial-machine-learning-for-urban-development-39-320.jpg)
![Completion & Reconstruction 47
• Voxelize building proxy from
footprint
• Find roofs with photo-
consistency in aerial images
• Apply graph-cuts:
•Building
•Building-ground
•Ground
[*] I. Garcia-Dorado I. Demir, D. Aliaga.
2013. “Automatic Urban Modeling Using
Volumetric Reconstruction with Surface
Graph-cuts”. Computers & Graphics.](https://image.slidesharecdn.com/geospatialmachinelearningforurbandevelopment-181114165928/85/Geospatial-machine-learning-for-urban-development-47-320.jpg)
Geospatial machine learning for urban development
- 1. Geospatial Machine Learning for Urban Development Ilke Demir Facebook MLConf – The Machine Learning Conference
- 4. Understanding the World urban safety socioeconomic data & voting patterns poverty mapping disaster mapping
- 5. Open Datasets o DeepGlobe: https://deepglobe.org o Road extraction o Building detection o Land cover classification o SpaceNet: https://spacenetchallenge.github.io/ o Road network extraction o Building detection o Earth observation challenge: http://eochallenge.bemyapp.com/ o Water resource extraction o Change detection o Data fusion contest: http://www.grss-ieee.org/data-fusion-contest/ o Land cover land use classification from various sensor data o Functional map of the world: https://www.iarpa.gov/challenges/fmow.html o Labeling the world into land use categories
- 6. Case Study: DeepGlobe o Public datasets and benchmarks for scalable and reliable approaches o Satellite imagery is powerful as it is more structured than everyday images DeepGlobe focuses on machine learning and computer vision approaches on satellite images and brings together researchers with different perspectives by; o Publishing public datasets and baselines o Creating public challenges to benchmark different approaches o Organizing a workshop to sparkle new collaborations and ideas
- 7. DeepGlobe Tracks Road Extraction Challenge: - Maps, accessibility, and connectivity - Economic and developmental inclusion - Crisis response
- 8. DeepGlobe Tracks Building Detection Challenge: - Population dynamics and demographics - Disaster recovery and damage coordination - Urban development
- 9. DeepGlobe Tracks Land Cover Classification Challenge: - Sustainable development - Automation in agriculture - Urban planning and growth
- 10. DeepGlobe Challenges 1. Road Extraction Challenge - DigitalGlobe Vivid+ - 50 cm/pixel - Pixel-wise manual annotation - 2 classes - Thailand, Indonesia, India - 8570 images of 2220km2 - 70%/15%/15% split - ~4% positive pixels - Diverse road networks - 345 participants - 2150 submissions - 84 results in the leaderboard
- 11. DeepGlobe Challenges 2. Building Detection Challenge - SpaceNet Buildings v2 - 31cm single-band panchromatic - 1.24m 8 band multi-spectral - Manual annotation of polygons - 2 classes - Las Vegas, Paris, Shanghai, Khartoum - 24586 images of 9623 km2 - 60%/20%/20% split - 302701 buildings - 296 participants - 576 submissions - 25 results in the leaderboard
- 12. DeepGlobe Challenges 3. Land Cover Classification Challenge - DigitalGlobe Vivid+ - 50 cm/pixel - Pixel-wise manual annotation - 7 classes - Thailand, India, Indonesia - 1146 images of 1717 km2 - 70%/15%/15% split - 20m minimum granularity area - 311 participants - 1155 submissions - 28 results in the leaderboard
- 13. DeepGlobe Results and Baselines Roads DeepLab variation Only data augmentation by rotation IoU score 0.545
- 14. DeepGlobe Results and Baselines Roads DeepLab variation Only data augmentation by rotation IoU score 0.545 Buildings Ensemble 3 U-Net models Boost by OpenStreetMap data F1 score 0.693
- 15. DeepGlobe Results and Baselines Roads DeepLab variation Only data augmentation by rotation IoU score 0.545 Buildings Ensemble 3 U-Net models Boost by OpenStreetMap data F1 score 0.693 Lands DeepLab variation Data augmentation by rotations and class weights IoU score of 0.433
- 16. DeepGlobe Winners Roads Buildings Land D-LinkNet: LinkNet with Pretrained Encoder and Dilated Convolution for High Resolution Satellite Imagery Road Extraction Lichen Zhou, BUPT Building Detection from Satellite Imagery using Ensemble of Size-specific Detectors Ryuhei Hamaguc hi, Pasco Dense Fusion Classmate Network for Land Cover Classification Chao Tian, Harbin Institute of Technology 1. Semantic Binary Segmentation using Convolutional Networks without Decoders Shubhra Aich*; William van der Kamp; Ian Stavness, University of Saskatchewan 2. Stacked U-Nets with Multi-Output for Road Extraction Tao Sun*; Zehui Chen; Wenxiang Yang; Yin Wang, Tongji University 3. D-LinkNet: LinkNet with Pretrained Encoder and Dilated Convolution for High Resolution Satellite Imagery Road Extraction Lichen Zhou*; Chuang Zhang; Ming Wu, Beijing University of Posts and Telecommunications 4. Fully Convolutional Network for Automatic Road Extraction from Satellite Imagery Alexander Buslaev*, Mapbox; Selim Seferbekov, Veeva Systems; Vladimir Iglovikov, Lyft Inc; Alexey Shvets Massachusetts Institute of Technology 5. Road Detection with EOSResUNet and Post Vectorizing Algorithm Oleksandr Filin*; Serhii Panchenko; Anton Zapara, EOS Data Analytics 6. Residual Inception Skip Network for Binary Segmentation Jigar Doshi*, CrowdAI 7. Roadmap Generation using a Multi-Stage Ensemble of Neural Networks with Smoothing-Based Optimization Dragos Costea*; Alina Marcu; Emil Slusanschi; Marius Leordeanu, University Politehnica of Bucharest 8. Rotated Rectangles for Symbolized Building Footprint Extraction Matthew Dickenson*; Lionel Gueguen, Uber 9. Building Detection from Satellite Imagery Using Composite Loss Function Sergey Golovanov*; Rauf Kurbanov; Aleksey Artamonov; Alex Davydow; Sergey Nikolenko, Neuromation 10. Building Detection from Satellite Imagery using Ensemble of Size- specific Detectors Ryuhei Hamaguchi*; Shuhei Hikosaka, Pasco Corporation 11. TernausNetV2: Fully Convolutional Network for Instance Segmentation Vladimir Iglovikov*, Lyft Inc; Selim Seferbekov, Veeva Systems; Alexander Buslaev, Mapbox; Alexey Shvets Massachusetts Institute of Technology 12. Semantic Segmentation based Building Extraction Method using Multi-source GIS Map Datasets and Satellite Imagery Weijia Li*; Conghui He; Jiarui Fang; Haohuan Fu, Tsinghua University 13. CNNs Fusion for Building Detection in Aerial Images for the Building Detection Challenge Remi Delassus*, Qucit - LaBRI; Romain Giot, Univ. Bordeaux 14. Building Extraction from Satellite Images Using Mask R-CNN with Building Boundary Regularization Kang Zhao*; Jungwon Kang; Jaewook Jung; Gunho Sohn, York University 15. Deep Aggregation Net for Land Cover Classification Tzu-Sheng Kuo*; Keng-Sen Tseng; Jia-Wei Yan; Yen-Cheng Liu; Yu- Chiang Frank Wang, National Taiwan University 16. Stacked U-Nets for Ground Material Segmentation in Remote Sensing Imagery Arthita Ghosh*; Max Ehrlich; Sohil Shah; Larry Davis; Rama Chellappa, University of Maryland 17. Land Cover Classification from Satellite Imagery With U-Net and Lovasz-Softmax Loss Alexander Rakhlin*; Alex Davydow; Sergey Nikolenko, Neuromation 18. Dense Fusion Classmate Network for Land Cover Classification Chao Tian*, Harbin Institute of Technology; Cong Li; Jianping Shi, Sensetime 19. NU-Net: Deep Residual Wide Field of View Convolutional Neural Network for Semantic Segmentation Mohamed Samy; Karim Amer*; Kareem Eissa; Mahmoud Shaker; Mohamed ElHelw, Nile University; 20. Feature Pyramid Network for Multi-Class Land Segmentation Selim Seferbekov*, Veeva Systems; Vladimir Iglovikov, Lyft Inc; Alexander Buslaev, Mapbox; Alexey Shvets Massachusetts Institute of Technology 21. Uncertainty Gated Network for Land Cover Segmentation Guillem Pascual*; Santi Seguí; Jordi Vitria, Universitat de Barcelona 22. Land Cover Classification With Superpixels and Jaccard Index Post-Optimization Alex Davydow*; Sergey Nikolenko, Neuromation
- 17. Changing the World urban planning simulations generative models
- 19. Shape Understanding Point clouds: Arch Physical objects: Urban spaces: Satellite Images: Architectural models:
- 20. Point clouds: Satellite images: Urban spaces: Architecture: Generative Modeling
- 21. Case Study: Street Addresses • 75% of the world lives without adequate addressing. What3Words • 4 billion people are ‘invisible’. United Nations • Haiti earthquake: 48 hours reaction time, 6 months complete road vectors. OpenStreetMap
- 22. Geocoding Solutions “issuer.lollipop.ripe” “ItsADream” “37.482825, -122.145661” “75682SB3084” “parrot.casino.failed” “SweetPotato” “102.22556, -12.166981” “7098HGT3083” • f(lat, lon) = “address” • Hashes, random words, manual tags • No spatial relation cue • No city/state/country • No accessibility • No geometry A B Humans need streets!
- 23. Traditional Addressing Systems London postal code system: Radial regions based on orientation and distance South Korea streets: Meter markers Japan block system: Hard to decipher Dubai addressing: Uses districts Berlin numbering: Zigzag house pattern
- 24. Our Generative Scheme • 5 alphanumeric fields • Hierarchical and linear descriptors • To close the gap between physical addresses and automated geocoding Road naming scheme: - distance from the center - orientation in odd parity Region naming scheme: - orientation wrt downtown - distance from downtown House numbering scheme: - meter markers on the road - block letters from the road “I7 Hacker Way, Menlo Park, CA, US”
- 25. Pipeline: Satellite Images ç • Irregular urban structure • Illumination/weather/country • Different road types
- 26. Pipeline: Road Predictions • Binary road masks • 19K*19K, 0.5m/pixel • SegNet
- 27. Pipeline: Road Network • Orientation based median filtering • Road segments by orientation bucketing
- 28. NF NH NE Pipeline: Regions • Road graph: Node=intersection, edge=road, weight=length • Partition for max inter, min intra connectivity, using normalized min-cut.
- 29. Pipeline: Naming • Orientation bucketing into N, S, W, E • Trace regions based on distance to CA • Orientation bucketing into major axes • Trace roads based on order
- 30. Pipeline: Address Cells • 5 meter marker along the road • Odd/even based on RHR • Distance field of roads: block offset
- 31. Results: Unmapped Developing Country • Improve coverage up to 80% • Processed more than 200 districts (and increasing!) • Regions follow natural boundaries • Road network is being discovered in non-urban settings
- 32. Results: Unmapped Developing Country • Improve coverage up to 80% • Processed more than 200 districts (and increasing!) • Regions follow natural boundaries • Road network is being discovered in non-urban settings
- 33. Results • Improve coverage up to 80% • Processed more than 200 districts (and increasing!) • Regions follow natural boundaries • Road network is being discovered in non-urban settings • Changing the world!
- 34. News & Ads! o Geospatial Modeling and Visualization, Special Issue in Big Earth Data Journal http://bit.ly/BigEarthData o SUMO Challenge: Understanding indoor scenes from 360 RGBD data https://sumochallenge.org/ o Challenges and opportunities for deep learning in remote sensing, Special session in Living Planet Symposium 2019 https://lps19.esa.int/ o EarthVision 2019! (coming soon…) o DeepGlobe v2! (coming some day…)
- 35. Thanks… and your turn! Generative Street Addresses Code: https://github.com/facebookresearch/street-addresses Paper: https://research.fb.com/publications/robocodes DeepGlobe Benchmark Papers: http://bit.ly/deepglobe_papers Website: http://deepglobe.org Dataset: http://bit.ly/deepglobe Ilke Demir e-mail: idemir@fb.com Twitter: @ilkedemir
- 37. Evaluation Metrics Roads Pixel-wise mean IoU Buildings Average F1 score Lands Pixel-wise mean IoU
- 38. Design Choices Linear: similar addresses stored in a linear fashion Hierarchical: top-down structure for spatial encapsulation Compressible: 5x4 max (chars x words) Universal: independent of local language Inquirable: useful for geometric, proximity-based, and type-ahead queries Extendible: dynamically modifiable for new places Robust: flexible for overestimation and noise StructuralDesignParameters forefficientcomputerimplementation Linear: closer addresses are given related names Hierarchical: top-down subdivision of the world Memorable: short and alphanumeric, easily convertible Intuitive: with a sense of direction and distance Topological: consistent with road topology Inclusive: with local names (city, state) Physical: consistent with natural boundaries SemanticDesignParameters foruserfriendliness Machine Needs Human Needs
- 39. 39Geometric Shape Processing: Satellite Images [*] I. Demir et al., 2018. “Generative Street Addresses from Satellite Imagery”. International Journal on Geo-Information (IJGI).
- 40. Output Maps and Tools • .osm maps with roads (meter marking and offsetting on the fly) • ID-tool of MapBox for on-demand inverse/forward geocoding • rtree extension for efficient spatial querying • Experimental mobile app for self navigation • 21.7% decrease in arrival time using Robocodes
- 41. Results: Evaluation with Ground Truth • System learns 90.51% of roads • Approximately 80% on average • Better in urban environments • Ground truth prepared as if training data
- 42. Results: Mapped US City • More than 95% of the roads are found (compared to OSM). • Traditional addresses are more established, however • Robocodes are contextually and spatially easier to remember.
- 43. Results: Comparison Automated geocoding: A: parrot.casino.failed B: issuer.lollipop.ripe - Have irrelevant words based on lat/lon. Robocodes: 715D.NE127.Dhule.MhIn 716C.NE127.Dhule.MhIn - Have hierarchical and linear addresses. Landmark based: Green Park Green Park - Have roads but no addresses or labels. OSM: lat/lon lat/lon - Have neither road geometry, nor labels.
- 44. Limitations & Future Work • Robotic meter marking and offsetting: • (i) use smart parcel subdivision, • (ii) adapt to population density. • Imperfect training data: sample more countries. • Metric to evaluate regions: supervised learning of land annotations.
- 45. Inaccessible Areas • To extend our format to cover areas that are not accessible by streets, we explored different implementations to cover such areas, which are 26*5 m away from any street. • Geocoding as a function (excluding the version field): f (info, lat, lon) = x.y.z.t • For places with roads, info={road network, city, country} f (R, C) = x.y.city.country • Extreme case: only reliable information is latitude/longitude! 45
- 46. f(C,lat,lon) = hash(round(lat,3)) + dir(lat) . hash(round(lon,3)) +dir(lon) . C L-A-T-dir.L-O-N-dir.name.area Inaccessible Areas: Blackholes! • Linear hashing: • 26 letters + 10 digits • 100m x 100 m granularity • Last letter is the hemisphere • Range: 359.999, longitude: 7PRZ W • Hierarchical hashing: • Enlarge the grid from to 1 km x 1 km • Using two floating points = three letters • Within each cell, re-hash it to a 36 x 36 grid = one letter • New resolution: 30m, represented by five letters 46 f(C,lat,lon) = hash(round(lat,2)) + hash(lat - round(lat,2)) + dir(lat) . hash(round(lon,2)) + hash(lon - round(lon,2)) + dir(lon) . C LlatLlatHlatDlat .LlonLlonHlonDlon . name . Ocean /Continent /etc
- 47. Completion & Reconstruction 47 • Voxelize building proxy from footprint • Find roofs with photo- consistency in aerial images • Apply graph-cuts: •Building •Building-ground •Ground [*] I. Garcia-Dorado I. Demir, D. Aliaga. 2013. “Automatic Urban Modeling Using Volumetric Reconstruction with Surface Graph-cuts”. Computers & Graphics.
- 48. Website: http://deepglobe.org Papers: http://bit.ly/deepglobe_papers Dataset: http://bit.ly/deepglobe Hashtags: #DeepGlobe