Brewing the Ultimate Data Fusion
2 likes693 views
The document discusses various data fusion techniques and tools in GIS, emphasizing the selection of ingredients like spatial and web data, and the choice of databases for effective data handling. It highlights the advantages of vector tiles over raster tiles in terms of storage and update efficiency, and outlines the use of UAVs for collecting geospatial data, alongside challenges in data processing without programming. Additionally, it introduces features for accessing and manipulating previous data rows to enhance data visualization and analysis.
Brewing the Ultimate Data Fusion
- 1. Brewing the Ultimate Data Fusion
- 2. 1. Choose your Potion Ingredients ● Spatial (GIS, CAD, BIM, raster, etc.) ● Web ○ FTP ○ Web Services - OGC, AWS, Dropbox, etc. ○ APIs ○ HTML webpages ○ JSON, XML, RSS feeds ● So many more!
- 3. 2. Choose a Cauldron NoSQL Relational database File-based database Visualization/BI tool like Tableau, Qlikmaps Metadata catalog
- 4. Master Database vs. Metadata Catalog
- 6. Awesome Vector Tile World Backgrounds Ana García de Vicuña Pablo Martínez
- 7. Vector Tiles Format in FME Geographic information stored in GeoJSON format into an SQLite database. Different layers for each zoom level.
- 8. Raster Tiles • Static PNG • Each collection only one style • Large storage cost • Tile generation takes time • Painful update of data Vector Tiles • GeoJSON – Dynamic PNG on the fly – Style defined in XML file • Each collection multiple styles • Less storage cost • Less generation time • Easy to update Raster Tiles vs. Vector Tiles
- 9. Multilevel Tile Cache Generation Engine Vector data text labels Vector Tile Mapnik XML Symbology PNG from Mapnik in the Server Tile Render
- 10. Vector and Raster Tile Composition Merge Raster Tiles (Orthoimage or Shaded Relief) with stylized Vector Tiles to get another background.
- 11. FME Workbench Input Workbench Output tiles Vie w Get the whole story at the
- 12. Data Fusion # 2
- 13. YVR & the Indoor Mapping Experience Carlos Silva GIS Specialist, Engineering Services Vancouver International Airport
- 14. YVR: 4 yrs ago (pre-GIS) CAD, 3 – 6 months post-construction Graphic design pamphlets, PDF
- 15. How the Mapping Process Works @YVR 1st Stage "Internal", FME 2nd Stage "Public", FME 3rd Stage "Uploads" via FME 4th Stage, "Apple AVF" Hideyuki KAMO
- 16. ArcMap: Map Production; Data Verification & Edits; Map & Feature Services (Internal Only)
- 17. YVR Digital Gateway - JSON
- 18. Where is YVR now?
- 19. 1) Public Terminal Maps (PDFs, Hard Copy Pamphlet, Website Maps (Leaflet with GeoJSON) 2) Internally: CAD 2 GIS via FME, accessible on an internal GIS application 3) YVR App & Website: Data Delivery via YVR Digital Gateway Where is YVR now?
- 20. Typical Issues/Solutions - timeliness - spatial errors/topology - additional data points (sources) changing - software & DB updates - I.T. updates KEY: Automation, standards, and API access Get the whole story at the
- 21. Data Fusion # 3
- 23. SCS Milestones in UAS Program • UAS Policy in place (FAA compliance) • UAS certification training of 60 • Southern Company is leading the electric utility industry in UAS Plant Farley UAS Inspec2on: Nuclear Industry first UAS Cer2fica2on Training Plant Branch, GA - 1000 feet above ground level
- 24. Geospatial data in the UAS workflow Fly Review Delivery (Option) Plan ArcGIS Pla+orm
- 25. From Excel and File Structure …is there a better way to visualize these missions and data products?
- 26. Transform missions from rows to points … project & data locations of each mission are stored in an Excel or HTML report created by FME and stored as an attachment on each mission point feature
- 27. From Excel into a Dashboard in Portal
- 28. Every Mission delivered as feature layer in Portal … mission data delivered as a feature layer by FME rendered identically using FME Python Caller and standard ESRI layer file. A web map is created in Portal, every picture is a point feature on the map and is hyperlinked for review purposes
- 29. UAS Data Products that can be processed NADIR Oblique Video Metadata Log files MEANINGFUL, RELIABLE and TIMELY DATA
- 30. Data Fusion # 4
- 31. P. Veenstra, TRC Pipelines and the AttributeCreator: Crunchy Magic for Accessing Previous and Next Rows
- 32. How do I… without programming? Challenge 1: Access the values in the previous or subsequent rows in a data set without scripting or programming? • Read data • Load data into a record-set or in-memory data set (array, list, dictionary) • Access data • Process data Challenge 2: Do this in a workspace that takes a set of points in the field (X,Y,Distance) and create a M-aware polyline with cumulative measure set for each vertex along the line? • Use the ‘AttributeCreator’
- 33. The Workspace • CSV file containing ID, X, Y, and Distance • Transform into M-Aware (measure) points and a polyline route • Each point feature will have a M value assigned to the vertex • Each polyline feature will have M values assigned to each vertex in the line AND will store the begin distance = 0 and the cumulative total distance = ? • Utilize AttributeCreator to access previous points in data set to calculate cumulative distance • Utilize MeasureSetter to make geometry M-Aware • PointConnector preserves the M values of the points and applies them to the polylines
- 34. The Workspace
- 35. Other magical artefacts … • Calculate true rhum (geodesic) distance between points … • Calculate simple 3D pythagorem distance between points … • Calculate the angle left or right from one vector to another … • Other than the expression editor in FME, it makes math for points on a line easy …