Simplifying the Complex: Serving Data from Pipeline Data Models
Download as PPTX, PDF0 likes1,082 views
The document discusses the challenges of accessing complex GIS pipeline data models for non-GIS professionals, emphasizing the difficulties of managing data stored in silos. It proposes a solution that involves building toolboxes using FME (Feature Manipulation Engine) to streamline data consumption and create a data warehouse that aggregates information from various sources. By employing a services-oriented, event-driven approach, the document highlights the importance of good data in fostering knowledge and problem-solving.
Simplifying the Complex: Serving Data from Pipeline Data Models
- 1. Simplifying the Complex: Serving data from Pipeline Data Models Peter Veenstra TRC
- 4. GIS is difficult, users want data … Non-GIS Professionals need access to data to do their jobs in XLS Industry Standard Pipeline Data Models are complex Linear Referencing Data stored in Silos – hard to synchronize or coordinate … Model, Load, As-Built – Re-build Silo 1 Silo 2
- 5. Big Gnarly Mess 1000’s if not 10’s of 10000’s miles of data overlaid on a single route. Linear events (in 100’s of tables) related to each other by common position along the pipeline but without formal relates. Potentially 7+ million segments to process. X Good Well Drained Test 1 - Current MAOP T MFL Type 1 Type 2b Type 2a X X X X X X Poor Poorly Drained Test 2 - Current
- 6. Solution Build Toolboxes using FME Move toolboxes to server. Let the data be consumed.
- 7. Toolboxes FME Toolkit for Pipeline Data Models (PDM) FME Toolkit for Linear Referencing (LRM) FME Toolkit for Database Integration (DIM) XYZM PL-AAA XYZM XYZM Sum Latest Merge
- 8. Move to FME Server What is Server? Why is it better than scheduled tasks and scripts? Planned Migration (build, then optimize)
- 9. Aggregation Data Warehouse FME Server Build a data ware-house aggregating data stored in different silos … - Shared coordinate plane - Hierarchy to Hierarchy translation - Dynamic Segmentation into massive table PODS Silo 1 Silo 2 Silo 3
- 10. X Good Well Drained Test 1 - Current MAOP T MFL Type 1 Type 2b Type 2a X X X X X X Poor Poorly Drained Test 2 - Current FME Server Services Bus Query Engine Tableau Hierarchy Finder Data Extractor Data Delta Finder/Synchronizer Data Health 2016: Status and Progress Tracker
- 11. A lot to get your head around … Subscribe, publish, consume … respond. Services-oriented rather than task-oriented Event-driven rather than procedural …
- 12. GIS is difficult, users want data … "Good data leads to good knowledge, and good knowledge solves problems." Good data eh!
- 13. Thank you! Peter Veenstra, TRC pveenstra@trcsolutions.com 816-820-7841