Harnessing the Power of FME for Near Real-Time Data Transformations
2 likes641 views
The document details the Iowa Department of Transportation's use of FME technology for real-time data transformations in winter operations, particularly for tracking snowplows and road conditions. It highlights the implementation of innovative tools like Track-a-Plow and a Winter Cost Calculator, which help monitor operational efficiency and cost analysis. Future plans include enhancing data integration and developing predictive analytics for better winter service management.
Harnessing the Power of FME for Near Real-Time Data Transformations
- 1. Harnessing the Power of FME for Real-Time Data Transformations David Runneals
- 2. David Runneals • Student at Northwest Missouri State – GIS – Emergency & Disaster Management • GIS Intern – Iowa DOT aka “The Doctor” formerly known as “The FME Ninja” Photo Source: The Wand Company
- 3. Quick Stats • FME Desktop – 10 floating licenses • FME Server – 6 engines
- 4. Iowa DOT • Winter – Snowplows – Road Conditions • Year Round Operations – 511 Events – ITS Devices – Waze – ETL Processes FME Workspaces
- 6. Quick Stats Roadways: 24,122 lane miles – 3,687 lane miles of Interstates/ramps – 9,403 centerline miles of other primary routes – 3 winter service levels Equipment: • 892 Snowplows • 53 Graders • 133 Endloaders • 12 Snowblowers • 76 Snowblower attachments • 12 Tow Plows Winter Operations Material: • 18-24 million gallons of brine • ~150,000 tons of salt applied
- 7. 10% salt reduction = $1.4 million in savings Why track plows?
- 8. Laborious record keeping Auditing real use of materials Response consistency (and data for analysis of storm response/material usage and future research aka cool projects) Why track plows?
- 10. Winter Operations: 2010-13 • Started AVL project in 2010 • 1 Ping/Minute per truck • FME Server did ETL between vendor and Iowa DOT databases • Utilized GeoCortex for internal visualization and reporting
- 11. Winter Operations: 2013-14 – Data stored in KML files on Azure – App hosted on AGOL • February 2014: Launched Plow Cams • January 2014: Track-a-Plow Launched
- 12. Winter Operations: 2014-15 • Track-a-Plow Updated – Changed writers to AGOL Feature Services • PlowCams increased to ~420 phones
- 13. Winter Operations: 2015-16 • Track-a-Plow Updated – 2 different AVL vendors – Performance Improved • Added AGOL Feature Collections for high availability & performance • Winter Cost Calculator Soft Launch – Bugs and calculations were still being worked out
- 14. Winter Operations: 2016-17 • Track-a-Plow Updated – All trucks now on new AVL technology • 10 second AVL pings (15 second material/data pings) • IT does data pull from vendor – Added road conditions from ND, SD, NE, KS, MO, IL, WI, and MN for situational awareness – Added MN DOT Plow Cams • Winter Cost Calculator Launched
- 17. Place your screenshot here Track-a-Plow • First in the United States! • 8+ other states have followed (We still have the cleanest end-client UI though) • 3 other states have plow cams • Still the only one state that makes our real-time data free and easily accessible with no logins!
- 18. Winter Operations: Stats 40,942 Views 301,719 Views 279,659 Views 0 5 10 15 20 25 30 35 Winter 2014-2015 Winter 2015-2016 Winter 2016-2017 Millions Usage Statistics TrackaPlow Views Plow Location Data Service Requests Plow Cam Data Service Requests
- 20. Winter Cost Calculator • Helps users understand the cost of keeping Iowa’s state-owned roads clear during winter – Provides material, labor, equipment, and total costs – See costs on a statewide, county, major metro, road segment, or custom level – Tracks costs for the last 48 hours
- 21. THE WORKSPACE Winter Cost Calculator
- 22. Place your screenshot here Winter Cost Calculator 2017 State IT Innovation of the Year AS FEATURED ON
- 24. The Future of Winter Operations • Staff/COTS for analysis • More data to the public • More data integration • Plow and DMS notification • Predictive analytics – “Based on past data this segment is expected to be partially covered (85% confidence) from 1am to 5am.”
- 26. Traffic Operations FME ETL • 2014 (KML) – 511 point data – Iowa Road Conditions • 2015 (AGOL) – Real-time Waze data – ITS devices (Cameras/DMS Signs) – Inrix “abnormal traffic” – Archiving Historic Road Conditions
- 27. Traffic Operations FME ETL • 2016 – Archiving Waze data – Providing monthly reports of Waze and 511 entries to the Traffic Management Center • 2017 – Surrounding States Road Conditions
- 28. Traffic Operations: Stats 0 5 10 15 20 25 2014 2015 2016 Service Requests (Millions) Data Service Usage Statistics 511 Events Traffic Cameras DMS (Digital Signs)
- 32. Place your screenshot here Bridge Condition Story Map Goal: Provide more timely bridge data as it changes during the year to allow customers to make more informed decisions.
- 33. Best Practices & Lessons Learned
- 34. Growing Pains • App Views – 15,028 & 22,249 • Road Condition FS – 1.9 mil. & 2.3 mil. – Hit a peak of 268,000 requests in a single hour 11/20 – 11/21/2015 Statewide Event
- 36. ArcGIS Online Data Services Real-Time High Availability • Ensure public facing feature services are non-editable – Python scripts to enable/disable editing – Use “hosted feature layer views” (3/2017 release) • Don’t use filters on layers in maps/apps Best: Use Feature Collections!
- 37. Open Data = Power Photo Source: Meme Generator
- 38. THANK YOU! David Runneals | david@runneals.com runneals/GISProjects @RunnealsGIS