Dive in with Databases – FME Summer Camp 2018
1 like2,726 views
The FME Summer Camp focused on the latest updates and enhancements in database handling, including new readers and writers for various formats such as Teradata and SAP HANA. Key strategies for data load validation and risk reduction were discussed, emphasizing the usage of staging tables and transaction interval adjustments. The session also highlighted the migration to the ESRI Utility Network and offered resources for further learning on database integration.
Dive in with Databases – FME Summer Camp 2018
- 1. – FME Summer Camp – Dive in with Databases Tia McKillop Mark Stoakes
- 2. Database Explorer Agenda ● What’s New in FME 2018 ○ New Formats ○ Harmonization ○ Format Enhancements ○ New Transformer ● Demo: Let the database do the work!
- 4. New Formats ● Teradata Reader & Writer ● SAP HANA Reader & Writer ● Geodatabase on SAP HANA Reader & Writer ● Esri Geodatabase Mosaic Dataset Writer Welcome Campers
- 6. Harmonization ● File Geodatabase ○ Writer Feature Type: fme_db_operation ○ Reader Feature Type: WHERE Clause ○ Added SQL Before/After ● ArcSDE Geodatabase ○ Reader Feature Type: WHERE Clause ● SQL Server Reader ○ Added SQL Before/After ● ODBC / JDBC ○ Added Before/After SQL FME Camp
- 7. Updates to Existing Formats ● SQL Server, DB2 ○ XML data type support ● Oracle 12c, MySQL ○ JSON data type support ● GeoPackage ○ Curved geometry support ● Geodatabase ArcSDE ○ Selective Attributes Support
- 8. ➔ Why use the FeatureJoiner? ➔ It’s FASTER than FeatureMerger ➔ It’s SIMPLER than FeatureMerger ➔ It uses STANDARD language New Transformer
- 9. Joining with the FeatureJoiner
- 10. Mode Venn Diagram Joined UnjoinedLeft UnjoinedRight Left All Matches and Unmatched Left Features Nothing Unmatched Right Features Inner All Matches ONLY Unmatched Left Features Unmatched Right Features Full Everything Nothing Nothing
- 11. TRANSFORMER UPDATE FeatureWriter Check out the FeatureWriter Doc
- 12. LET THE DATABASE DO THE WORK
- 13. Reducing Data Load Risk: Recovery Strategies Don’t let your database loads stall… David Reksten, INSER SA - Uses staging tables and validates those prior to loading to production. Erik Bodewitz, SpatialBiz - Set FME transaction interval to 1 to easily trace the problem feature, OR - Set transaction interval to a large value to rollback the entire load, OR - Uses staging tables and validates those prior to loading to production. Dave Ellerbeck, Global Informations Systems - Uses Ignore Failed Features on Geodatabase writer. - Uses SQLExecutor to manage parent/child inserts. - Barbara Stephan, Landesbetrieb Geoinformation und Vermessung - Loads ~40 tables using staging tables and validates those prior to using FME to mirror those into production tables.
- 14. Let Your Database Do The Work & Data Load Validation Add additional data to database tables. Validate load, reduce risk.
- 15. DEMO: LET THE DATABASE DO THE WORK
- 16. Esri Utility Network Management Extension Migrate data from ArcGIS to Utility Network Migration Workspace: - Schema Mapping - Build Assemblies (if required) - Write to Asset Package Staging Geodatabase Load Data into Utility Network (Esri Data Loader)
- 17. Resources: More information on FME & Databases ● Tutorials on the FME Knowledge Center ○ knowledge.safe.com ■ Oracle ■ PostGIS ■ SQL Server ■ Esri Geodatabase ■ Smallworld ○ Let the Database do the Work fme.ly/dbwork ● Webinars ○ www.safe.com/webinars ● ArcGIS Utility Networks
- 18. Poll Question: Would you be interested in more resources on how to work with: 38% - Oracle | 37% - PostGIS | 58% - SQL Server 74% - Esri Geodatabase | 4% - Smallworld
- 19. Join our next camp activity on July 25: Web Data & Reporting Zipline
- 20. Q&A