rTest, a Testing Tool for FME Workspaces
0 likes1,694 views
This document introduces rTest, an open source testing tool for FME workspaces. It was created to test large, generic FME workspaces for a geographical dataset quality control application called Qualigeo. rTest allows describing test scenarios and checking workspace outputs automatically through HTML reports. This improves reliability, speeds development, and reduces maintenance costs compared to traditional manual testing. The document outlines rTest's capabilities and limitations, and argues that testing tools are essential for large FME projects.
rTest, a Testing Tool for FME Workspaces
- 1. rTest : a testing tool for FME Workspaces Olivier Gayte
- 2. Olivier Gayte Founder of Veremes French FME reseller Platinum Partner
- 3. Our problem Qualigéo FME-based application for geographical datasets quality control Big generic FME workspaces FME 2010 Old python API (pyfme) Dozens of Transformers Several source parameters and hundreds of use cases
- 4. Our problem Qualigéo How to upgrade it? How to improve it? How to optimize it? and how to be sure to get a correct result? with better performances?
- 5. Traditional testing method Log analysis Success/Failure Feature count Data Inspector Visual inspection Third-part tools Sql-plus ArcMap Manual control Human time-consuming Doesn’t see some errors Inefficient with generic workspaces handling several source parameters
- 6. Looking for a testing tool • Java • JavaScript • PHP • Python • FME • Ok. We will do it!
- 7. Requirements specification Our ideal testing tool • Output independent (number, format, method) • Non intrusive • FME-version independant • Performance testing • Reliable • Compatible with FME Server
- 9. How does it work? • Describes the workspaces and the associated tests • Runs workspaces • Checks results • Writes reports • Html report • Success/Failure • Performance scenarioPlayer.fmwScenario Report
- 12. Hello World! A very simple use case Extract country information from a point into a Spatialite dataset
- 13. Tests to be made • Lat Long – In a simple polygon (Switzerland) – In a donut (South Africa) – In an aggregate (Canada) – With negative values – On a border – On a triple point – With null value – With 50 digits – With 0,0 – Ouside of extent – …
- 14. The scenario
- 17. Feedback • Qualigeo V2 – Faster (40%) – Increased reliability – Easier maintenance – More frequent releases
- 18. rTest today
- 19. Open Source • Open Source – https://github.com/veremes/rtest • Current version – FME 2017
- 20. Limitations • Checked formats – Oracle, PostgreSql, Shp, Spatialite • Checking operators – regExp (+ sql), featuresCount, featuresCountByFtAndGeometry, attributeValue • …but easy to improve – checkerXML.fmw – rTest.xsd
- 21. General considerations on FME workspaces development
- 22. Cost of a FME Project Number of days For developing Maintenance cost Standard FME Project FME Project with a testing tool
- 23. Organization for a big FME project
- 24. Conclusion Testing tools are essentials to any IT Project rTest is a proposal for the FME community You can use it, improve it or build a new one from scratch but… …we need a testing tool!