The Snap Solution to Turn Transport Data into EU Compliance
2 likes331 views
This document discusses an innovative semantic solution called SNAP to help transport stakeholders comply with EU Regulation 2017/1926, which requires the sharing of transport data through National Access Points using standard data formats. The SNAP solution uses a reference ontology derived from Transmodel to convert heterogeneous transport data into the required formats. It provides a semantic converter called Chimera that can customize data lifting from source formats into the ontology and lowering into target formats using mapping files. Current work involves finalizing the Transmodel ontology, piloting SNAP with different stakeholders and data, and investigating reversible mappings.
![6
EU Regulation 2017/1926: Status
• The first deadline for the provisioning of the first set of static transport data through NAPs is
December 1st, 2019
• According to [1], very few transport stakeholders are ready to provide their data and services
in compliance with the standards requested by the EU Regulation.
• To enable a data conversion process, they rely on their in-house support (which may lack
knowledge and skills related to the Regulation) or turn to external technology providers that
provide custom and often expensive solutions.
[1] Hendriks L., Jorna R., et Al. (2018). Monitoring and Harmonisation of National Access Points in Europe.
Available at: www.its-platform.eu/filedepot_download/1971/6491](https://image.slidesharecdn.com/semantics-presentation-190911053918/85/The-Snap-Solution-to-Turn-Transport-Data-into-EU-Compliance-5-320.jpg)
The Snap Solution to Turn Transport Data into EU Compliance
- 1. An Innovative Semantic Solution to Turn Transport Data into EU Compliance Marco Comerio (Cefriel)
- 2. 3 • The EU Regulation 2017/1926 • Requirements • Impact on Transport Stakeholders • Challenges & Opportunities for the Semantic Web Community • The SNAP solution • Ongoing & Next Steps Agenda
- 3. 4 • Objective: establish an interoperability framework enabling European industry players to make their business applications ‘interoperate’ and provides the travelers with a new seamless travel experience • Barriers: • insufficient accessibility of transport data • lack of service and data interoperability • Key enablers: • appropriate data sharing mechanism • making data interoperable by means of a common set of data exchange standards EU Mission: multimodal travel information service
- 4. 5 EU Regulation 2017/19261: Requirements • Each EU Member State is required to set up a National Access Point (NAP) • allowing access to static data (e.g., timetables, network topology, list of services offered at a station / airport) and dynamic data (e.g., delays, cancellations) related to different transport modes (air, train, bus, ferry, metro, tram, car/bike-sharing, car-pooling, etc.) • defined according to specific standard data formats such as NeTEx CEN / TS 16614 and SIRI CEN / TS 15531 • described using national application profiles (e.g., DCAT-AP) 1 https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX%3A32017R1926
- 5. 6 EU Regulation 2017/1926: Status • The first deadline for the provisioning of the first set of static transport data through NAPs is December 1st, 2019 • According to [1], very few transport stakeholders are ready to provide their data and services in compliance with the standards requested by the EU Regulation. • To enable a data conversion process, they rely on their in-house support (which may lack knowledge and skills related to the Regulation) or turn to external technology providers that provide custom and often expensive solutions. [1] Hendriks L., Jorna R., et Al. (2018). Monitoring and Harmonisation of National Access Points in Europe. Available at: www.its-platform.eu/filedepot_download/1971/6491
- 6. 7 EU Regulation 2017/1926: Impact on Transport Stakeholders Obligations • Provide datasets to the NAP compliant to the requested data formats • Provide metadata description of the datasets Challenge • Turn available data into the requested data formats and, in case, enrich them with additional data sources Benefit • Enhance the knowledge and the adoption of their transport offerings • Potential revenues from data trading Transport Authorities and Operators Transport Infrastructure Managers
- 7. 8 • Reference ontologies • unambiguously describe the operational aspects of the transport domain • Metadata profiles for transport datasets • harmonize the metadata description of datasets through a semantically-enhanced metadata profile • Semantic Web-based transport data converters • turn available transport data into specific data formats and, in case, enrich them with additional data sources • e.g., a specific (semantic) converter enables the translation of transportation schedule, geographic and fare information expressed in GTFS to a NeTEx specification preserving the original meaning EU Regulation 2017/1926: Challenges & Opportunities
- 8. Conceptualization • Acquire knowledge of the domain • Acquire knowledge of data formats/standards • Define the reference ontology Sharing • Define asset types • Define asset descriptors Governance • Identify actors, roles and associated tasks • Define of the lifecycle of each asset type • Define processes to automate the lifecycle management Conversion • Define models and rules to enable the conversion process Our Contributions
- 9. 10 • The EU Regulation 2017/1926 • Requirements • Impact on Transport Stakeholders • Challenges & Opportunities for the Semantic Web Community • The SNAP solution • Ongoing & Next Steps Agenda
- 10. 11 • Snap supports the process of compliance to the EU Regulation 2017/1926 providing • a reference ontology for the transport domain • a new solution supporting the mapping of heterogeneous transport data to the target standard requested by the EU Regulation • Snap is an EIT Digital Innovation Activity: a co-financed innovation project carried out by the EIT Digital Partners, with the goal of launching on the market a new product or service The Snap Project www.snap-project.eu
- 11. Customers’ Needs • Make transport data compatible with the standards established by EU Regulation 2017/1926 • Enrich own datasets with other data • Render own datasets interoperable with other data • Minimize the investment and time effort needed to achieve compliance, enrichment and/or interoperability Target Customers • Transport authorities • Transport operators • Infrastructure Managers Strategic Partners • NAP Management Board • Standardization bodies Snap Desirability
- 12. • A reference ontology for the transportation domain The Snap Solution Transmodel Ontology
- 13. • A reference ontology for the transportation domain The Snap Solution Transmodel Ontology Source Data Preparation • Open source components • FIWARE modules for data preparation
- 14. • A reference ontology for the transportation domain The Snap Solution Target Data Format Source Data Format Transmodel Ontology Source Data Preparation • Open source components • FIWARE modules for data preparation • Chimera converter for data conversion and enrichment through semantic Web technologies
- 15. • A reference ontology for the transportation domain The Snap Solution Target Data Format Source Data Format Transmodel Ontology LIFTING Specification LOWERING Specification Mapping File Mapping File Source Data Preparation • Customization services to enable interoperability in transport: • Mapping definition, enabling lifting to the reference ontology and lowering to target formats • Open source components • FIWARE modules for data preparation • Chimera converter for data conversion and enrichment through semantic Web technologies
- 16. 17 The Transmodel Ontology • The Snap solution is empowered by a reference ontology • derived from a sub-part of the CEN Transmodel1, the European reference data model for public transport information (e.g., timetable, fares) • under finalization by the Universidad Politecnica de Madrid using the LOT (Linked Open Terms) Methodology2 • The ontology is a starting point for a potential collaboration with the CEN TC278-WG3-SG4 that has formally defined the Transmodel 1www.transmodel-cen.eu 2http://lot.linkeddata.es/
- 17. 18 The Chimera converter • Semantic Interoperability allows avoiding the burden of point-to-point interoperability • The ontology provides an interoperability layer to guarantee decoupled conversion pipelines • Lifting and lowering processes can be defined once for each data format • Chimera provides two different options for the lifting process • RML mappings • Java annotations • Chimera provides two different options for the lowering process • Java annotations • Apache Velocity template
- 18. 19 LIFTING: RML Lifter • This block accepts mappings defined through the RML mapping language • RML extends R2RML (SQL) allowing also mappings from heterogeneous data sources (CSV, XML, JSON) through the definition of iterators. • Built-in support for data enrichment referencing mappings defined for sources with different data formats • This block uses the mapper from https://github.com/RMLio/rmlmapper-java
- 19. 20 LOWERING: Template Lowerer block • This block is able to apply an Apache Velocity template (https://velocity.apache.org) to extract data and build a structured document • SPARQL queries defined at the beginning of the template are executed and template variables are bound to their output • Such variables are then used to access the contents of the results while filling the template • This approach allows for high flexibility (custom queries and custom template structure) but is not specific or optimized for semantic applications
- 20. 21 LIFTING and LOWERING: Annotation-based Lifter and Lowerer • Annotation-based block exploits Java annotations to define mappings for Java classes Lifting: • marshall from source data format to Java instances • unmarshall from Java instances to RDF Lowering • unmarshall from RDF to Java instances • marshall from Java instances to target data format • This approach is useful for web-services where an interface descriptor is used (Java classes can be automatically generated and annotated).
- 21. 22 The Chimera converter • The Chimera converter can be customized to fulfill more complex requirements by means of a set of building blocks supporting: • Lifting • Data enrichment • Inference enrichment • Lowering • Chimera is based on Apache Camel and inspired by Enterprise Integration Patterns (EIP) • Other Apache Camel blocks can be integrated in the pipeline (e.g., I/O, un/marshall,…) https://camel.apache.org/
- 22. 23 The Chimera converter Lowering blockLifting block Source Message Target Message • A custom pipeline can be defined. For a basic conversion, a lifting and lowering block are required.
- 23. 24 The Chimera converter Source Message • A custom pipeline can be defined. For a basic conversion, a lifting and lowering block are required. • The RDF graph can be seen as a global variable shared among the blocks of the pipeline RDF graph (Reference Ontology terms)
- 24. 25 The Chimera converter Lifting block Source Message Mapping to Reference Ontology • A custom pipeline can be defined. For a basic conversion, a lifting and lowering block are required. • The RDF graph can be seen as a global variable shared among the blocks of the pipeline RDF graph (Reference Ontology terms)
- 25. 26 The Chimera converter Lifting block Data enricher Source Message Mapping to Reference Ontology RDF data • A custom pipeline can be defined. For a basic conversion, a lifting and lowering block are required. • The RDF graph can be seen as a global variable shared among the blocks of the pipeline RDF graph (Reference Ontology terms)
- 26. 27 The Chimera converter Lifting block Data enricher Source Message Inference enricher Mapping to Reference Ontology RDF data • A custom pipeline can be defined. For a basic conversion, a lifting and lowering block are required. • The RDF graph can be seen as a global variable shared among the blocks of the pipeline RDF graph (Reference Ontology terms) Reference Ontology
- 27. 28 The Chimera converter Lowering blockLifting block Data enricher Source Message Inference enricher Mapping to Reference Ontology Target Message Mapping from Reference Ontology RDF data • A custom pipeline can be defined. For a basic conversion, a lifting and lowering block are required. • The RDF graph can be seen as a global variable shared among the blocks of the pipeline Reference Ontology Export RDF graph
- 28. 29 SWOT Analysis STRENGTHS • Flexibility • Reusability WEAKNESSES • Handmade mappings (no tooling) • Semantic/Logic skills required OPPORTUNITIES • Conceptualization of the domain • Applicability to different domains • Semantic NAPs with Transmodel RDF data THREATS • Lack of domain knowledge leads to bad ontologies and mappings INTERNAL EXTERNAL POSITIVE NEGATIVE
- 29. 30 • The EU Regulation 2017/1926 • Requirements • Impact on Transport Stakeholders • Challenges & Opportunities for the Semantic Web Community • The SNAP solution • Ongoing & Next Steps Agenda
- 30. 31 Ongoing & Next Steps • Finalization of the Transmodel ontology • Expected to be published in October on https://github.com/oeg-upm/transmodel-ontology • 3 Pilots of the Snap solution • covering different metropolitan areas (Madrid, Milano and Genova) • different stakeholders (transport authorities, transport operators, infrastructure managers) • different transport modes (air, bus, metro) • different source data formats (GTFS, proprietary format) • Reversible mappings: investigation of the feasibility to exploit a ReversibleRMLMapper capable of using RML mappings defined for the lifting block also for the lowering process
- 31. 32 Marco Comerio (Cefriel) marco.comerio@cefriel.com For more info: Website: www.snap-project.eu Twitter: @snapEUproject Thank You!