1-5 stars: Metadata on the Openness Level of Open Data Sets in Europe
Download as PPTX, PDF1 like1,074 views
The document evaluates the openness levels of open data sets in Europe, highlighting the need for improved reusability and better metadata quality to facilitate data access. It analyzes the publicdata.eu catalog and identifies challenges such as linguistic diversity, consistency in naming, and the prevalence of proprietary formats. Despite a significant number of datasets having some form of open license, many remain in non-machine-readable formats, hindering effective reuse.
1-5 stars: Metadata on the Openness Level of Open Data Sets in Europe
- 1. 1-5 stars: Metadata on the Openness Level of Open Data Sets in Europe Sébastien Martin, Muriel Foulonneau, Slim Turki
- 2. Context & Objectives • • • • Level of reuse of open data is still disappointing. Development of open data requires a better reusability of data. Degree of openness is a key success factor. Catalogs listing data have a crucial role. Analyse PublicData.eu catalogue (i) identify the quality of a sample of metadata properties, which are critical to enable data reuse (ii) study the stated level of data openness. 21/11/2013 1-5 stars: Metadata on the Openness Level of Open Data Sets in Europe 2
- 3. PublicData.eu • • Many local and national portals to provide access to public sector open datasets - 114 EU catalogues on datacatalogs.org Gather datasets across geographic and institutional boundaries PublicData.eu • • • • • • pan-European catalogue launched under the FP7 LOD2 project. aggregates data from CKAN open data catalogues all over Europe. collects data from 26 sources 1st to be published in Europe in 2011 data beyond the European Union, e.g., Serbian datasets. not exhaustive, it represents a unique aggregation of European datasets. • • 17.027 datasets UK: largest provider 21/11/2013 3
- 4. Methodology Descriptions of datasets collected in May 2013 236 distinct dataset properties identified, partially due to • • linguistic diversity; some providers adapt property names in their language problems of consistency in naming (upper / lower case, spaces / underscore for a single field). Major challenge to understand the content of the PublicData.eu Data collected and analysed to identify information made available on data openness and reusability in particular the licensing conditions and the data formats. 21/11/2013 1-5 stars: Metadata on the Openness Level of Open Data Sets in Europe 4
- 5. Tim Berners-Lee’s evaluation scale ★ Available on the web (whatever format) but with an open license, to be Open Data ★★ Available as machine-readable structured data ★★★ 2 + non-proprietary format ★★★★ ★★★★★ 21/11/2013 3 + Use open standards from W3C (RDF and SPARQL) to identify things 4 + Link your data to other people’s data to provide context 1-5 stars: Metadata on the Openness Level of Open Data Sets in Europe 5
- 6. ★ Data Licences 13.535 / 17.027 datasets have at least 1 license indication 12.470 datasets can be considered having some form of open license 73,24% 769 datasets have a Creative Commons license Significant number of datasets have a national license: • • • apie v2 to publish information created by French public authorities UK-crown which “covers material created by civil servants, ministers and government departments and agencies” in the UK, UK Open Government License 128 datasets with an explicitly closed license 21/11/2013 1-5 stars: Metadata on the Openness Level of Open Data Sets in Europe 6
- 7. ★★ Machine readable format • Facilitates data reusability • 4.051 / 17.027 with content_TYPE • 11.285 with at least one indication about format • 56 datasets in RDF • Dominant proportion of spreadsheets type’s formats Distribution of formats 40% not a machine readable format 34% of datasets available in a machine readable format machine readability cond. for openness levels of 2★ and > 21/11/2013 1-5 stars: Metadata on the Openness Level of Open Data Sets in Europe 7
- 8. ★★★ Use of non-proprietary formats Creates ambiguities as the openness nature of formats can be debated in some cases: • • Certain formats are proprietary but their specifications are open. Some formats have been open at a certain point of time but additions and further evolutions remain proprietary In many cases, value of property was too vague to determine whether the format was or not proprietary. It was possible to identify: • • For 49% of the datasets, a non-proprietary format For 21% a proprietary format. Use of proprietary formats is a critical issue for improving the level of openness of datasets. 21/11/2013 1-5 stars: Metadata on the Openness Level of Open Data Sets in Europe 8
- 9. ★★★★ Use of open standards from W3C Including HTML, XML, and RDF in particular. • XML-based formats may be entirely independent from W3C (e.g. KML) Availability in W3C standards: 9,5% of datasets Availability in XML based formats: 10% Information remains unknown in most cases 21/11/2013 1-5 stars: Metadata on the Openness Level of Open Data Sets in Europe 9
- 10. ★★★★★ Linked data Linked data are only mentioned in the description of a single dataset (Brandweer Amsterdam-Amstelland Uitrukberichten) for which the format is described as “linked data api, rdf json”. 58 datasets mention RDF (or RDFa) as a format or content type, i.e., 0,34%. 21/11/2013 1-5 stars: Metadata on the Openness Level of Open Data Sets in Europe 10
- 11. Level of openness (1/2) 6.891 / 17.027 datasets show at least one information about their degree of openness. All come from Data.gov.uk (8 689 datasets) For a majority of datasets, the level of openness is unknown. • 21/11/2013 Coherent with lack of licensing information without which it is impossible to conclude on even ★ openness level. 1-5 stars: Metadata on the Openness Level of Open Data Sets in Europe Distribution of openness levels in UK datasets 11
- 12. Level of openness (2/2) Approximate level of openness derived from licensing and format properties • • 73,24% of the datasets should have ★ or above. Reference to 5★ should take into consideration linkages, cannot be inferred from dataset metadata. Level of openness according to Format and License related properties Data openness mainly related to 1st level of compliance: licensing issue. • 21/11/2013 Data providers have clearly not focused on publication of data in reusable formats. 1-5 stars: Metadata on the Openness Level of 12 Open Data Sets in Europe
- 13. Conclusion • Limited openness of datasets advertised as open data • Heterogeneity of associated metadata Difficulty for reusers to (i) discover datasets, despite the creation of large catalogues of datasets, and to (ii) effectively reuse machine readable and contextualized data. ★ may be sufficient to ensure transparency of gov. action, facilitating reuse of data through services is not served below 2★ Confirmed risks regarding major challenges that data providers have to face: (i) language barrier and (ii) lack of consistency of metadata. Harmonization of practices, training and tools necessary to ensure that datasets are available in relevant formats. 21/11/2013 1-5 stars: Metadata on the Openness Level of Open Data Sets in Europe 13
- 14. 1-5 stars: Metadata on the Openness Level of Open Data Sets in Europe Sébastien Martin, Muriel Foulonneau, Slim Turki Contact: muriel.foulonneau@tudor.lu
Editor's Notes
- #6: The study uses the Tim Berners-Lee’s five star evaluation scale.
- #7: The one star openness level depends upon data licenses. Licensing information can be found in 10 distinct metadata properties, i.e., licence, License, licence_url, License_details, License_ID, License_summary, License_title, License_uri, License_url, and mandate.
- #8: The two star level depends upon the format in which the data is made available.