Linked Statistical Data 101
Download as PPTX, PDF1 like554 views
This document discusses linked statistical data and its benefits. It provides an overview of key concepts like open data, linked data, and the W3C RDF DataCube specification. It also presents a case study of a statistical office in Aragon, Spain that has published local government data as linked open data. Publishing data this way allows for easier reuse by both internal and external developers. It facilitates integration with other datasets and enables complex queries across multiple sources. Overall, representing statistical data using semantic web standards like RDF DataCube provides advantages for data sharing and reuse.
![What is Open Data?
• Open data is data that can be freely used, re-used
and redistributed by anyone - subject only, at most, to
the requirement to attribute and share alike
• Key aspects:
- Availability and access: the data must be available as a
whole and at no more than a reasonable reproduction cost,
preferably by downloading over the Internet. The data must
also be available in a convenient and modifiable form.
- Re-use and redistribution: the data must be provided
under terms that permit re-use and redistribution including
the intermixing with other datasets.
- Universal participation: everyone must be able to use, re-
use and redistribute - there should be no discrimination
against fields of endeavour or against persons or groups
[source: Open Data Handbook, http://opendatahandbook.org/en/what-is-open-data/ ]](https://image.slidesharecdn.com/euc1601ocorchopresentation-170119101417/85/Linked-Statistical-Data-101-4-320.jpg)
![Relevant Legislation. Europe and Spain
• Open Access Initiative (2001). Scientific information; > 510 orgs
• Aarhus Convention (1998). Right to participate and access; 41
countries and the EU
• PSI Directives. PSI reuse (2003/98/EC and 2013/37/UE)
• Convention about access to official documentation (2009)
- 12 countries
• Law 37/2007. PSI reuse (transposition of directive 2003/98/EC)
- Modified in law 18/2015 (BOE 10/07/2015, directive 2013/37/UE )
• Law 11/2007. Citizen access to public services, and rights to good
quality services
• RD 4/2010 Esquema Nacional de Interoperabilidad
- Open standards, technology neutral, open source
• RD 1495/2011 It develops Law 37/2007 for national agencies
• Norma Técnica de Interoperabilidad (19/02/2013, BOE 4/3/2013)
[source: based on a presentation from Antonio Rodríguez Pascual (CNIG)]](https://image.slidesharecdn.com/euc1601ocorchopresentation-170119101417/85/Linked-Statistical-Data-101-5-320.jpg)
![INFRASTRUCTURE
MICRODATA
MACRODATA
i
Cartography, streets,
directories, codes…
ANALYSTS
JOURNALISTS
CITIZENS
RESEARCHERS
METADATA
Which type of data and which (re)users?
[source: Alberto González Yanes (ISTAC)]](https://image.slidesharecdn.com/euc1601ocorchopresentation-170119101417/85/Linked-Statistical-Data-101-17-320.jpg)
Linked Statistical Data 101
- 1. Linked Statistical Data 101 ESS Workshop on dissemination of official statistics as open data 18-19 January 2017, Malta Oscar Corcho Escuela Técnica Superior de Ingenieros Informáticos Universidad Politécnica de Madrid Campus de Montegancedo sn, 28660 Boadilla del Monte, Madrid http://www.oeg-upm.net/ ocorcho@fi.upm.es
- 2. Contents • Foundations of (Linked) Open Data - For public administrations, in general - For statistical offices, in particular • Linked Statistical Data by example - A use case from IAEST (Aragón Statistical Office) • A bit of technical background - W3C RDF DataCube • Preparing the discussion on benefits for different types of stakeholders 2
- 3. Contents • Foundations of (Linked) Open Data - For public administrations, in general - For statistical offices, in particular • Linked Statistical Data by example - A use case from IAEST (Aragón Statistical Office) • A bit of technical background - W3C RDF DataCube • Preparing the discussion on benefits for different types of stakeholders 3
- 4. What is Open Data? • Open data is data that can be freely used, re-used and redistributed by anyone - subject only, at most, to the requirement to attribute and share alike • Key aspects: - Availability and access: the data must be available as a whole and at no more than a reasonable reproduction cost, preferably by downloading over the Internet. The data must also be available in a convenient and modifiable form. - Re-use and redistribution: the data must be provided under terms that permit re-use and redistribution including the intermixing with other datasets. - Universal participation: everyone must be able to use, re- use and redistribute - there should be no discrimination against fields of endeavour or against persons or groups [source: Open Data Handbook, http://opendatahandbook.org/en/what-is-open-data/ ]
- 5. Relevant Legislation. Europe and Spain • Open Access Initiative (2001). Scientific information; > 510 orgs • Aarhus Convention (1998). Right to participate and access; 41 countries and the EU • PSI Directives. PSI reuse (2003/98/EC and 2013/37/UE) • Convention about access to official documentation (2009) - 12 countries • Law 37/2007. PSI reuse (transposition of directive 2003/98/EC) - Modified in law 18/2015 (BOE 10/07/2015, directive 2013/37/UE ) • Law 11/2007. Citizen access to public services, and rights to good quality services • RD 4/2010 Esquema Nacional de Interoperabilidad - Open standards, technology neutral, open source • RD 1495/2011 It develops Law 37/2007 for national agencies • Norma Técnica de Interoperabilidad (19/02/2013, BOE 4/3/2013) [source: based on a presentation from Antonio Rodríguez Pascual (CNIG)]
- 6. An Explosion of Open Data Portals
- 7. Open Data and how to publish it 1) In a posterboard - For those with a lot of free time available - Or those who happen to be there at the right time Adapted from: Antonio Rodríguez Pascual (IGN)
- 8. Open Data and how to publish it 2) On a Web page or mobile app - For people, but not downloadable Adapted from: Antonio Rodríguez Pascual (IGN)
- 9. Open Data and how to publish it 3) In files - These can be downloaded and use by humans in information systems (XML, HTML, CSV, GTFS, etc.) - Luckily, it is not a scanned PDF Adapted from: Antonio Rodríguez Pascual (IGN)
- 10. Open Data and how to publish it 4) Via Web Services - They can be used by systems (sometimes persons) - They allow generating added value - Ease of integration in the application logic Adapted from: Antonio Rodríguez Pascual (IGN)
- 11. All together…, Shaken, not stirred…
- 12. What is Linked Data? 1. Use URIs to identify rsources 2. Use HTTP URIs, so that they can be found 3. Use de-referenceable URIs, that is, provide useful data (RDF, JSON, SPARQL) 4. Include links to other URIs. • http://www.w3.org/DesignIssues/ LinkedData.html
- 13. Open Data and how to publish it 5) Via APIs (semantically enhanced) and linked - To be used by systems (and sometimes persons) - It allows generating added-value services - Standardised formats (JSON, JSON-LD, RDF) - Standardised models (vocabularies, ontologies)
- 14. Difficult to reuse √ Reusable. Not open √ Reusable, open Difficult to link together √ Reusable, open, complete, easier to link Data representation formats And many more: JSON, JSON-LD, Shapefiles, KMZ, KML, PC-Axis, etc.
- 15. Recap: The 5-star categorisation from TBL
- 16. Contents • Foundations of (Linked) Open Data - For public administrations, in general - For statistical offices, in particular • Linked Statistical Data by example - A use case from IAEST (Aragón Statistical Office) • A bit of technical background - W3C RDF DataCube • Preparing the discussion on benefits for different types of stakeholders 16
- 17. INFRASTRUCTURE MICRODATA MACRODATA i Cartography, streets, directories, codes… ANALYSTS JOURNALISTS CITIZENS RESEARCHERS METADATA Which type of data and which (re)users? [source: Alberto González Yanes (ISTAC)]
- 18. Our use case: Aragón • IAEST - Instituto Aragonés de Estadística • Good open data ecosystem - Aragón Open Data • http://opendata.aragon.es/ - Zaragoza • http://datos.zaragoza.es/ 18
- 19. Reports and templates from Oracle BI Current Web application for local statistics Statistics about municipalities
- 20. Statistics about municipalities • At IAEst Web - http://www.aragon.es/DepartamentosOrganismosPublicos/In stitutos/InstitutoAragonesEstadistica/AreasGenericas/ci.Esta disticaLocal.detalleDepartamento • At OpenDataAragón - http://opendata.aragon.es/catalogo/edificios-superficie-y- vivienda-comarcas
- 21. Reports and templates from Oracle BI Current Web application for local statistics What have we done?
- 22. SPARQL Elda Linked Data Transformation process API Publication process General architecture This is not the purpose of my talk https://github.com/aragonopendata/local-data-aragopedia
- 23. URIs for datasets • Let’s look for the dataset on “Number of homes per owner per municipality” - Número de hogares por tipo de propietario por municipio • The dataset has a URI - http://opendata.aragon.es/recurso/iaest/dataset/01- 010013TM
- 24. What is behind that URI? 24 This is not the purpose of my talk
- 25. URIs for each observation • And now we can point to specific observations in this dataset - In 2001, the number of buildings owned by one person in the municipality of Ilche • http://opendata.aragon.es/recurso/iaest/observacion/01- 010013TM/00794aab-964f-35c7-8e7c-156c9bc60133 25
- 26. URIs for each observation 26
- 27. And links to other URIs in Aragón • The municipality of Ilche - http://opendata.aragon.es/recurso/territorio/Municipio/Ilche - This information is owned by another department of the Government of Aragón 27
- 28. And links to codelists • Types of owners - http://opendata.aragon.es/kos/iaest/clase-de-propietario • The community • A person • A society • A public organisation 28
- 29. SPARQL endpoint The women population in Zaragoza in the age range of 0-15 years growed until 2013 and then reduced select distinct ?year ?personas where { ?x a qb:Observation . ?x qb:dataSet <http://opendata.aragon.es/recurso/iaest/dataset/03-030005TM> . ?x <http://purl.org/linked-data/sdmx/2009/dimension#refPeriod> ?year . ?x <http://purl.org/linked-data/sdmx/2009/dimension#refArea> <http://opendata.aragon.es/recurso/territorio/Municipio/Zaragoza>. ?x <http://opendata.aragon.es/def/iaest/dimension#edad-grandes-grupos> <http://opendata.aragon.es/kos/iaest/edad-grandes-grupos/0-a-15> . ?x <http://opendata.aragon.es/def/iaest/dimension#sexo> <http://opendata.aragon.es/kos/iaest/sexo/mujeres>. ?x <http://opendata.aragon.es/def/iaest/medida#personas> ?personas . } ORDER BY ?year Examples at https://github.com/aragonopendata/local-data-aragopedia/blob/master/consultas.md
- 30. Contents • Foundations of (Linked) Open Data - For public administrations, in general - For statistical offices, in particular • Linked Statistical Data by example - A use case from IAEST (Aragón Statistical Office) • A bit of technical background - W3C RDF DataCube • Preparing the discussion on benefits for different types of stakeholders 30
- 34. Observations in a dataset 34 qb:DataSetqb:Observation qb:dataSet rdf:type iaest-data:01-010003M/22001/030-045 aod:Abiego sdmx:refArea Iaest-codelist:superficie030-045 iaest:superficieUtil “1”^^xsd:int Iaest:numeroHogares iaest:01-010003M qb:dataSet rdf:type
- 36. Describing the dataset 36 qb:DataSet qb:DataStructureDefinition qb:ComponentSpecification qb:ComponentProperty sdmx:refArea iaest:superficieUtil qb:structure qb:component qb:componentProperty rdf:type rdf:type iaest:01-010003M iaest--dsd:01-010003M qb:structure qb:component qb:measure iaest:numeroHogares qb:dimension qb:dimension rdf:typerdf:type
- 37. Dimensions 37 qb:DataSet qb:DataStructureDefinition rdfs:range qb:concept qb:DimensionProperty qb:MeasureProperty qb:Observation esadm:MunicipioIaest:SuperficieUtil qb:ComponentSpecification qb:ComponentProperty rdfs:subClassOf qb:dataSet iaest:numeroHogaressdmx:refAreaiaest:superficieUtil rdf:type rdf:type rdfs:range xsd:int rdfs:range qb:structure qb:component qb:componentProperty
- 39. Contents • Foundations of (Linked) Open Data - For public administrations, in general - For statistical offices, in particular • Linked Statistical Data by example - A use case from IAEST (Aragón Statistical Office) • A bit of technical background - W3C RDF DataCube • Preparing the discussion on benefits for different types of stakeholders 39
- 40. Why Linked Statistical Data? (I) • Facilitate data (re)use by developers outside our organisation • Data access APIs (according to standards) • Do they prefer CSVs, PCAxis, SDMX, RDF? • Fine-grained data granularity (refer to specific facts) • Integration with other data sources from other public or private organisations - E.g., Government of Aragón for municipalities • Allow for queries across datasets - E.g., tell me how many municipalities may benefit from this funding that I am making available with these restrictions: number of registered companies lower than 5 and unemployed population higher than 15%
- 41. Why Linked Statistical Data? (II) • Internal benefits as well - Codelists are made available and more visible internally - Methodology and metadata explicitly described as part of the RDF DataCube data (e.g., reference years in datasets) 41
- 42. Linked Statistical Data 101 ESS Workshop on dissemination of official statistics as open data 18-19 January 2017, Malta Oscar Corcho Escuela Técnica Superior de Ingenieros Informáticos Universidad Politécnica de Madrid Campus de Montegancedo sn, 28660 Boadilla del Monte, Madrid http://www.oeg-upm.net/ ocorcho@fi.upm.es
Editor's Notes
- #3: Linked Statistical Data 101 In this introductory talk we will discuss the main foundations for the application of Linked Data principles into official statistics. We will briefly introduce what Linked Data is, as well as the main principles, languages and technologies behind it (URIs, RDF, SPARQL). We will also discuss about the different formats in which data can be made available on the Web (e.g., RDF Turtle, JSON-LD, CSV on the Web). We will then move into providing a detailed presentation, with step by step examples based on existing Linked Statistical Data sources, of the W3C recommendation RDF DataCube, which is the basis for the dissemination of statistical data as Linked Data. Finally, we will provide some examples of applications, and the opportunities that this approach offers for the development of the proofs of concepts selected by Eurostat and to be discussed during the meeting.
- #4: Linked Statistical Data 101 In this introductory talk we will discuss the main foundations for the application of Linked Data principles into official statistics. We will briefly introduce what Linked Data is, as well as the main principles, languages and technologies behind it (URIs, RDF, SPARQL). We will also discuss about the different formats in which data can be made available on the Web (e.g., RDF Turtle, JSON-LD, CSV on the Web). We will then move into providing a detailed presentation, with step by step examples based on existing Linked Statistical Data sources, of the W3C recommendation RDF DataCube, which is the basis for the dissemination of statistical data as Linked Data. Finally, we will provide some examples of applications, and the opportunities that this approach offers for the development of the proofs of concepts selected by Eurostat and to be discussed during the meeting.
- #17: Linked Statistical Data 101 In this introductory talk we will discuss the main foundations for the application of Linked Data principles into official statistics. We will briefly introduce what Linked Data is, as well as the main principles, languages and technologies behind it (URIs, RDF, SPARQL). We will also discuss about the different formats in which data can be made available on the Web (e.g., RDF Turtle, JSON-LD, CSV on the Web). We will then move into providing a detailed presentation, with step by step examples based on existing Linked Statistical Data sources, of the W3C recommendation RDF DataCube, which is the basis for the dissemination of statistical data as Linked Data. Finally, we will provide some examples of applications, and the opportunities that this approach offers for the development of the proofs of concepts selected by Eurostat and to be discussed during the meeting.
- #31: Linked Statistical Data 101 In this introductory talk we will discuss the main foundations for the application of Linked Data principles into official statistics. We will briefly introduce what Linked Data is, as well as the main principles, languages and technologies behind it (URIs, RDF, SPARQL). We will also discuss about the different formats in which data can be made available on the Web (e.g., RDF Turtle, JSON-LD, CSV on the Web). We will then move into providing a detailed presentation, with step by step examples based on existing Linked Statistical Data sources, of the W3C recommendation RDF DataCube, which is the basis for the dissemination of statistical data as Linked Data. Finally, we will provide some examples of applications, and the opportunities that this approach offers for the development of the proofs of concepts selected by Eurostat and to be discussed during the meeting.
- #40: Linked Statistical Data 101 In this introductory talk we will discuss the main foundations for the application of Linked Data principles into official statistics. We will briefly introduce what Linked Data is, as well as the main principles, languages and technologies behind it (URIs, RDF, SPARQL). We will also discuss about the different formats in which data can be made available on the Web (e.g., RDF Turtle, JSON-LD, CSV on the Web). We will then move into providing a detailed presentation, with step by step examples based on existing Linked Statistical Data sources, of the W3C recommendation RDF DataCube, which is the basis for the dissemination of statistical data as Linked Data. Finally, we will provide some examples of applications, and the opportunities that this approach offers for the development of the proofs of concepts selected by Eurostat and to be discussed during the meeting.