Fix "SDMX Data APIs"
Download as PPTX, PDF0 likes581 views
The document outlines the OECD's role in collecting and analyzing data from its 38 member countries, offering an overview of various initiatives such as the Programme for International Student Assessment (PISA) and efforts to combat tax avoidance through the Base Erosion and Profit Shifting (BEPS) initiative. It highlights the Statistical Data and Metadata Exchange (SDMX) standard created in 2002 to improve data comparability among international organizations and national institutes, along with programming resources for using SDMX APIs. Additionally, it provides useful links to documentation and tools for data access and analysis.
Fix "SDMX Data APIs"
- 1. SDMX DATA APIS Jakob Fix, Digital Engineering Manager Public Affairs and Communications Directorate
- 2. Introduction • About the OECD • Statistics • The SDMX standard • Explore the OECD SDMX API • Hands-on exercise with the ECB API 2
- 3. About the OECD • 1948: OEEC to implement Marshall Plan • 1961: OEEC reformed to become OECD • Today 38 member countries, latest to become a member: Costa Rica • Not just economy, but labour, trade, environment, education, trade, technology, (even the Blockchain and artificial intelligence) • Activities include: • Collect data from member countries and other sources • Analyse trends and developments • Provide advice to policy makers • Publish to disseminate knowledge (Freemium model ) 3
- 4. PISA • “Programme for International Student Assessment” • 80+ countries • Every three years (unless Covid) • 600k students representing 32m+ 15 yr-olds • Goal is to help countries improve their educational systems, by learning from others • You can test yourself: https://www.oecd.org/pisa/test/
- 5. BEPS • Initiative to end Base Erosion and Profit Shifting • 135+ countries to end tax avoidance by multinational enterprises (MNE) • Agreed in 2021 • Many tax avoidance mechanisms neutralised • Recover taxes (between 100 and 240B USD pa lost) • 15% tax rate for MNEs • One of the 15 actions is on “Data Analysis” • More information: https://www.oecd.org/tax/beps/
- 6. International Statistics • International organisations (e.g. World Bank, OECD, IMF, BIS, United Nations, Eurostat, ILO, FAO) • National statistics institutes (e.g. ONS in UK, Census Bureau in US, INSEE in France) • Collect data from member countries (and beyond) to advance research • Data is messy, data providers have different standards and change over time • Lots of time spent to normalise data and make it comparable • Exchange mechanisms were email and FTP
- 7. SDMX to the rescue • 2002: Statistical Data and Metadata eXchange Initiative • Today a living ISO standard maintained by the community via SDMX.org • Goals are to • Exchange data robustly between parties • Accompany data with sufficient and unambiguous metadata • Initially intended for Electronic Data Interchange (EDI) • Later extended to also be useful in web-based applications using RESTful APIs
- 8. It’s all about the context 42 Switzerland 2021 (in percent) Women parliamentarians Observation Dimension Dimension Dimension Dimension
- 9. Let’s look at some metadata
- 10. SDMX API URL Structure • https://stats.oecd.org/ ⇒ protocol and host • restsdmx/sdmx.ashx/GetData/ ⇒ path to endpoint • DP_LIVE/ ⇒ flowRef (or dataset identifier) • AUS.WOMENPOLVOICE.MINISTERWOMEN+POLREP RES.PC.A ⇒ key (combination of dimensions)
- 11. Hands-on programming with Python • Open the Jupyter notebook, or the Colaboratory project
- 12. Thank you! • (Resources on the next slide)
- 13. Resources • Documentation • SDMX documentation: https://www.sdmx.org/ • OECD Data API SDMX-XML documentation: https://data.oecd.org/api/sdmx-ml-documentation/ • ECB SDMX API documentation: https://sdw-wsrest.ecb.europa.eu/help/ • The Python exercise is based in large parts on this walkthrough: https://pandasdmx.readthedocs.io/en/v1.0/walkthrough.html • Another, older Python example: https://www.pythonsherpa.com/tutorials/4/ • Implementations • OECD Data Portal: https://data.oecd.org/ • OECD Compare your Country: https://www.compareyourcountry.org/ • IPAC dashboard (uses React components that encapsulate the data retrieval and the chart creation): https://www.oecd.org/climate-action/ipac/dashboard • Tools • Python library pandaSDMX: https://pandasdmx.readthedocs.io/en/v1.0/