ONS Local and Data Science Community Workshop 1: How to use APIs
Download as PPTX, PDF0 likes208 views
The document provides an introduction to Application Programming Interfaces (APIs), explaining their significance in data communication and analysis. It highlights the advantages of using APIs for accessing formatted, filtered, and real-time data, emphasizing increased efficiency and quality. A practical demonstration using the ONS Census API is included, showcasing the use of Python and integration with tools like Power BI and Tableau.
ONS Local and Data Science Community Workshop 1: How to use APIs
- 1. Introduction to Application Programming Interfaces, and ONS’ APIs Andrew Fergusson Lead Data Scientist ONS ⊃ Digital Publishing ⊃ Data Service andrew.fergusson@ons.gov.uk Teams, 6 July 2023
- 2. Agenda • What are APIs? • Advantages of APIs • Practical demonstration of APIs
- 3. What is an API? More importantly why should you care?
- 5. Data wrangling sucks • Spreadsheets are presentational, not designed for analysis • Cross tabs typically only have two dimensions • Got to re-interpret the column headers, unpivot the data • Contains more data than you need • Not value add!
- 6. Application Programming Interfaces • A set of rules and protocols that allows software and data sources to communicate • Standards-backed, e.g. HTTP Methods (information transfer), HTTP Status Codes (like 200 Okay 401 Unauthorized), JSON, XML • Can be accessed using URLs which contain your request details (a HTTP GET) or with a JSON file a (HTTP PUT)
- 7. Some useful APIs (Not just for reading data!) • The ONS Census and CMD API: https://developer.ons.gov.uk/ • The NHS Fingertips API: https://fingertips.phe.org.uk/api • HMRC’s suite of APIs: https://developer.service.hmrc.gov.uk/api- documentation/docs/api • DWP’s suite of APIs: https://www.api.gov.uk/dwp/#department-for-work-and- pensions
- 8. APIs are a set of procedures to access data There may be lots of new terms here, but consuming APIs is risk- free. The worst outcome of a bad API call you’ll experience from an ONS API is that it didn’t understand your query. Photo by Kevin Fitzgerald on Unsplash
- 9. Advantages of APIs Why should you use APIs?
- 10. Why you should use APIs • Formatted for analysis • Filtered for your patch • No transformation errors • Automated updates • Real-time data (depends!) • Enriched data sources • Increased efficiency • Enhanced quality • Collaboration and sharing • Agility and adaptability
- 11. Using or not using an API is your choice Configuring a service to use an API may take a bit more effort but you’ll get all of that time back and then some. Photo by Bernd 📷 Dittrich on Unsplash
- 12. Practical demonstration of APIs Introduction to the ONS Census API
- 13. Notes about this demonstration • This demonstration is using Visual Studio Code, a free Integrated Developer Environment, and Python a programming language • The examples are configurable, and the presentational document will be available after this session • This example is more advanced than the autumn sessions, but it is necessary to demonstrate how the ONS Census API works • PowerBI and Tableau can consume APIs directly, and interpret the API’s responses in JSON for you • Programming in Python will not be required • Being familiar with how the API structures data helps you think about your requests
Editor's Notes
- #11: Formatted for analysis: no more wrangling and copy/pasting from excel spreadsheets Filtered for your patch: no more downloading all of England’s data to report on Salford No transformation errors: the API returns data in the format you request, meaning you won’t make a mistake wrangling and get incorrect analysis Automated updates: when new data is released, the connections still work so you just need to refresh the API; if you are working with time series this is especially good Real-time data: It may be true that real-time data is available from an API most analysis ONS Local performs isn’t real time (i.e. live), it’s generally a programmatic view of a static set of observations which get updated monthly, etc. An example of a realtime dataset is Manchester’s Oxford Road Cycle Counter which is updated each time a cyclist passes a set point Enriched data sources: APIs can return detailed information and notes at almost any level: so instead of a warning on the note section about a set of observations, it’s specific to the exact scope but doesn’t ruin your spreadsheet with symbols or footnotes in an observation cell with the value you want. You can’t sum up observations when the values 3, 4, 5, and x. Increased efficiency: set it and forget it. APIs are versioned, which means when you program against a version of an API, it will continue to work even if the data updates; no more broken spreadsheets. API developers will release a whole new API version (i.e. v2 after v1) if they’re going to introduce any changes which will break v1 functionality Enhanced quality: The Census API has perturbation, and there are other advantages Collaboration and sharing: Imagine being able to provide access to all of the UK’s census information but just with a link, the power you get with an internet connection and not having to download the large amounts of data locally – not that you would be allowed to access the microdata – means it’s easier to collaborate Agility and adaptability: As these datasets are filterable fully custom, ad hoc analysis is just selecting the new analysis. Is the report originally ethnicity vs age for a local authority’s wards, but someone wants to see highest level of education obtained vs age, it’s just changing one parameter!