How to connect to API endpoint and query data using Python?

Recipe Objective

In big data scenarios , we are going to connect to multiple API endpoints and need to retrieve the data from the api which is the very first step of data extraction and perform some processing like parsing , cleaning, transformation on the data for deriving business value out of the data.

Access Snowflake Real-Time Project to Implement SCD's

System requirements :

• Install the python module as follows if the below modules are not found:
• pip install requests
• The below codes can be run in Jupyter notebook , or any python console
• In this scenario we are using an open api to make an requests and the link open api Click Here
• Link to some public api’s for you Open-API's

Step 1: Import the module

The most common library for making requests and working with APIs is the requests library. You’ll need to import it. Let’s start with that important step:

import requests

Step 2: Making an HTTP request

To make a request to the API, there are different types of requests like GET, POST etc. GET request is the most commonly using one It used to get the data from api.when get the request is successful then it will give the response status code 200 , to make GET request we will use the get() method.

Sample code to make a request :

import requests response = requests.get('https://ghibliapi.herokuapp.com/films/') if response.status_code == 200: print("Succesful connection with API.") print('-------------------------------') data = response.json() print(data) elif response.status_code == 404: print("Unable to reach URL.") else: print("Unable to connect API or retrieve data.")

In the above code if the request response statuscode is 200, then it will print the "Successful connection with API." and also prints data in the json object format , otherwise it will print the "unable to reach URL".

Output of the above code:

Step 3: To query the data by sending the params

To query the specific data from the api will pass the "params" variable as send argument in the get method.

import requests response = requests.get('https://ghibliapi.herokuapp.com/films/', params={'id' :"4e236f34-b981-41c3-8c65-f8c9000b94e7"}) if response.status_code == 200: print("Successful connection with API.") print('-------------------------------') data = response.json() elif response.status_code == 404: print("Unable to reach URL.") else: print("Unable to connect API or retrieve data.") for record in data: print("Title: {},\n Release_Date: {},\n Director: {},\n".format(record['title'] , record['release_date'],record['director']))

Output of the above code: In the above code we query the specific data from the api and print the data in the sorted structure, Here the result is it will print specific id of data.