Using ap is to gather data
- 1. SMAC LAB, LSU Sep 7, 2018 SMAC Talks Using APIs to Gather Data Instructor: Dr. Ke (Jenny) Jiang
- 2. Using APIs to Gather Data Installation Download Anaconda Python 3.6 to run Python scripts. Anaconda is free to download. Once you’ve installed Anaconda, you can create/edit scripts in Spyder.
- 3. Spyder Using APIs to Gather Data Open the files
- 4. Spyder Using APIs to Gather Data Open the files
- 5. Spyder Let’s Play.. Copy & Paste the Syntax to the Console Using APIs to Gather Data
- 6. Using APIs to Gather Data More efficient way to run the Syntax
- 7. Using APIs to Gather Data Search Text in the Syntax
- 8. Using APIs to Gather Data The search Results being Highlighted
- 9. Using APIs to Gather Data Go to “variables_strings-numbers.py” first
- 10. Using APIs to Gather Data Application Programming Interfaces * allow you to access data or services on other servers on the web URL http://api.nytimes.com/svc/community/v3/user-content/by-date.json returns data when loaded
- 11. API Documentation * what you can and can't do with the API
- 12. NYT APIs Documentation http://developer.nytimes.com Community API http://developer.nytimes.com/community_api_v3.json#/README
- 13. Something to be aware of… * APIs are usually rate limited (not to request data to an api too frequently) * Sign up for an authorization key to use APIs
- 14. Request an API Key http://developer.nytimes.com Request an API Key
- 15. Request an API Key Name Email Web Community API Click
- 16. Find the Requested API Key Click Check Your Mailbox API Key
- 17. use aip to collect nyt comments.py Download Code:
- 18. Open the Code in Spyder Click
- 19. Step 1: Import Packages Step 2: Get Data Copy the string key into the variable api_key Get data in .json format in a specific day (2018-08-31)
- 20. Step 3: Get Data as a JSON object * We’ve collected 25 comments from the date we specified in the URL * BUT there were many more comments made that day * We want to loop through and collect them all Step 4: Parse it into a Pandas DataFrame
- 21. Step 5: How Many Comments? *Output: We need to collect 331 times, since we only get 25 comments at a time. * If you are making more than 1,000 calls per day (via an application or any other method) to an individual API, NYT will notice that and assume you're using our API for a commercial purpose. Please contact NYT if you have hit that limit but do not think you are using the API for a commercial purpose.
- 22. Step 6: Collect All the Comments Slow our script down Line 90: Use your own path to save the comments
- 23. Step 7: Who Are Most Active? Most active users Users comments >= 5 What kind of comments
- 24. Step 8: Open Saved Comments