Data Gathering with The Web Observatory
0 likes335 views
The document discusses the Web Observatory and data gathering. The Web Observatory is a platform for gathering data from webpages through scraping. It discusses scraping data from webpages, investigating page structure, using APIs if available, and extracting, transforming and loading data into the desired format. Resources for hands-on scraping include codepen.io and github wiki pages.
Data Gathering with The Web Observatory
- 1. THE WEB OBSERVATORY DATA GATHERING WITH EUGENE SIOW & XIN WANG 29 JANUARY 2016
- 2. WHAT IS THE WEB OBSERVATORY? SEARCH + ACCESS
- 3. DATA | WO | APP OPEN PRIVATE NoSQL SQL STREAMS LINKED DATA JS PYTHON NODE
- 4. GATHERING DATA WITH SCRAPING DATA ON WEBPAGES DATA I CAN USE transform ( )
- 5. THE PROCESS OF SCRAPING INVESTIGATE THE STRUCTURE OF THE PAGE CHECK IF THERE IS AN API {APPLICATION PROGRAMMING INTERFACE} USE CHROME’S INSPECTOR OR FIREBUG EXTRACT, TRANSFORM, LOAD WHAT IS YOUR DESIRED END FORMAT?
- 6. HANDS-ON RESOURCES codepen.io/xgfd/pen/wMyQWb github.com/eugenesiow/datathon2016/wiki DATA-DRIVEN APPS USING THE WO DATA GATHERING webobservatory.soton.ac.uk THE SOTON WEB OBSERVATORY BACKGROUNDS FROM THE HUBBLE SPACE TELESCOPE