Web scraping in python
3 likes1,455 views
Web scraping with Python allows users to automatically extract data from websites by specifying CSS or XML paths to grab content and store it in a database. Popular libraries for scraping in Python include lxml, BS4, and Scrapy. The document demonstrates building scrapers using Beautiful Soup and provides tips for making scrapers faster through techniques like threading, queues, profiling, and reducing redundant scraping with memcache.
1 of 10
Downloaded 11 times
More Related Content
What's hot (20)
Ad
Viewers also liked (20)
Ad
Similar to Web scraping in python (20)
Recently uploaded (20)
Web scraping in python
- 1. Web Scraping with Python by @sauravtom (work is progress …)
- 2. Data Scraping Automated Process Specify css or xml path grab the content store it in a database
- 3. Who uses Scrapers ? Scrapers as backbone of Big Data Importance in Industry level as well as indie projects.
- 4. Why choose python ? Robust, flexible and powerful Relatively lesser development time Easy to learn and use Huge standard library, thorough documentation and helpful community.
- 5. Scraping libraries in python lxml BS4 Scrapy Mechanize twill ...
- 6. Scraper Demonstration in bs4 Inspect the element Find the node Plug it in (some code and pictures)
- 7. Making Scrapers faster Thread and Queues (some code ...)
- 8. Detecting bottlenecks Introduction to profiling in python (some code)
- 9. Making Scrapers even faster Using memcache to reduce redundant scraping (some code)
- 10. Thats it !! (links to the code present in these slides)