Making easy roommate easy
Download as ODP, PDF0 likes895 views
The document discusses a web scraping tool called Easy Roommate that allows users to find housing accommodations by parsing profile data from the Easy Roommate website. Some limitations of the site include an inability to filter profiles by user-specified criteria or hide unwanted profiles. The author describes using web scraping and parsing tools to download profile data from the site and check if new or edited profiles match user needs. Considerations around separating parsing code from business logic and handling multiple listings from one user are also mentioned.
Making easy roommate easy
- 1. Making Easy Roommate Easy Andrew Grimm @andrewjgrimm
- 2. Easy Roommate <Insert sexist jokes here `coz most of the audience is male>
- 3. Web site where people can find accommodation, or vice versa
- 4. The Good Profiles have structured data
- 5. Means you can determine whether a flat is unsuitable without contacting them
- 6. The Bad You can't search only for places compatible with you
- 7. You can't hide or delete profiles you're not interested in
- 8. Profiles you've already read get bumped to the top when they get edited
- 9. The Ugly Web scraping to the rescue!
- 10. Download and parse the profiles
- 11. Check whether they're already mentioned in your notes (plain text file)
- 12. Describe whether new ones are compatible or incompatible
- 13. Web scraping tools Mechanize for getting past authentication
- 14. Nokogiri to parse the HTML Not regular expressions – sorry Cthulhu!
- 15. Ternary logic Some fields have “Not Disclosed” in them
- 16. Rather than saying the profile meets your criteria or doesn't meet them, you can say it isn't known
- 18. Coding considerations Should code for parsing an object be separate from business logic for the object itself?
- 19. Some pages have different structures
- 20. One person may have an ad for each room in a unit – how do you tell they're distinct? (Use their unique IDs)
- 21. Coding considerations II Decided not to use a database Too lazy to install on all my computers Currently only looking at some properties in a profile YAGNI
- 22. To do Automate commute time calculation rather than manually visit 131500.com.au
- 23. Find a flat! @andrewjgrimm