Finding Anything: Real-time Search with IndexTank
- 1. Finding anything: Real-time search with IndexTank Tim Spence April 19, 2011
- 2. About the Presenter Tim Spence Senior Infrastructure Engineer at MedHelp ( http://www.medhelp.org/ )
- 4. Recently converted to Ruby
- 5. In love with Open Source Software
- 6. More at http://whyhello.im/tim
- 7. Agenda State of search today
- 8. Quick survey: how much time/effort did YOU spend implementing search on your webapp?
- 9. Examples of services that need improved search
- 10. IndexTank to the rescue
- 11. Case study: reddit.com
- 12. Agenda, continued How I found out about IndexTank
- 13. Two apps I built with IndexTank
- 14. Live Demo
- 16. The State of Search Today Not well implemented at all Search works, but...
- 17. Barely How many pages of results do you typically browse through before finding what you were looking for?
- 18. Or do you give up and head for google site search instead?
- 19. Survey Time! How much time/effort did YOU spend implementing search on your webapp?
- 20. How many times have you iterated on your search feature?
- 21. When was the last time someone thanked you for building a powerful, reliable search feature for your webapp?
- 22. My Opinion Search as an in-app feature is an afterthought
- 23. Minimal implementation is the norm
- 24. If it wasn't for MySQL/MS-SQL full text indexing, most apps probably wouldn't even have a search feature
- 25. Most good web apps don't make it easy for users to find specific content outside of predetermined navigation
- 26. Let's pick on some apps! These are companies with great products, but their search comes up short
- 27. Don't worry–they can take it!
- 28. App #1: Github
- 29. App #1: Github
- 30. App #1: Github Interface is decent Search repos, code, users, or everything
- 31. Search by language However... Can't do much with results but browse
- 32. Check out this example
- 33. App #1: Github
- 34. App #1: Github Why these results aren't so hot Can't search by most recently maintained
- 35. Can't search by most popular (most watched)
- 36. Are you ready to browse 1,297 results? Advanced search capabilities exist, but not the best interface recency/popularity implemented, but require specific arguments
- 37. App #2: Amazon Web Services ”Hey, I bet I can find an AMI from the community for the exact EC2 setup I need”
- 39. App #2: Amazon Web Services
- 40. App #2: Amazon Web Services Notice something missing? No search
- 41. Only sort by date, title Ready to browse 934 results? I'd rather build my own AMI Incredible missed opportunity o/s search
- 42. Stack search
- 43. etc...
- 44. Fact: Github & Amazon aren't the only ones Lots of good web services
- 45. Massive quantities of quality content
- 46. Unfortunately not discoverable in meaningful ways
- 47. Interlude: Sites with great search Foodspotting Proximity
- 48. Recency
- 49. Rating Medhelp Content category
- 50. Promoted content Other sites I overlooked? Whose search do you like?
- 51. What was the point of that last slide? Search can be useful if it is valued as a feature
- 52. Any company willing to invest in the resources can build and host a high quality search engine
- 53. However, must you roll your own?
- 54. Enter Search as a Service No need for you to invest in additional infrastructure
- 55. No need to reinvent the wheel Search is a solved problem
- 56. Let the experts refine it
- 57. IndexTank to the rescue! Hosted–no load on your infrastructure
- 58. Powerful We'll get into the details next Always Improving Search IS their product Freemium
- 60. Let's talk features Real-time search Real-time indexing–results immediately available Custom scoring
- 61. Autocomplete
- 62. Faceting
- 63. Geo search
- 65. Real-time search Real-time indexing results immediately available Index multiple docs/sec
- 66. Overwrite existing docs as you wish Changes also immediately available
- 67. Custom Scoring Implementer has full control over how results are returned
- 68. Choose which fields are searched
- 69. Use pre-written scoring functions
- 70. Or write your own
- 71. Custom Scoring
- 72. Everyone loves autocomplete Saves users time
- 73. Potentially avoids spelling errors Not for hunters/peckers Adds a degree of intelligence to the search process
- 74. Faceting Does it make sense for you to categorize documents in your index? In all cases, YES Consider your advanced users and the narrow results they seek Don't make anyone sift through irrelevant results
- 75. Faceting
- 76. Geo It's 2011 Location is more relevant than ever before
- 77. Mobile is skyrocketing–every client has a GPS IndexTank has built-in geo proximity search capability
- 78. Geo
- 79. Advanced Text Search (Beta) Fuzzy search (Did you mean...?)
- 80. Stemming Alternate word forms (tense, possession, etc...) Alternate spellings Misspellings
- 81. Other Benefits Zero maintenance
- 82. Scalability included for free
- 83. Easy implementation Clients available in many languages
- 84. Excellent documentation–Let's check it out Excellent support Humans or bots? You decide Dog food: their site search is done well
- 86. Case Study: reddit.com High traffic news aggregator (> 1.0E9 pvs/mo) with tons of content
- 87. Who remembers how bad reddit's search was? When it even worked Can't blame them for trying Many attempts, but none worked IndexTank excelled in all areas
- 88. Let's check it out now
- 89. My experience with IndexTank Discovered through Heroku/IndexTank contest
- 90. Built my first irl Rails app in an afternoon/evening w/ fellow hacker Chris Saylor (@cwsaylor)
- 91. Didn't win the contest but learned how easy it is to quickly create highly targeted search
- 92. App #1: Toxosis Searchable database of toxic release data supplied by U.S. E.P.A.
- 93. Hosted at http://toxosis.heroku.com/
- 94. Search enabled on many fields including city/state/zip, toxin
- 95. Additional fields can be added to index When I have time, of course...
- 96. More personal backstory Still in the business of reinventing myself as a Rails developer
- 97. How to get a Rails gig? Develop an app multiple Rails apps and show it them off
- 98. Opportunities are everywhere–contests, hackathons, and weekend hacks for developer community
- 99. App #2: SXSWdex Searchable database of 2011 SXSW attendees
- 100. Hosted at http://sxswdex.heroku.com/
- 101. Design goal: do a better job than SXSW official site
- 102. Search within bio, company, location, name
- 104. The moment we've all been waiting for Let's build an app!
- 105. Questions? Q&A time with an IndexTank engineer