Ferret A Ruby Search Engine
5 likes1,467 views
This document provides an overview of the Ferret Ruby search engine. It discusses what Ferret is, key concepts like indexes and documents, how to install and index documents in Ferret, how to perform searches using the Ferret Query Language (FQL), and how to integrate Ferret into Rails applications using the Acts As Ferret plugin. The document also provides pointers to online and printed resources for learning more about Ferret and full-text search engines.
![Example Documents
Create some Documents
[“This”, “is also”, “a document”]](https://image.slidesharecdn.com/ferret-090801074430-phpapp02/85/Ferret-A-Ruby-Search-Engine-33-320.jpg)
Ferret A Ruby Search Engine
- 1. Ferret A Ruby Search Engine Brian Sam-Bodden
- 2. Agenda • What is Ferret? • Concepts • Fields • Indexing • Installing Ferret
- 3. Agenda • The Recipe • Documents • Ferret::Index::Index • FQL • Ferret in you App
- 4. Agenda • Ferret in Rails • Resources
- 5. What is Ferret? • Information Retrieval (IR) Library • Full-featured Text Search Engine • Inspired on the Search Engine • Port to Ruby by David Balmain
- 6. What is Ferret? • Initially a 100% pure Ruby port • Since 0.9 many core functions are implemented in C • Fast! Now Faster than Lucene ;-)
- 7. Concepts
- 8. Concepts • Index : Sequence of documents
- 9. Concepts • Index : Sequence of documents • Document : Sequence of fields
- 10. Concepts • Index : Sequence of documents • Document : Sequence of fields • Field : Named sequence of terms
- 11. Concepts • Index : Sequence of documents • Document : Sequence of fields • Field : Named sequence of terms • Term : A text string, keyed by field name
- 12. Fields of a Document in an Index
- 13. Fields of a Document in an Index • Fields are individually searchable units that are:
- 14. Fields of a Document in an Index • Fields are individually searchable units that are: • Stored: The original Terms of the fields are store
- 15. Fields of a Document in an Index • Fields are individually searchable units that are: • Stored: The original Terms of the fields are store • Indexed: Inverted to rapidly find all Documents containing any of the Terms
- 16. Fields of a Document in an Index • Fields are individually searchable units that are: • Stored: The original Terms of the fields are store • Indexed: Inverted to rapidly find all Documents containing any of the Terms • Tokenized: Individual Terms extracted are indexed
- 17. Fields of a Document in an Index • Fields are individually searchable units that are: • Stored: The original Terms of the fields are store • Indexed: Inverted to rapidly find all Documents containing any of the Terms • Tokenized: Individual Terms extracted are indexed • Vectored: Frequency and location of Terms are stored
- 18. It’s all about Indexing • Indexing is the processing of a source document into plain text tokens that Ferret can manipulate • For any non-plaintext sources such as PDF, Word, Excel you need to: • Extract • Analyze
- 20. Installing Ferret gem install ferret
- 23. Installing Ferret }
- 24. Installing Ferret } Pick the latest version for your platform
- 25. The Recipe
- 26. The Recipe 1. Create some Documents
- 27. The Recipe 1. Create some Documents 2. Create an Index
- 28. The Recipe 1. Create some Documents 2. Create an Index 3. Adding Documents to the Index
- 29. The Recipe 1. Create some Documents 2. Create an Index 3. Adding Documents to the Index 4. Perform some Queries
- 30. Example Documents Create some Documents
- 31. Example Documents Create some Documents “Any String is a Document”
- 32. Example Documents Create some Documents
- 33. Example Documents Create some Documents [“This”, “is also”, “a document”]
- 34. Example Documents Create some Documents
- 35. Example Documents Create some Documents
- 36. Ferret::Index::Index Create an Index
- 37. Ferret::Index::Index Create an Index • Indexes are encapsulated by the class
- 38. Ferret::Index::Index Create an Index • Indexes are encapsulated by the class ➡ Ferret::Index::Index
- 39. Ferret::Index::Index Create an Index • Indexes are encapsulated by the class ➡ Ferret::Index::Index • Use the alias Ferret::I for convenience
- 40. Ferret::Index::Index Create an Index • Indexes are encapsulated by the class ➡ Ferret::Index::Index • Use the alias Ferret::I for convenience • Index can be persistent
- 41. Ferret::Index::Index Create an Index • Indexes are encapsulated by the class ➡ Ferret::Index::Index • Use the alias Ferret::I for convenience • Index can be persistent ➡ index = Ferret::I.new(:path = > ‘/somepath’)
- 42. Ferret::Index::Index Create an Index • Indexes are encapsulated by the class ➡ Ferret::Index::Index • Use the alias Ferret::I for convenience • Index can be persistent ➡ index = Ferret::I.new(:path = > ‘/somepath’) • Or, completely in Memory
- 43. Ferret::Index::Index Create an Index • Indexes are encapsulated by the class ➡ Ferret::Index::Index • Use the alias Ferret::I for convenience • Index can be persistent ➡ index = Ferret::I.new(:path = > ‘/somepath’) • Or, completely in Memory ➡ index = Ferret::I.new()
- 44. Ferret::Index::Index Adding Documents to the Index • Index provides the add_document method • It also provides the << alias • Adding documents is then as easy as: ➡ index << “This is a document” ➡ index << {:first => “Bob”, :last => “Smith”}
- 45. Ferret::Index::Index Perform some Queries
- 46. Ferret::Index::Index Perform some Queries • Index provides the search and search_each methods
- 47. Ferret::Index::Index Perform some Queries • Index provides the search and search_each methods • search method takes a query and a an optional set of parameters:
- 48. Ferret::Index::Index Perform some Queries • Index provides the search and search_each methods • search method takes a query and a an optional set of parameters: ➡ search(query, options = {})
- 49. Ferret::Index::Index Perform some Queries • Index provides the search and search_each methods • search method takes a query and a an optional set of parameters: ➡ search(query, options = {}) • The search_each method provides an iterator block
- 50. Ferret::Index::Index Perform some Queries • Index provides the search and search_each methods • search method takes a query and a an optional set of parameters: ➡ search(query, options = {}) • The search_each method provides an iterator block ➡ search_each(query, options = {}) {|doc, score| ... }
- 51. Playing with Ferret in irb
- 52. Playing with Ferret in irb
- 53. Ferret Query Language • Ferret own Query Language, FQL is a powerful way to specify search queries • FQL supports many query types, including: • Term • Range • Phrase • Wild • Field • Fuzz • Boolean
- 54. Index.explain • The explain method of Index describes how a document score against a query • Very useful for debugging • and for learning how Ferret works
- 55. Index.explain
- 56. Ferret in your App Application Database Web User Manual File System Input Get User’s Present Gather Data Search Results Query Index Documents Search Index Ferret Index
- 57. Ferret in Rails • Acts As Ferret is an ActiveRecord extension • Available as a plugin • Provides a simplified interface to Ferret • Maintained by Jens Kramer
- 58. Ferret in Rails • Adding an index to an ActiveRecord model is as simple as:
- 59. Ferret in Rails • Adding an index to an ActiveRecord model is as simple as:
- 60. Ferret in Rails • Simple model has two searchable fields title and body:
- 61. Ferret in Rails • After a quick rake db:migrate we now have some data to play with • Fire up the Rails Console and let’s see what acts_as_ferret can do for our models
- 62. Ferret in Rails
- 63. Want more? • Ferret is improving constantly • Acts As Ferret seems to catch up quickly • Real-life usage seems to require some good engineering on your part • Background indexing • Hot swap of indexes?
- 64. Want more? • We only covered the simplest constructs in Ferret • Ferret’s API provides enough flexibility for the most demanding searching needs
- 65. Online Resources • http://ferret.davebalmain.com • http://lucene.apache.org • http://lucenebook.com • http://projects.jkraemer.net/acts_as_ferret
- 67. Thanks!