Boston elasticsearch meetup October 2012
- 1. Igor Motov igor@motovs.org twitter: @imotov github: imotov
- 3. Sonian Inc. •Cloud-based email archiving •Founded in 2007 •Headquarters: Newton, MA
- 4. Small team of about15 developers distributed from Campinas, Brazil to Vancouver, Canada
- 5. Using elasticsearch since June 2010, v0.8.0
- 6. We have about 6 billion records indexed in elasticsearch
- 7. 100,000 Netflix DVD Titles
- 8. 3,000,000 Pages in en.wikipedia.org
- 9. 22,000,000 Books in Library of Congress catalog
- 10. 150,000,000 Linked-in profiles
- 11. 3,000,000,000 Estimated bing.com index size
- 12. 6,000,000,000 Sonian Inc. index size
- 13. 50,000,000,000 Estimated google.com index size
- 14. Infrastructure
- 16. Ingestion (safe): Clojure Search Engine: elasticsearch Web App: Ruby on Rail Deployment: Chef Monitoring: Sensu
- 17. 10 clusters 6 AWS Regions 2-17 nodes in each cluster
- 18. Custom version of elasticsearch based on 0.19.9 with several plugins
- 19. jetty plugin • jetty-based http transport • SSL support • Authentication • Request logging (json, plain)
- 20. Request logs are also indexed in elasticsearch
- 21. Open source https://github.com/sonian/elasti csearch-jetty
- 22. Zookeeper plugin Zookeeper-based discovery Replacement for zen discovery Experimental!
- 23. Open source https://github.com/sonian/elasti csearch-zookeeper
- 24. Valve plugin •Custom jetty plugin filter •Rejects bulk indexing requests if cluster is overloaded
- 25. Lessons learned in the last two years or
- 26. Proper Care and Feeding of Elasticsearch Nodes
- 27. Rule1: Give nodes plenty of space Running out of disk space or memory is the simplest way to corrupt your index.
- 28. Make sure elasticsearch doesn’t swap It reduces performance and causes nodes to leave clusters
- 30. Increase the number of open file descriptors to 64k.
- 31. Rule 2: Distributed but well connected All nodes should be able to talk to each other all the time
- 32. Otherwise your cluster might get split-brain syndrome
- 34. Rule 3: Throttle the bulk indexing load Asynchronous architecture makes es scalable and fast, but susceptible to running out of memory under excessive bulk indexing load.
- 35. Rule 4: Try to make all shards approximately the same size Elasticsearch allocates shards based on the number of shards. It doesn’t consider shard sizes or available disk space.
- 36. 4 rules for happy elasticsearch 1. Give nodes plenty of space 2. Distributed but well connected 3. Throttle the load 4. Make all shards the same size
- 37. Questions?
- 38. More Information Latest stable release: 0.19.10 Web Site: http://www.elasticsearch.org/ Follow @elasticsearch on twitter IRC: #elasticsearch on irc.freenode.net GitHub: https://github.com/elasticsearch/elasticsearch Mailing list: elasticsearch on http://groups.google.com/ Stackoverflow tag: elasticsearch