![Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
Configuration (stand-alone)
• Configuration config/elasticsearch.yml
cluster.name: "elasticsearch-imotov"
network.host: "127.0.0.1"
discovery.zen.ping.multicast.enabled: false
discovery.zen.ping.unicast.hosts: ["localhost"]
unique
name
listen only
on localhost
disable
multicast
search for other
nodes on localhost](https://image.slidesharecdn.com/boston-hackathon-131010052233-phpapp01/85/Hopper-Elasticsearch-Hackathon-10-320.jpg)
![Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
Searching
{
"took" : 5,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"failed" : 0
},
"hits" : {
"total" : 1,
"max_score" : 6.1357985,
"hits" : [ {
"_index" : "test-data",
"_type" : "cities",
"_id" : "21",
"_score" : 6.1357985, "_source" : {"rank":"21","city":"Boston",...}
} ]
}
}](https://image.slidesharecdn.com/boston-hackathon-131010052233-phpapp01/85/Hopper-Elasticsearch-Hackathon-19-320.jpg)
![Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
Boolean Queries
$ curl -XGET "http://localhost:9200/test-data/cities/_search?pretty" -d '{
"query": {
"bool": {
"should": [{
"match": { "state": "Texas"}
}, {
"match": { "state": "California"}
}],
"must": {
"range": {
"population2012": {
"from": 500000,
"to": 1000000
}
}
},
"minimum_should_match": 1
}
}
}'](https://image.slidesharecdn.com/boston-hackathon-131010052233-phpapp01/85/Hopper-Elasticsearch-Hackathon-21-320.jpg)
![Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
Sorting and Paging
$ curl -XGET "http://localhost:9200/test-data/cities/_search?pretty" -d '{
"query": {
"match_all": { }
},
"sort": [
{"state": {"order": "asc"}},
{"population2010": {"order": "desc"}}
],
"from": 0,
"size": 20
}'](https://image.slidesharecdn.com/boston-hackathon-131010052233-phpapp01/85/Hopper-Elasticsearch-Hackathon-23-320.jpg)
![Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
Disabling stopwords in elasticsearch.yml
index:
analysis:
analyzer:
default:
type: "custom"
tokenizer: "standard"
filters: ["lowercase"]](https://image.slidesharecdn.com/boston-hackathon-131010052233-phpapp01/85/Hopper-Elasticsearch-Hackathon-26-320.jpg)
Hopper Elasticsearch Hackathon
- 1. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited Elasticsearch Quick Introduction Hopper Elasticsearch Hackathon Boston, MA - Sep 27, 2013
- 2. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited About Me • Igor Motov • Developer at Elasticsearch Inc. • Github: imotov • Twitter: @imotov
- 3. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited About Elasticsearch Inc. • Founded in 2012 • By the people behind the Elasticsearch and Apache Lucene • http://www.elasticsearch.com • Headquarters:Amsterdam and Los Altos, CA • We provide • Training (public & onsite) • Development support • Production support subscription (SLA)
- 4. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited About Elasticsearch • Real time search and analytics engine • JSON-oriented,Apache Lucene-based • Automatic Schema Detection • Enables control of it when needed • Distributed • Scales Up+Out, Highly Available • Multi-tenancy • Dynamically create/delete indices • API centric • Most functionality is exposed through an API
- 5. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited Basic Concepts • Cluster • a group of nodes sharing the same set of indices • Node • a running Elasticsearch instance (typically JVM process) • Index • a set of documents of possibly different types • stored in one or more shards • Type • a set of documents in an index that share the same schema • Shard • a Lucene index, allocated on one of the nodes
- 6. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited Basic Concepts - Document • JSON Object • Identified by index/type/id { "rank": 21, "city": "Boston", "state": "Massachusetts", "population2010": 617594, "land_area": 48.277, "density": 12793, "ansi": 619463, "location": { "lat": 42.332, "lon": 71.0202 }, "abbreviation": "MA" }
- 7. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited Downloading elasticsearch • http://www.elasticsearch.org/download/ Windows Everything else
- 8. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited What’s in a distribution? . ├── LICENSE.txt ├── NOTICE.txt ├── README.textile ├── bin │ ├── elasticsearch │ ├── elasticsearch.in.sh │ └── plugin ├── config │ ├── elasticsearch.yml │ └── logging.yml ├── data │ └── elasticsearch ├── lib │ ├── elasticsearch-x.y.z.jar │ ├── ... │ └── └── logs ├── elasticsearch.log └── elasticsearch_index_search_slowlog.log executable scripts node config files data storage libs log files
- 9. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited Configuration (multicast) • Configuration config/elasticsearch.yml cluster.name: "elasticsearch-imotov" unique name
- 10. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited Configuration (stand-alone) • Configuration config/elasticsearch.yml cluster.name: "elasticsearch-imotov" network.host: "127.0.0.1" discovery.zen.ping.multicast.enabled: false discovery.zen.ping.unicast.hosts: ["localhost"] unique name listen only on localhost disable multicast search for other nodes on localhost
- 11. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited Starting elasticsearch • Foreground • Background $ bin/elasticsearch -f $ bin/elasticsearch
- 12. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited Is it running? { "ok" : true, "status" : 200, "name" : "Hensley Fargus", "version" : { "number" : "0.90.5", "build_hash" : "c8714e8e0620b62638f660f6144831792b9dedee", "build_timestamp" : "2013-09-17T12:50:20Z", "build_snapshot" : false, "lucene_version" : "4.4" }, "tagline" : "You Know, for Search" } $ curl -XGET "http://localhost:9200/?pretty"
- 13. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited Communicating with Elasticsearch • REST API • Curl • Ruby • Python • PHP • Perl • JavaScript (community supported) • Binary Protocol • Java
- 14. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited Pick your client • Java • included in distribution • Ruby, PHP, Perl, Python • http://www.elasticsearch.org/blog/unleash-the-clients- ruby-python-php-perl/ • Everything Else • http://www.elasticsearch.org/guide/clients/
- 15. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited Indexing a document $ curl -XPUT "http://localhost:9200/test-data/cities/21" -d '{ "rank": 21, "city": "Boston", "state": "Massachusetts", "population2010": 617594, "land_area": 48.277, "density": 12793, "ansi": 619463, "location": { "lat": 42.332, "lon": 71.0202 }, "abbreviation": "MA" }' {"ok":true,"_index":"test-data","_type":"cities","_id":"21","_version":1}
- 16. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited Getting a document { "_index" : "test-data", "_type" : "cities", "_id" : "21", "_version" : 1, "exists" : true, "_source" : { "rank": 21, "city": "Boston", "state": "Massachusetts", "population2010": 617594, "land_area": 48.277, "density": 12793, "ansi": 619463, "location": { "lat": 42.332, "lon": 71.0202 }, "abbreviation": "MA" } } $ curl -XGET "http://localhost:9200/test-data/cities/21?pretty"
- 17. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited Updating a document $ curl -XPUT "http://localhost:9200/test-data/cities/21" -d '{ "rank": 21, "city": "Boston", "state": "Massachusetts", "population2010": 617594, "population2012": 636479, "land_area": 48.277, "density": 12793, "ansi": 619463, "location": { "lat": 42.332, "lon": 71.0202 }, "abbreviation": "MA" }' {"ok":true,"_index":"test-data","_type":"cities","_id":"21","_version":2}
- 18. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited Searching $ curl -XGET 'http://localhost:9200/test-data/cities/_search?pretty' -d '{ "query": { "match": { "city": "Boston" } } }'
- 19. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited Searching { "took" : 5, "timed_out" : false, "_shards" : { "total" : 1, "successful" : 1, "failed" : 0 }, "hits" : { "total" : 1, "max_score" : 6.1357985, "hits" : [ { "_index" : "test-data", "_type" : "cities", "_id" : "21", "_score" : 6.1357985, "_source" : {"rank":"21","city":"Boston",...} } ] } }
- 20. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited Range Queries $ curl -XGET "http://localhost:9200/test-data/cities/_search?pretty" -d '{ "query": { "range": { "population2012": { "from": 500000, "to": 1000000 } } } }'
- 21. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited Boolean Queries $ curl -XGET "http://localhost:9200/test-data/cities/_search?pretty" -d '{ "query": { "bool": { "should": [{ "match": { "state": "Texas"} }, { "match": { "state": "California"} }], "must": { "range": { "population2012": { "from": 500000, "to": 1000000 } } }, "minimum_should_match": 1 } } }'
- 22. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited MatchAll Query $ curl -XGET "http://localhost:9200/test-data/cities/_search?pretty" -d '{ "query": { "match_all": { } } }'
- 23. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited Sorting and Paging $ curl -XGET "http://localhost:9200/test-data/cities/_search?pretty" -d '{ "query": { "match_all": { } }, "sort": [ {"state": {"order": "asc"}}, {"population2010": {"order": "desc"}} ], "from": 0, "size": 20 }'
- 24. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited Analysis • By default string are • Divided into words (tokens) • All tokens are converted to lower-case • English stop words are removed • a, an, and, are, as, at, be, but, by, for, if, in, into, is, it, no, not, of, on, or, such, that, the, their, then, there, these, they, this, to, was, will, with
- 25. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited Analysis Example • “Elasticsearch is a powerful open source search and analytics engine.” 1. elasticsearch 2. powerful 3. open 4. source 5. search 6. analytics 7. engine
- 26. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited Disabling stopwords in elasticsearch.yml index: analysis: analyzer: default: type: "custom" tokenizer: "standard" filters: ["lowercase"]
- 27. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited Customizing the mapping curl -XPUT 'http://localhost:9200/my_index/' -d '{ "settings": { "index": { "number_of_shards": 1, "number_of_replicas": 0 } }, "mappings": { "my_type": { "properties": { "description": { "type": "string" }, "sku": { "type": "string", "index": "not_analyzed" }, "count": { "type": "integer" }, "price": { "type": "float" }, "location": { "type": "geo_point" } } } } }' exact match analyzed text geo location
- 28. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited Elasticsearch Reference • http://www.elasticsearch.org/guide/
- 29. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited Ideas for hackathon • Explore data • wikipedia • twitter • enron emails • Play with Kibana • Build Elasticsearch plugins • Get elasticsearch T-shirt
- 30. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited Elasticsearch Meetup http://www.meetup.com/Elasticsearch-Boston/
- 31. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited We are hiring http://www.elasticsearch.com/about/jobs/