Academy PRO: Elasticsearch. Data management
Download as PPTX, PDF1 like138 views
The document details methods for managing documents in Elasticsearch, including creating, retrieving, updating, and deleting documents. It covers usage of APIs for individual and bulk operations, document types, and mappings, emphasizing the immutability of documents and conflict resolution strategies. Key features include data types, index management, and an example of document retrieval.
![Create Document
Custom ID Auto-generated (UUID) ID
PUT /{index}/{type}/{id}[?op_type=create]|[/_create]
{
"field": "value",
...
}
POST /{index}/{type}
{
"field": "value",
…
}
{
"_id": "10",
"_index": "github",
"_source": {
"language": "JavaScript",
"name": "d3",
"watchers_count": 49272
},
"_type": "repos",
"_version": 1,
"found": true
}
{
"_id": "AVRZdOpTUK45DyN031IC",
"_index": "github",
"_source": {
"language": "JavaScript",
"name": "d3",
"watchers_count": 49272
},
"_type": "repos",
"_version": 1,
"found": true
}](https://image.slidesharecdn.com/lecture2-161020112628/85/Academy-PRO-Elasticsearch-Data-management-5-320.jpg)
![Document Retrieval
Full Document Partial
GET /{index}/{type}/{id}
GET /github/repos/10[?pretty]
{
"_id": "10",
"_index": "github",
"_source": {
"language": "JavaScript",
"name": "d3",
"watchers_count": 49272
},
"_type": "repos",
"_version": 1,
"found": true
}
{
"_id": "AVRZdOpTUK45DyN031IC",
"_index": "github",
"_source": {
"language": "JavaScript",
"name": "d3"
},
"_type": "repos",
"_version": 1,
"found": true
}
GET
/{index}/{type}/{id}?_source[=field1,..fieldN]
GET /github/repos/10?_source=name,language](https://image.slidesharecdn.com/lecture2-161020112628/85/Academy-PRO-Elasticsearch-Data-management-6-320.jpg)
![_update API
POST /{index}/{type}/{id}/_update[?retry_on_conflict=number_of_times]
{
“doc”: {
“field1”: value
“fieldN”: valueN
}
}
curl -XPOST /github/repos/2/_update -d '{"doc": {"tags": ["kernel", "c"]}}'](https://image.slidesharecdn.com/lecture2-161020112628/85/Academy-PRO-Elasticsearch-Data-management-10-320.jpg)
![Retrieve multiple docs (_mget)
POST /_mget
{
“docs”: [
{
“_id”: “ID”,
“_index”: “my_index”,
“_type”: ”type”
}
{...}
]
}
POST /{index}/{type}/_mget
{
“ids”: [“1”,..”N”]
}
curl -XPOST localhost:9200/github/repos/_mget -d '{"ids": [2, 3]}'](https://image.slidesharecdn.com/lecture2-161020112628/85/Academy-PRO-Elasticsearch-Data-management-12-320.jpg)
![Available data types
● Simple:
○ String: string
○ Whole number: byte , short , integer , long
○ Float: float , double
○ Boolean: boolean
○ Date: date
● Multivalue: { "tag": [ "search", "nosql" ]}
● Empty: null, [ ], [ null ]
● Nested: every nested level has type object](https://image.slidesharecdn.com/lecture2-161020112628/85/Academy-PRO-Elasticsearch-Data-management-15-320.jpg)
Academy PRO: Elasticsearch. Data management
- 1. Elasticsearch. Data management Binary Studio Academy PRO 2016 binary-studio.com
- 2. Documents
- 3. Structure { "_id": "AVRZdOpTUK45DyN031IC", "_index": "github", "_source": { "forks_count": 13294, "html_url": "https://github.com/mbostock/d3", "language": "JavaScript", "name": "d3", "watchers_count": 49272 }, "_type": "repos", "_version": 1, "found": true }
- 5. Create Document Custom ID Auto-generated (UUID) ID PUT /{index}/{type}/{id}[?op_type=create]|[/_create] { "field": "value", ... } POST /{index}/{type} { "field": "value", … } { "_id": "10", "_index": "github", "_source": { "language": "JavaScript", "name": "d3", "watchers_count": 49272 }, "_type": "repos", "_version": 1, "found": true } { "_id": "AVRZdOpTUK45DyN031IC", "_index": "github", "_source": { "language": "JavaScript", "name": "d3", "watchers_count": 49272 }, "_type": "repos", "_version": 1, "found": true }
- 6. Document Retrieval Full Document Partial GET /{index}/{type}/{id} GET /github/repos/10[?pretty] { "_id": "10", "_index": "github", "_source": { "language": "JavaScript", "name": "d3", "watchers_count": 49272 }, "_type": "repos", "_version": 1, "found": true } { "_id": "AVRZdOpTUK45DyN031IC", "_index": "github", "_source": { "language": "JavaScript", "name": "d3" }, "_type": "repos", "_version": 1, "found": true } GET /{index}/{type}/{id}?_source[=field1,..fieldN] GET /github/repos/10?_source=name,language
- 7. Document Presense Document Removal HEAD /{index}/{type}/{id} 200 || 404 DELETE /{index}/{type}/{id} { "_id": "AVRJ6Cm2HLfrw-ZrHWLm", "_index": "fifa", "_type": "players", "_version": 2, -- incremented always "found": true -- or false }
- 8. Document Updates ● Documents are IMMUTABLE --> each update creates new “snapshot” with ++_version ● Whole document update with PUT /{index}/{type}/{id} <body> emits next action sequence: ○ Retrieve data from an old doc ○ Change it ○ Delete old doc ○ Index new (updated) doc ● Better to use update API (made this in single action) ● Updates are conflict exposed
- 9. Dealing with conflicts 5 4` 4` 4 4 o/ Concurrency causes problems :) Pessimistic control (db): lock (row, table, record) Optimistic control (elastic): conflict won’t occur. Update will just fail. PUT /{index}/{type}/{id}?version=N <body> Next update with the same version will return 409 HTTP status
- 10. _update API POST /{index}/{type}/{id}/_update[?retry_on_conflict=number_of_times] { “doc”: { “field1”: value “fieldN”: valueN } } curl -XPOST /github/repos/2/_update -d '{"doc": {"tags": ["kernel", "c"]}}'
- 11. Bulk operations
- 12. Retrieve multiple docs (_mget) POST /_mget { “docs”: [ { “_id”: “ID”, “_index”: “my_index”, “_type”: ”type” } {...} ] } POST /{index}/{type}/_mget { “ids”: [“1”,..”N”] } curl -XPOST localhost:9200/github/repos/_mget -d '{"ids": [2, 3]}'
- 13. Insert multiple docs (_bulk) action: create|index|update|delete metadata:{“_index”:””, “_type”:””, “_id”:””} body: {“title”: “New article title”} curl -s -XPOST localhost:9200/_bulk --data-binary "@filename_with_newlines_format";
- 14. Data Mapping
- 15. Available data types ● Simple: ○ String: string ○ Whole number: byte , short , integer , long ○ Float: float , double ○ Boolean: boolean ○ Date: date ● Multivalue: { "tag": [ "search", "nosql" ]} ● Empty: null, [ ], [ null ] ● Nested: every nested level has type object
- 16. Single field mapping { "tag": { "type": "string", "index": "analyzed", -- analyzed | not_analyzed | no "analyzer": "english" } }
- 17. Assign mapping for a type { "mappings": { "repos": { "properties": { "name": { "type": "string" }, "language": { "type": "string" }, "watchers_count": { "type": "long" } } } } } PUT /{index} <mapping> { "_id": "10", "_index": "github", "_source": { "language": "JavaScript", "name": "d3", "watchers_count": 49272 }, "_type": "repos", "_version": 1, "found": true } Document Example:
- 18. Index management
- 19. Basic Index API Create index Delete index DELETE /{index} || /{index1,indexN} || /_all Update settings
- 20. To be continued ...