Multi model-databases
- 1. Polyglot Persistence & Multi-Model Databases 1 Michael Hackstein @mchacki www.arangodb.org
- 2. Michael Hackstein @mchacki ‣ ArangoDB Core Team ‣ Web Frontend ‣ Graph visualisation ‣ Graph features ! ! ‣ Organiser of cologne.js ‣ Master’s Degree (spec. Databases and Information Systems) 2
- 3. Once upon a time… ‣ Use a SQL-based database ‣ Implement logic to transform your data into table format ‣ Create / Generate complex queries ‣ Implement logic to transform data from table format in required format 3
- 4. 4 Shopping-Cart RecommendationsSales-History Customer Product-Catalog An e-commerce system in Relational World 4
- 5. Main Categories of NoSQL DBs 5 Key/Value Store Document Store Graph Database Source: Andrew Carol Polyglot Persistence
- 6. Key-Value Store ‣ Map value data to unique string keys (identifiers) ‣ Treat data as opaque (data has no schema) ‣ Can implement scaling and partitioning easily due to simplistic data model ‣ Key-value can be seen as a special case of documents 6
- 7. Document Store ‣ Normally based on key-value stores (each document still has a unique key) ‣ Allow to save documents with logical similarity in “databases” or “collections” ‣ Treat data records as attribute-structured documents (data is no more opaque) ‣ Often allow querying and indexing document attributes 7
- 9. Polyglot Persistence ‣ „Use the right tool for the job“ ‣ If you have structured data with some differences ‣ Use a document store ‣ If you have relations between entities and want to efficiently query them ‣ Use a graph database ‣ If you manage the data structure yourself and do not need complex queries ‣ Use a key-value store ‣ If you have structured data where all objects have equal attributes ‣ Use a relational database 9
- 10. 10 Recommendations Product-CatalogShopping-Cart Sales-History Customer KeyValueStore Single-Model-Databases 10 { “userID": 239178239, “productID”: 128623883, “number": 5, “price”: 12.20, } { “userID": 239178239, “productID”: 128623883, “number": 5, “price”: 12.20, } DocumentStore GraphStore { “Name": "Smith", “lastLogin”: “2012-11-01", “Visits": 121, “shipping address”: “abc”, “shipping address”: “def” } { “Name": "Meyer", “lastLogin”: “2012-11-21", “Visits": 20, “shipping address”: “xyz”, } DocumentStore 423453453 4328, “shirt”, “L”, 1, 12.99 6378, “sweater”, “M”, 2, 37.95 3245, “sweater”, “blue”, 1, 99.95 3245, “pants”, “32/34”, “black”, 1, 99.95 => 874365563 5463, “shirt”, “S”, 1, 9.99 6378, “sweater”, “M”, 2, 37.95 3245, “pants”, “32/34”, “black”, 1, 99.95 => { “type“: "pants", “waist": 32, “length”: 34, “color": "blue", “material”: “cotton" } { “type“: "television", “diagonal screen size": 46, “hdmi inputs": 3, “wall mountable": true, “built-in digital tuner": true, “dynamic contrast ratio”: “50,000:1”, Resolution”: “1920x1080” } { “type": "sweater", “color": "blue", “size": “M”, “material”: “wool”, “form”: “turtleneck" } { “type": "sweater", “color": "blue", “size": “M”, “material”: “wool”, “form”: “turtleneck" } DocumentStore
- 11. Benefits & Costs 11 ‣ Native mapping of data into DB ‣ DB optimized ‣ Queries are tailored for your data format ‣ Focus on writing business logic ‣ Several technologies involved ‣ Experts required for each ‣ Administration effort is huge ‣ Application logic has to interface with several sources ‣ Data has to be stored redundantly and has to be kept in sync
- 12. Syncing Requirements 12 Customers Products Recommendations Sales-History ExtractEdges SuggestonlyAvailableProducts Filtersuggestions Find connected users Buyers Purchases ‣ Problem: Different Data-Formats ‣ On small systems: One database could be sufficient
- 13. Multi Model Database 13 ‣ Can store several kinds of data models: ‣ Acts as a key-value store ‣ Acts as a document store ‣ Stores graphs natively ‣ Delivers query mechanisms for all data models
- 14. 14 ‣ a multi-model database (document store & graph database) ‣ is open source and free (Apache 2 license) ‣ offers convenient queries (via HTTP/REST and AQL) ‣ offers high performance and is memory efficient ‣ uses JavaScript throughout (V8 built into server) ‣ doubles as a web and application server (Foxx) ‣ offers many drivers for a wide range of languages ‣ is easy to use with web frontend and good documentation ‣ enjoys good professional as well as community support ‣ and recently added sharding in Version 2.0.
- 15. Different query interfaces Different scenarios require different access methods: ‣ Query a document by its unique id / key: GET /_api/document/users/12345 ‣ Query by providing an example document: PUT /_api/simple/by-example { "name": "Michael", "age": 26 } ‣ Query via AQL: FOR user IN users FILTER user.active == true RETURN { name: user.name } ‣ Graph Traversals 15
- 16. Why another query language? ‣ Initially, we implemented a subset of SQL's SELECT ‣ It didn't fit well ‣ UNQL addressed some of the problems ‣ Looked dead ‣ No working implementations ‣ XQuery seemed quite powerful ‣ A bit too complex for simple queries ‣ JSONiq wasn't there when we started 16
- 17. Other Document Stores ‣ MongoDB uses JSON/BSON as its “query language” ‣ Limited ‣ Hard to read & write for more complex queries ‣ CouchDB uses Map/Reduces ‣ It‘s not a relational algebra, and therefore hard to generate ‣ Not easy to learn 17
- 18. ArangoDB Query Language (AQL) ‣ We came up with AQL mid-2012 ‣ Declarative language, loosely based on the syntax of XQuery ‣ Other keywords than SQL so it's clear that the languages are different ‣ Implemented in C and JavaScript 18
- 19. Extendable through JS ‣ Dynamic Language that enriches ArangoDB ‣ Multi Collection Transactions ‣ Graph Traversals ‣ Cascading deletes/updates ‣ Aggregate data from multiple queries into a single response ‣ Data-intensive operations ‣ Actions, Foxx, Application Server 19
- 20. Application Server / Action Server ‣ ArangoDB can answer arbitrary HTTP requests directly ‣ You can write your own JavaScript functions (“actions”) that will be executed server-side ‣ Includes a permission system ‣ Build your own API rapidly using the Foxx framework ! ! ➡ You can use it as a database or as a combined database/ application server 20
- 21. 21 Product-CatalogShopping-Cart Sales-History Recommendations Customer DocumentStore DocumentStore DocumentStoreGraphStore KeyValueStore Use Case: Multi-Model-Databases 21
- 22. Syncing Requirements 22 Customers Products Recommendations Sales-History ExtractEdges SuggestonlyAvailableProducts Filtersuggestions Find connected users Buyers Purchases ‣ Data-format is unified, no transformations ‣ On small systems: (Single Database) ‣ Separate into collections ‣ No redundancy ‣ Gain transaction possibilities
- 23. Join our growing community 23 .. working on the geo index, the full text search and many APIs: Ruby, Python, PHP, Java, D, ...
- 24. Summary ‣ Multi-Model databases simplify database setup ‣ Cross-out transformation of data-formats ! ‣ ArangoDB ‣ Based on web standards: HTTP, REST, JSON ‣ Flexible querying: Documents, Graphs in the same language ‣ Reduce Server <-> Database communication: ‣ Define your own API using Foxx ‣ Execute data-intensive code within the database ‣ You can even use ArangoDB as our application server 24
- 25. 25 Thank you ! ‣ Further questions? ‣ Follow me on twitter/github: @mchacki ‣ Or write me a mail: mchacki@arangodb.org 25