MySQL User Group NL: MySQL 8.0 Document Store- NoSQL with all the benefits of a transactional RDBMS

MySQL 8.0
a Document Store with all the benefits of a transactional RDBMS

Frédéric Descamps - MySQL Community Manager - Oracle
3 / 124

Save the date !
Copyright @ 2018 Oracle and/or its affiliates. All rights reserved.
4 / 124

Safe Harbor Statement
The following is intended to outline our general product direction. It is intended for
information purpose only, and may not be incorporated into any contract. It is not a
commitment to deliver any material, code, or functionality, and should not be relied up in
making purchasing decisions. The development, release and timing of any features or
functionality described for Oracle´s product remains at the sole discretion of Oracle.
5 / 124

about me - http://about.me/lefred
Who am I ?
6 / 124

Frédéric Descamps
@lefred
MySQL Evangelist
Hacking MySQL since 3.23
devops believer
living in Belgium 🇧🇪
http://lefred.be

7 / 124

RDBMS & MySQL
Relational Databases
8 / 124

Data Integrity
9 / 124

Data Integrity
normalization
10 / 124

Data Integrity
normalization
constraints (foreign keys, ...)
11 / 124

Data Integrity
normalization
Atomicity, Consistency, Isolation, Durability
12 / 124

Data Integrity
normalization
ACID compliant
13 / 124

Data Integrity
normalization
ACID compliant
transactions
14 / 124

Data Integrity
normalization
ACID compliant
transactions
SQL
15 / 124

Data Integrity
normalization
ACID compliant
transactions
SQL
powerfull query language
16 / 124

NoSQL & MySQL
NoSQL Databases
17 / 124

NoSQL or Document Store
Schemaless
18 / 124

Schemaless
no schema design, no normalization, no foreign keys, no data types, ...
19 / 124

Schemaless
very quick initial development
20 / 124

Schemaless
Flexible data structure
21 / 124

Schemaless
embedded arrays or objects
22 / 124

Schemaless
valid solution when natural data can´t be modelized optimaly into a relational
model
23 / 124

Schemaless
model
objects persistance without the use of any ORM - mapping oobject-oriented
24 / 124

Schemaless
model
JSON
25 / 124

Schemaless
model
JSON
close to frontend
26 / 124

Schemaless
model
JSON
close to frontend
native in JS
27 / 124

Schemaless
model
JSON
close to frontend
native in JS
easy to learn
28 / 124

How DBAs see data
29 / 124

How DBAs see data How Developers see data
{
"GNP" : 249704,
"Name" : "Belgium",
"government" : {
"GovernmentForm" :
"Constitutional Monarchy, Federation",
"HeadOfState" : "Philippe I"
},
"_id" : "BEL",
"IndepYear" : 1830,
"demographics" : {
"Population" : 10239000,
"LifeExpectancy" : 77.8000030517578
},
"geography" : {
"Region" : "Western Europe",
"SurfaceArea" : 30518,
"Continent" : "Europe"
}
}
30 / 124

What if there was a way to provide both SQL
and NoSQL on one stable platform that has
proven stability on well know technology with
a large Community and a diverse ecosystem ?
31 / 124

DBMS or NoSQL ?
32 / 124

DBMS or NoSQL ?
Why not both ?
33 / 124

The MySQL Document Store !
SQL is now optional !
34 / 124

Developers:
[x] schemaless
[x] rapid prototying/simpler APIs
[x] document model
[x] transactions
Operations:
[x] performance
management/visibility
[x] robust replication, backup,
restore
[x] comprehensive tooling
ecosystem
[x] simpler application schema
upgrades
Business Owner:
[x] don't lose my data == ACID trx
[x] capture all my data =
extensible/schemaless
[x] product on schedule/time to
market = rapid developement
A solution for all
35 / 124

MySQL Document Store
the Solution
36 / 124

37 / 124

Built on
the MySQL
JSON Data
type and
Proven
MySQL
Server
Technology
Provides a schema flexible JSON Document Store
No SQL required
No need to define all possible attributes, tables, etc.
Uses new X DevAPI
Can leverage generated column to extract JSON values
into materialized columns that can be indexed for fast
SQL searches.
Document can be ~1GB
It's a column in a row of a table
It cannot exceed max_allowed_packet
Allows use of modern programming styles
No more embedded strings of SQL in your code
Easy to read
Also works with relational Tables
Proven MySQL Technology
38 / 124

X DevAPI Connectors for
C++, Java, .Net, Node.js, Python, PHP
working with Communities to help them supporting
it too
New MySQL Shell
Command Completion
Python, JavaScrips & SQL modes
Admin functions
New Util object
A new high-level session concept that can scale from
single MySQL Server to a multiple server environment
Non-blocking, asynchronous calls follow common
language patterns
Supports CRUD operations
39 / 124

Starting using MySQL DS in few minutes

40 / 124

MySQL Shell Info
41 / 124

Migration from MongoDB to MySQL DS
For this example, I will use the well known restaurants collection:
42 / 124

We need to dump the data to a file and we will use MySQL Shell with the Python
interpreter to load the data.
43 / 124

We need to dump the data to a file and we will use MySQL Shell with the Python
interpreter to load the data.
44 / 124

Dump and load using MySQL Shell & Python
this example is inspired by @datacharmer's work:
https://www.slideshare.net/datacharmer/mysql-documentstore
$ mongo --quiet --eval 'DBQuery.shellBatchSize=30000;
db.restaurants. nd().shellPrint()'
| perl -pe 's/(?:ObjectId|ISODate)(("[^"]+"))/ $1/g' > all_recs.json
45 / 124

Dump and load using MySQL Shell & Python
this example is inspired by @datacharmer's work:
https://www.slideshare.net/datacharmer/mysql-documentstore
$ mongo --quiet --eval 'DBQuery.shellBatchSize=30000;
db.restaurants. nd().shellPrint()'
| perl -pe 's/(?:ObjectId|ISODate)(("[^"]+"))/ $1/g' > all_recs.json
46 / 124

47 / 124

Even better in MySQL Shell 8.0.13 [preview]
48 / 124

Dump from MongoDB:
49 / 124

Dump from MongoDB:
And import with the new util function in the Shell:
50 / 124

Let´s query
51 / 124

Let´s query
That´s too much records to show it here... let´s limit it
52 / 124

53 / 124

Some more examples
54 / 124

Some more examples
Let´s add a selection criteria:
55 / 124

Using IN...
56 / 124

Syntax slightly different than MongoDB
57 / 124

Syntax slightly different than MongoDB
> db.restaurants. nd({"cuisine": "French",
"borough": { $not: /^Manhattan/} },
{"_id":0, "name": 1,"cuisine": 1, "borough": 1}).limit(2)
{ "borough" : "Queens", "cuisine" : "French",
"name" : "La Baraka Restaurant" }
{ "borough" : "Queens", "cuisine" : "French",
"name" : "Air France Lounge" }
58 / 124

CRUD operations
59 / 124

CRUD operations for collections
Add a document
60 / 124

Modify a document
61 / 124

Remove a document
62 / 124

MySQL Document Store Objects Summary
63 / 124

Document Store Full ACID !
It relies on the proven MySQL InnoDB´s strength & robustness:
64 / 124

innodb_ ush_log_at_trx_commit = 1
65 / 124

innodb_doublewrite = ON
66 / 124

sync_binlog = 1
67 / 124

sync_binlog = 1
transaction_isolation = REPEATABLE-READ|READ-
COMMITTED|...
68 / 124

sync_binlog = 1
transaction_isolation = REPEATABLE-READ|READ-
COMMITTED|...
We do care about your data !
69 / 124

MySQL Document Store supports transactions
70 / 124

MySQL Document Store supports transactions
71 / 124

72 / 124

what about my old SQL ?
The hidden part of the iceberg
73 / 124

JSON datatype is behind the scene
native datatype (since 5.7.8)
74 / 124

JSON values are stored in MySQL tables using UTF8MB4
75 / 124

Conversion from "native" SQL types to JSON values
76 / 124

JSON manipulation functions (JSON_EXTRACT, JSON_KEYS, JSON_SEARCH,
JSON_TABLES, ...)
77 / 124

JSON_TABLES, ...)
Generated/virtual columns
78 / 124

JSON_TABLES, ...)
Indexing JSON data
79 / 124

JSON_TABLES, ...)
Indexing JSON data
Foreign Keys to JSON data
80 / 124

JSON_TABLES, ...)
Indexing JSON data
Foreign Keys to JSON data
SQL Views to JSON data
81 / 124

How does the MySQL Document Store work ?
82 / 124

What does a collection look like on the server ?
83 / 124

_id Every document has a unique identifier called the document
ID, which can be thought of as the equivalent of a table´s
primary key. The document ID value can be manually assigned
when adding a document. If no value is assigned, a document
ID is generated and assigned to the document automatically !
Use getDocumentId() or getDocumentIds() to
get _ids(s)

84 / 124

Mapping to SQL examples
createCollection('mycollection')
85 / 124

CREATE TABLE `test`.`mycoll` (
doc JSON,
_id VARCHAR(32)
GENERATED ALWAYS AS (doc->>'$._id') STORED
PRIMARY KEY
) CHARSET utf8mb4;
86 / 124

doc JSON,
_id VARCHAR(32)
PRIMARY KEY
) CHARSET utf8mb4;
mycollection.add({'test': 1234})
87 / 124

doc JSON,
_id VARCHAR(32)
PRIMARY KEY
) CHARSET utf8mb4;
mycollection.add({'test': 1234})
INSERT INTO `test`.`mycoll` (doc)
VALUES (
JSON_OBJECT('_id','663807fe367ee6114e0e5458bdac28bf',
'test',1234)
);
88 / 124

Mapping to SQL examples (2)
mycollection.find("test > 100")
89 / 124

Mapping to SQL examples (2)
mycollection.find("test > 100")
SELECT doc
FROM `test`.`mycoll`
WHERE (JSON_EXTRACT(doc,'$.test') > 100);
90 / 124

SQL and JSON Example
91 / 124

same as:
92 / 124

same as:
we can use it in SQL:
93 / 124

SQL and JSON Example (2): validation
94 / 124

95 / 124

96 / 124

SQL and JSON Example (3): explain
97 / 124

SQL and JSON Example (3): explain
98 / 124

SQL and JSON Example (4): add index
99 / 124

SQL and JSON Example (4): add index
100 / 124

Indexes in Document Store
It's also possible to create indexes without using SQL syntax:
101 / 124

Indexes in Document Store
It's also possible to create indexes without using SQL syntax:
102 / 124

SQL and JSON Example (5): arrays
103 / 124

104 / 124

105 / 124

106 / 124

NoSQL as SQL
JSON_TABLE
107 / 124

NoSQL as SQL (2)
JSON_TABLE
108 / 124

NoSQL as SQL - aggregation
109 / 124

NoSQL as SQL - aggregation (2)
110 / 124

SQL and NoSQL - JOINS
111 / 124

112 / 124

113 / 124

SQL and NoSQL - JOINS (2)
114 / 124

115 / 124

116 / 124

SQL and NoSQL - and back to JSON Document
117 / 124

NoSQL or SQL
118 / 124

what do I gain ?
Conclusion
119 / 124

schemaless
flexible data structure
easy to start (CRUD)
Conclusion
This is the best of the two worlds in one product !
Data integrity
ACID Compliant
Transactions
SQL
120 / 124

122 / 124

http://lefred.be/content/top-10-reasons-for-
nosql-with-mysql/
123 / 124

Thank you !
Any Questions ?
share your 💕 for MySQL on social media using @mysql #MySQL8isGreat
124 / 124

MySQL User Group NL: MySQL 8.0 Document Store- NoSQL with all the benefits of a transactional RDBMS

More Related Content

What's hot (20)

Similar to MySQL User Group NL: MySQL 8.0 Document Store- NoSQL with all the benefits of a transactional RDBMS (20)

More from Frederic Descamps (20)

Recently uploaded (20)

MySQL User Group NL: MySQL 8.0 Document Store- NoSQL with all the benefits of a transactional RDBMS