PHP Detroit -- MySQL 8 A New Beginning (updated presentation)

8.0 A New Beginning
Dave Stokes
MySQL Community Manager
David.Stokes@Oracle.com @Stoker
Slides -> https://slideshare.net/davidmstokes
Blog -> https://elephantdolphin.blogger.com

Safe Harbor Agreement
THE FOLLOWING IS INTENDED TO OUTLINE OUR GENERAL PRODUCT
DIRECTION. IT IS INTENDED FOR INFORMATION PURPOSES ONLY, AND MAY
NOT BE INCORPORATED INTO ANY CONTRACT. IT IS NOT A COMMITMENT TO
DELIVER ANY MATERIAL, CODE, OR FUNCTIONALITY, AND SHOULD NOT BE
RELIED UPON IN MAKING PURCHASING DECISIONS. THE DEVELOPMENT,
RELEASE, AND TIMING OF ANY FEATURES OR FUNCTIONALITY DESCRIBED
FOR ORACLE'S PRODUCTS REMAINS AT THE SOLE DISCRETION OF ORACLE.
2

MySQL News
● 23 years old! Oracle owned for eight years!
● MySQL 8.0 is the current Generally Available release
● Document Store
● Group Replication
● We’re Hiring
3

MySQL 8?
What happened to MySQL 6 and
MySQL 7??
4

Well..
● Previous GA is 5.7 (October 2015)
● MySQL Cluster is 7.5.10
● There was a MySQL 6 in the pre-Sun days, kinda like the PHP version six
that nobody really talks about except in hushed tones and with great
sadness
Engineering thought the new data dictionary and other new features justified the
new major release number.
5

1. Data Dictionary
Before MySQL 8 -- Meta Data Stored in files!
You have a plethora of files out there -- .FRM
.MYD .MYI .OPT and many more just waiting for
something to go bad -- now store relevant
information in data dictionary!
This means you are no longer dependent in the
number of inodes on your system, somebody
rm-ing the files at just the wrong time, and a
whole host of other problems.
Innodb is robust enough to rebuild all
information to a point in time in case of
problems. So keep EVERYTHING in internal
data structures. And that leads to transactional
ALTER TABLE commands.
6

System Tables are now InnoDB
Previously, these were MyISAM (non transactional) tables. This change applies
to these tables: user, db, tables_priv, columns_priv, procs_priv, proxies_priv.
7

Good News!?
So now you can have
millions of tables
within a schema.
The bad news is
that you can have
millions of tables
within a schema.
8

2.CTEs & Windowing Functions
Long requested, Common Table Expression and Windowing Functions have a
wide variety of uses.
● CTEs are handy subquery-like statements often used in quick
calculations
● Windowing Functions are great for iterating over a selected set of rows
for things like statistical calculations
9

Windowing
Function
The key word is
OVER
SELECT name, department_id,
salary, SUM(salary)
OVER (PARTITION BY
department_id) AS
department_total
FROM employee
ORDER BY department_id, name
10

Another
Example
Windowing
functions are great
when dealing with
dates
SELECT date, amount,
sum(amount)
OVER w AS ‘sum’
FROM payments
WINDOW w AS
(ORDER BY date
RANGE BETWEEN INTERVAL 1
WEEK PRECEDING AND
CURRENT ROW)
ORDER BY date;
11

CTEs
..are like derived
tables but the
declaration is
BEFORE the query
WITH qn AS (SELECT
t1 FROM mytable)
SELECT * FROM qn.
12

JOINing two CTEs 13
WITH
cte1 AS (SELECT a, b FROM table1),
cte2 AS (SELECT c, d FROM table2)
SELECT b, d FROM cte1 JOIN cte2
WHERE cte1.a = cte2.c;

Common
Table
Expression -
recursive
+------+
| n |
+------+
| 1 |
| 2 |
| 3 |
| 4 |
| 5 |
| 6 |
| 7 |
| 8 |
| 9 |
| 10 |
+------+
10 rows in set (0,00 sec)
WITH RECURSIVE my_cte AS
(
SELECT 1 AS n
UNION ALL
SELECT 1+n FROM my_cte
WHERE n<10
)
SELECT * FROM my_cte;
14

3. Optimizer & Parser
● Descending indexes
● Optimizer trace output now includes more information about filesort operations, such as key and
payload size and why addon fields are not packed.
● The optimizer now supports hints that enable specifying the order in which to join tables.
● New sys variable to include estimates for delete marked records includes delete marked records in
calculation of table and index statistics. This work was done to overcome a problem with "wrong"
statistics where an uncommitted transaction has deleted all rows in the table.
● Index and Join Order Hints -- User controls order
● NOWAIT and SKIPPED LOCKED to bypass locked records
15

How SKIP LOCKED or NOWAIT look
START TRANSACTION;
SELECT * FROM seats WHERE seat_no BETWEEN 2 AND 3 AND booked = 'NO'
FOR UPDATE SKIP LOCKED;
...
COMMIT;
START TRANSACTION
SELECT seat_no
FROM seats JOIN seat_rows USING ( row_no )
WHERE seat_no IN (3,4) AND seat_rows.row_no IN (12)
AND booked = 'NO'
FOR UPDATE OF seats SKIP LOCKED
FOR SHARE OF seat_rows NOWAIT;
16

Contention-Aware Transaction Scheduling CATS
The CATS algorithm is based on a simple intuition:
not all transactions are equal, and not all objects
are equal. When a transaction already has a lock
on many popular objects, it should get priority
when it requests a new lock. In other words,
unblocking such a transaction will indirectly
contribute to unblocking many more transactions
in the system, which means higher throughput and
lower latency overall.
17

4. Roles
MySQL now supports roles, which are named collections of
privileges. Roles can be created and dropped. Roles can
have privileges granted to and revoked from them. Roles
can be granted to and revoked from user accounts. The
active applicable roles for an account can be selected
from among those granted to the account, and can be
changed during sessions for that account.
Set up and account for a certain function and then assign
users who need that function.
18

5. Character Sets
MySQL 8
IS by default
UTF8MB4! 19

Not all UTf8 equal
utf8mb4_0900_ai_ci:
0900 refers to Unicode
Collation Algorithm version.
- ai refers to accent
insensitive.
- ci refers to case
insensitive.
Previously UTF8 was actually UTF8MB3
● 3 bytes, no emojis
● Supplementary multilingual plane
support limited
● No CJK Unified Ideographs Extension
B are in supplementary ideographic
plane
Upgrade problem expected!
Also supports GB18030 character set!
20

6. Invisible Indexes
An invisible index is not used by the optimizer at all, but is
otherwise maintained normally. Indexes are visible by
default. Invisible indexes make it possible to test the effect
of removing an index on query performance, without
making a destructive change that must be undone should
the index turn out to be required
22

7. SET PERSIST
mysql> SET PERSIST innodb_buffer_pool_size = 512 * 1024 * 1024;
Query OK, 0 rows affected (0.01 sec)
23

Why SET PERSIST
A MySQL server can be configured and
managed over a SQL connection thus
removing manual file operations (on
configuration files) to be done by
DBAs. This feature addresses the
usability issues described above, and
allows MySQL to be more easily
deployed and configured on cloud
platforms.
The file mysqld-auto.cnf is created
the first time a SET PERSIST
statement is executed. Further SET
PERSIST statement executions will
append the contents to this file. This
file is in JSON format and can be
parsed using json parser.
Timestamp & User recorded
24

Other new
features not
dependant on
server GA
Decoupling features like Group
Replication and Document Store
from release cycle to make
updates easier
● Add new features via a plug-in
● Make upgrades less onerous
● Easier management of features
Yes, we know that servers
can be hard to manage and
get harder when they are in
the cloud and out of reach
of ‘percussive maintenance’
techniques.
25

8. 3G Geometry
“GIS is a form of digital mapping technology.
Kind of like Google Earth but better.”
-- Arnold Schwarzenegger
Governor of California
26

8. 3D Geometry
● World can now be flat or ellipsoidal
● Coordinate system wrap around
● Boot.Geometry & Open GID
● Code related to geometry parsing, computing bounding boxes
and operations on them, from the InnoDB layer to the
Server layer so that geographic R-trees can be supported
easily in the future without having to change anything in
InnoDB
27

9. JSON -- A big change in Databases
We can use a JSON field to eliminate one of the issues of traditional database
solutions: many-to-many-joins
This allows more freedom to store unstructured data (data with pieces missing)
You still use SQL to work with the data via a database connector but the JSON
documents in the table can be manipulated directly in code.
Joins can be expensive. Reducing how many places you need to join data can help
speed up your queries. Removing joins may result in some level of denormalization
but can result in fast access to the data. 28

Plan for Mutability
Schemaless designs are focused on mutability. Build your
applications with the ability to modify the document as
needed (and within reason)
29

Remove Many-to-Many Relationships
● Use embedded arrays and lists to store relationships among documents.
This can be as simple as embedding the data in the document or
embedding an array of document ids in the document.
● In the first case data is available as soon as you can read the document
and in the second it only takes one additional step to retrieve the data. In
cases of seldom read (used) relationships, having the data linked with an
array of ids can be more efficient (less data to read on the first pass)
30

->> Operator
MySQL 8 adds a new unquoting extraction operator ->>, sometimes also referred to as
an inline path operator, for use with JSON documents stored in MySQL. The new
operator is similar to the -> operator, but performs JSON unquoting of the value as
well.
The following three expressions are equivalent:
● JSON_UNQUOTE( JSON_EXTRACT(mycol, "$.mypath") )
● JSON_UNQUOTE(mycol->"$.mypath")
● mycol->>"$.mypath"
Can be used with (but is not limited to) SELECT lists, WHERE and HAVING clauses,
and ORDER BY and GROUP BY clauses. 31

JSON_PRETTY
mysql> SELECT JSON_PRETTY(doc) FROM countryinfo LIMIT 1;
{
"GNP": 828,
"_id": "ABW",
"Name": "Aruba",
"IndepYear": null,
"geography": {
"Region": "Caribbean",
"Continent": "North America",
"SurfaceArea": 193
},
"government": {
"HeadOfState": "Beatrix",
"GovernmentForm": "Nonmetropolitan Territory of The Netherlands"
},
"demographics": {
"Population": 103000,
"LifeExpectancy": 78.4000015258789
}
}
32

JSON_ARRAYAGG
mysql> SELECT col FROM t1;
+--------------------------------------+
| col |
+--------------------------------------+
| {"key1": "value1", "key2": "value2"} |
| {"keyA": "valueA", "keyB": "valueB"} |
+--------------------------------------+
2 rows in set (0.00 sec)
mysql> SELECT JSON_ARRAYAGG(col) FROM t1;
+------------------------------------------------------------------------------+
| JSON_ARRAYAGG(col) |
+------------------------------------------------------------------------------+
| [{"key1": "value1", "key2": "value2"}, {"keyA": "valueA", "keyB": "valueB"}] |
+------------------------------------------------------------------------------+ 33

JSON_OBJECTAGG()
mysql> SELECT id, col FROM t1;
+------+--------------------------------------+
| id | col |
+------+--------------------------------------+
| 1 | {"key1": "value1", "key2": "value2"} |
| 2 | {"keyA": "valueA", "keyB": "valueB"} |
+------+--------------------------------------+
2 rows in set (0.00 sec)
mysql> SELECT JSON_OBJECTAGG(id, col) FROM t1;
+----------------------------------------------------------------------------------------+
| JSON_OBJECTAGG(id, col) |
+----------------------------------------------------------------------------------------+
| {"1": {"key1": "value1", "key2": "value2"}, "2": {"keyA": "valueA", "keyB": "valueB"}} |
+----------------------------------------------------------------------------------------+
1 row in set (0.00 sec)
34
Both JSON_ARRAY_AGG and
JSON_OBJECTAGG() work with
both JSON and non JSON
COLUMNS!

JSON_STORAGE_SIZE &
JSON_STORAGE_FREE
mysql> CREATE TABLE jtable (jcol JSON);
Query OK, 0 rows affected (0.42 sec)
mysql> INSERT INTO jtable VALUES
-> ('{"a": 1000, "b": "wxyz", "c": "[1, 3, 5, 7]"}');
Query OK, 1 row affected (0.04 sec)
mysql> SELECT
-> jcol,
-> JSON_STORAGE_SIZE(jcol) AS Size,
-> JSON_STORAGE_FREE(jcol) AS Free
-> FROM jtable;
+-----------------------------------------------+------+------+
| jcol | Size | Free |
+-----------------------------------------------+------+------+
| {"a": 1000, "b": "wxyz", "c": "[1, 3, 5, 7]"} | 47 | 0 |
+-----------------------------------------------+------+------+
1 row in set (0.00 sec) 35

JSON_TABLE - Structure your unstructured data
SELECT jt.first_name,
jt.last_name,
jt.contact_details
FROM json_documents,
JSON_TABLE(data, '$'
COLUMNS (first_name VARCHAR(50 CHAR) PATH '$.FirstName',
last_name VARCHAR(50 CHAR) PATH '$.LastName',
contact_details VARCHAR(200 CHAR)
FORMAT JSON WITH WRAPPER PATH '$.ContactDetails')) jt
WHERE id > 25;
FIRST_NAME LAST_NAME CONTACT_DETAILS
--------------- --------------- ----------------------------------------
John Doe [{"Email":"john.doe@example.com","Phone"
:"44 123 123456","Twitter":"@johndoe"}]
Jayne Doe [{"Email":"jayne.doe@example.com","Phone
":""}] 36
JSON_TABLE is used for
making JSON data look like
relational data, which is
especially useful when creating
relational views over JSON data,

MySQL Document Store
Relational databases such as MySQL usually required a document schema to be defined before documents can be stored.
A new plug-in enables you to use MySQL as a document store, which is a schema-less, and therefore schema-flexible, storage system
for documents.
When using MySQL as a document store, to create documents describing products you do not need to know and define all possible
attributes of any products before storing them and operating with them.
This differs from working with a relational database and storing products in a table, when all columns of the table must be known and
defined before adding any products to the database.
This allows you to choose how you configure MySQL, using only the document store model, or combining the flexibility of the document
store model with the power of the relational model.
37

Using the MySQL Document Store with the X DevAPI PECL Extension 38
#!/usr/bin/php
<?PHP
// Connection parameters
$user = 'root'; $passwd = 'hidave'; $host = 'localhost'; $port = ' 33060';
$connection_uri = 'mysqlx://'.$user.':'.$passwd.'@'.$host.':'.$port;
// Connect as a Node Session
$nodeSession = mysql_xdevapigetNodeSession($connection_uri);
// "USE world_x"
$schema = $nodeSession->getSchema("world_x");
// Specify collection to use
$collection = $schema->getCollection("countryinfo");
// Query the Document Store
$result = $collection->find('_id = "USA"')->fields(['Name as
Country','geography as Geo','geography.Region'])->execute();
// Fetch/Display data
$data = $result->fetchAll();
var_dump($data);
?>

10. Resource Groups
Groups can be established so that threads execute according to the resources available to the group. Group attributes enable control
over its resources, to enable MySQL supports creation and management of resource groups, and permits assigning threads running
within the server to particular group or restrict resource consumption by threads in the group. DBAs can modify these attributes as
appropriate for different workloads.
For example, to manage execution of batch jobs that need not execute with high priority, a DBA can create a Batch resource group, and
adjust its priority up or down depending on how busy the server is. (Perhaps batch jobs assigned to the group should run at lower priority
during the day and at higher priority during the night.) The DBA can also adjust the set of CPUs available to the group.
CREATE RESOURCE GROUP Batch
TYPE = USER
VCPU = 2-3 -- assumes a system with at least 4 CPUs
THREAD_PRIORITY = 10;
INSERT /*+ RESOURCE_GROUP(Batch) */ INTO t2 VALUES(2);
39

11. Histograms - Indexing without indexes!
A histogram is an approximation of the data distribution for a column. It can tell you with a reasonably accuray whether your data is skewed
or not, which in turn will help the database server understand the nature of data it contains.
Histograms comes in many different flavours, and in MySQL we have chosen to support two different types: The “singleton” histogram and
the “equi-height” histogram. Common for all histogram types is that they split the data set into a set of “buckets”, and MySQL automatically
divides the values into buckets, and will also automatically decide what type of histogram to create.
Note that the number of buckets must be specified, and can be in the range from 1 to 1024. How many buckets you should choose for your
data set depends on several factors; how many distinct values do you have, how skewed is your data set, how high accuracy do you need
etc. However, after a certain amount of buckets the increased accuracy is rather low. So we suggest to start at a lower number such as 32,
and increase it if you see that it doesn’t fit your needs.
40

Histograms
mysql> ANALYZE TABLE customer UPDATE HISTOGRAM ON c_mktsegment WITH 1024 BUCKETS;
+---------------+-----------+----------+---------------------------------------------------------+
| Table | Op | Msg_type | Msg_text |
+---------------+-----------+----------+---------------------------------------------------------+
| dbt3.customer | histogram | status | Histogram statistics created for column 'c_mktsegment'. |
+---------------+-----------+----------+---------------------------------------------------------+
41

Two reasons for why you might consider a
histogram instead of an index
Maintaining an index has a cost. If you have an index, every
INSERT/UPDATE/DELETE causes the index to be updated.
This is not free, and will have an impact on your
performance.
A histogram on the other hand is created once and never
updated unless you explicitly ask for it.
It will thus not hurt your
INSERT/UPDATE/DELETE-performance.
42
If you have an index, the optimizer will do what we call
“index dives” to estimate the number of records in a given
range.
This also has a certain cost, and it might become too costly
if you have for instance very long IN-lists in your query.
Histogram statistics are much cheaper in this case, and
might thus be more suitable.

12. Bye Bye MEMORY Storage Engine
The TempTable storage engine replaces the MEMORY storage engine as the
default engine for in-memory internal temporary tables. The TempTable
storage engine provides efficient storage for VARCHAR and VARBINARY
columns.
Performance is ten times better than 5.7!!
43

https://stackoverflow.com/questions/5050
5236/mysql-8-0-group-by-performance
MySQL 8.0 uses a new storage engine, TempTable, for internal temporary tables. (See MySQL Manual for details.) This
engine does not have a max memory limit per table, but a common memory pool for all internal tables. It also has its own
overflow to disk mechanism, and does not overflow to InnoDB or MyISAM as earlier versions.
The profile for 5.7 contains "converting HEAP to ondisk". This means that the table reached the max table size for the
MEMORY engine (default 16 MB) and the data is transferred to InnoDB. Most of the time after that is spent accessing the
temporary table in InnoDB. In MySQL 8.0, the default size of the memory pool for temporary tables is 1 GB, so there will
probably not be any overflow to disk in that case.
44

13. X DevAPI on by default on port 33060
45

SQL + NoSQL
Schema-less NoSQL
JSON Document Store
with ACID compliance.
And you can also access
relational data!
46
1GB documents
versus
Mongo’s 16MB!

The 10 Best Restaurants of Different Cuisines
WITH cte AS (SELECT doc->>"$.name" AS name,
doc->>"$.cuisine" AS cuisine,
(SELECT AVG(score) FROM JSON_TABLE(doc, "$.grades[*]"
COLUMNS (score INT PATH "$.score")) AS r) AS avg_score
FROM restaurants)
SELECT *, RANK() OVER (PARTITION BY cuisine ORDER BY
avg_score) AS `rank`
FROM cte ORDER BY `rank`, avg_score DESC LIMIT 10;
+------------------------------------------+------------------+-----------+------+
| name | cuisine | avg_score | rank |
+------------------------------------------+------------------+-----------+------+
| Ravagh Persian Grill | Iranian | 15.6667 | 1 |
| Camaradas El Barrio | Polynesian | 14.6000 | 1 |
| Ellary'S Greens | Californian | 12.0000 | 1 |
| General Assembly | Hawaiian | 11.7500 | 1 |
| Catfish | Cajun | 9.0000 | 1 |
| Kopi Kopi | Indonesian | 8.6667 | 1 |
| Hospoda | Czech | 7.8000 | 1 |
| Ihop | Pancakes/Waffles | 7.2000 | 1 |
| New Fresco Toetillas Tommy'S Kitchen Inc | Chinese/Cuban | 7.0000 | 1 |
| Snowdonia Pub | English | 7.0000 | 1 |
+------------------------------------------+------------------+-----------+------+
10 rows in set, 13 warnings (1.4284 sec)
47
This query uses
JSON_TABLE to
structure the schema-less
data within a CTE and
then the CTE is queried
to get the top 10
restaurants with a
Windowing Function

The 10 Best Restaurants of Different Cuisines
The JSON_TABLE, CTE, and Windowing Function 49
This query uses JSON_TABLE to
structure the schema-less data within
a CTE and then the CTE is queried to
get the top 10 restaurants with a
Windowing Function
FROM restaurants)
+------------------------------------------+------------------+-----------+------+
+------------------------------------------+------------------+-----------+------+
| Catfish | Cajun | 9.0000 | 1 |
| Hospoda | Czech | 7.8000 | 1 |
+------------------------------------------+------------------+-----------+------+

Download Today
https://dev.mysql.com/downloads/mysql/
Or Docker images -> https://hub.docker.com/_/mysql/
50

The Unofficial MySQL 8 Optimizer Guide
51
http://www.unofficialmysqlguide.com/
Server Architecture
B+tree indexes
Explain
Optimizer Trace
Logical Transformations
Example Transformations
Cost-based Optimization
Hints
Comparing Plans
Composite Indexes
Covering Indexes
Visual Explain
Transient Plans
Subqueries
CTEs and Views
Joins
Aggregation
Sorting
Partitioning
Query Rewrite
Invisible Indexes
Profiling Queries
JSON and Generated Columns
Character Sets

Whew!
More features being added!
52

MySQL
Group
Replication
MySQL 5.7 or later
53
MySQL Group Replication is a MySQL Server plugin that enables you to create
elastic, highly-available, fault-tolerant replication topologies.
There is a built-in group membership service that keeps the view of the group
consistent and available for all servers at any given point in time. Servers can leave
and join the group and the view is updated accordingly. Sometimes servers can
leave the group unexpectedly, in which case the failure detection mechanism
detects this and notifies the group that the view has changed. This is all automatic.

We have gone
About as far as
we can for now!
54

Buy My Book (please!)
55
What you need
to know to use
the MySQL
JSON data type
with lots of
examples!

Thanks!
Contact me:
@stoker
david.stokes@oracle.com
slideshare.net/davidmstokes
Elephantdolphin.blogger.com
56

PHP Detroit -- MySQL 8 A New Beginning (updated presentation)

More Related Content

What's hot (20)

Similar to PHP Detroit -- MySQL 8 A New Beginning (updated presentation) (20)

More from Dave Stokes (20)

Recently uploaded (20)

PHP Detroit -- MySQL 8 A New Beginning (updated presentation)