Upgrade to MySQL 5.7 and latest news planned for MySQL 8

Copyright © 2015, Oracleand/orits affiliates. Allrights reserved. |
Upgrading to MySQL 5.7
and latest news from OOW
Ted Wennmark
ted.wennmark@oracle.com

Program Agenda
MySQL 5.7
MySQL 5.7 Upgrade
MySQL 8
1
2
2
3

Program Agenda
MySQL 5.7
MySQL 5.7 Upgrade
MySQL 8
1
2
3
3

MySQL 5.7 is GA!
4
Enhanced InnoDB: faster online & bulk
load operations
Replication Improvements(incl. multi-
source,multi-threaded slaves...)
New Optimizer Cost Model: greater user
control& better query performance
Performance Schema Improvements
MySQLSYS Schema
Performance & Scalability Manageability
3 X Faster than MySQL 5.6
ImprovedSecurity: safer initialization,
setup & management
Native JSON Support
Andmany more new features andenhancements.Learnmore at: dev.mysql.com

MySQL 5.7 Sysbench Benchmark: SQL Point Selects
3x Faster than MySQL 5.6
4x Faster than MySQL 5.5 1,600,000 QPS
0
200 000
400 000
600 000
800 000
1 000 000
1 200 000
1 400 000
1 600 000
1 800 000
8 16 32 64 128 256 512 1 024
QueriesperSecond
Connections
MySQL5.7:SysbenchOLTP Read Only (SQLPoint Selects)
MySQL 5.7
MySQL 5.6
MySQL 5.5
Intel(R)Xeon(R) CPU E7-8890 v3
4sockets x 18 cores-HT (144 CPU threads)
2.5Ghz, 512GB RAM
Linuxkernel 3.16
5

MySQL 5.7 Sysbench Benchmark: OLTP Read Only
0
200 000
400 000
600 000
800 000
1 000 000
1 200 000
8 16 32 64 128 256 512 1 024
QueriesperSecond
Connections
MySQL5.7:SysbenchOLTP Read Only
MySQL 5.7
MySQL 5.6
MySQL 5.5
2.5Ghz, 512GB RAM
Linuxkernel 3.16
~ 1,000,000 QPS
6

MySQL 5.7 Sysbench Benchmark: OLTP Read Write
1.5x Faster than MySQL 5.6
0
100 000
200 000
300 000
400 000
500 000
600 000
700 000
8 16 32 64 128 256 512 1 024
QueriesperSecond
Connections
MySQL5.7:SysbenchOLTP Read Write
MySQL 5.7
MySQL 5.6
MySQL 5.5
2.5Ghz, 512GB RAM
Linuxkernel 3.16
7

• Optimizer and Parser refactoring
– Readability, maintainability and stability
– Separate parsing, optimizing, execution stages
– Easier feature additions, with lessened risk
• New hint framework
– Easier to manage
– With support for additional new hints
• ImprovedJSON EXPLAIN
• EXPLAINfor running thread
• New Cost based Optimizer
– Easier to extend
– Configurable and tunable
• mysql.server_cost and mysql.engine_cost tables
• APIfor where data resides:on diskor in cache
• InnoDB for internal temp tables
• Better ONLY_FULL_GROUP_BYmode
• Many specific new optimizations
• Generated Columns
MySQL 5.7: Optimizer Improvements
8
Queries execute faster, while using less CPU and disk space!

Optimizer Cost Model: Performance Improvements
DBT-3 (Size Factor 10, CPU bound)
0
20
40
60
80
100
Q3 Q7 Q8 Q9 Q12
Executiontimerelativeto5.6(%)
5out of 22 queriesget a much improvedquery plan (others remainthe same)
MySQL5.6
MySQL5.7
9

0
20
40
60
80
100
Q2 Q18
CPU bound
5.6
5.7
Optimizer Cost Model: Performance Improvements
DBT-3 (Size Factor 10)
2 out of 22 queriesgeta significantlyimprovedqueryplan (others remainthe same)
0
20
40
60
80
100
Q2 Q18
Diskbound
5.6
5.7
10

MySQL 5.7: Query Rewrite Plugin
• New pre and post parse query rewrite APIs
– Users can write their own plug-ins
• Provides a post-parse query plugin
– Rewrite problematic queries without the need to make application changes
– Add hints
– Modify join order
– Many more …
• Improve problematic queries from ORMs, third party apps, etc
• Eliminates many legacy use cases for proxies
11

MySQL 5.7: Optimizer - Cost Info in JSON EXPLAIN
• Expanded JSON EXPLAIN
– Now includes all available cost info
– Used for Visual Explain In MySQL Workbench
{
"query_block": {
"select_id": 1,
"cost_info": {
"query_cost": "200.40"
},
"table": {
"table_name": "nicer_but_slower_film_list",
"access_type": "ALL",
"rows_examined_per_scan": 992,
"rows_produced_per_join": 992,
"filtered": 100,
"cost_info": {
"read_cost": "2.00",
"eval_cost": "198.40",
"prefix_cost": "200.40",
"data_read_per_join": "852K"
},
"used_columns": [
"FID",
"title",
"description",
"category",
"price",
"length",
"rating",
"actors"
],
...
12

MySQL 5.7: JSON
• NativeJSON data type
– Native internal binary format for efficient processing & storage
• Built-in JSON functions
– Allowing you to efficiently store, search, update, and manipulate Documents
• JSON Comparator
– Allows for easy integration of Document data within your SQL queries
• Indexing of Documents using Generated Columns
– InnoDB supports indexes on both stored and virtual Generated Columns
– New expression analyzer automatically uses the best “functional” index available
• New inline syntax for easy SQL integration
13

MySQL 5.7: JSON and Text Datatype Comparison
# With feature column as JSON type
SELECT DISTINCT
feature->"$.type" as json_extract
FROM features;
+--------------+
| json_extract |
+--------------+
| "Feature" |
+--------------+
1 row in set (1.25 sec)
Unindexed traversal of 206K documents
# With feature column as TEXT type
SELECT DISTINCT
feature->"$.type" as json_extract
FROM features;
+--------------+
| json_extract |
+--------------+
| "Feature" |
+--------------+
Explanation: Binary format of JSON type is very efficient at searching. Storing as TEXT
performs over 10x worse at traversal.
14

MySQL 5.7: Functional Indexes with JSON
15
ALTER TABLE features ADD feature_type VARCHAR(30) AS (JSON_UNQUOTE(feature->'$.type'));
Query OK, 0 rows affected (0.01 sec)
Records: 0 Duplicates: 0 Warnings: 0
ALTER TABLE features ADD INDEX (feature_type);
Query OK, 0 rows affected (0.73 sec)
Records: 0 Duplicates: 0 Warnings: 0
SELECT DISTINCT feature_type FROM features;
+--------------+
| feature_type |
+--------------+
| "Feature" |
+--------------+
From table scan on 206K documents to index scan on 206K materialized values
Meta data change only(FAST).
Does not need to touch table..
Creates index only,does not
touch rowdata.
Down from 1.25 sec to 0.06 sec
Create table features (
id int primary key,
feature json
)

Memory Instrumentation
• Aggregates statisticsby
– Type of memory used
(caches, internal buffers, …)
– Thread/account/user/host
indirectlyperforming the
memory operation
• Attributes include
– Memory used (bytes)
– Operation counts
– High/Low Water Marks
Statement Instrumentation
• Stored Procedures
• Stored Functions
• Prepared Statements
• Transactions
Additional Information
• Replication slave status
• MDL lock instrumentation
• Status and variables per
thread
• Server stage tracking
• Track long running SQL
• Improvedconfiguration
and ease-of-use
• All while reducing total
footprint and overhead
MySQL 5.7: Performance Schema
16

MySQL 5.7: SYS Schema
Helper objects for DBAs, Developersand Operations staff
• Helps simplify DBA / Ops tasks
- Monitor server health, user, host statistics
- Spot, diagnose, and tune performance issues
• Easy to understand views with insights into
- IO hot spots, Locking, Costly SQL statements
- Schema, table and index statistics
• SYS is similar to
- Oracle V$ catalog views
- Microsoft SQL DMVs (Dynamic Mgmnt Views)
17

• Replaced custom code with Boost.Geometry
– For spatial calculations
– For spatial analysis
– Enabling full OGC compliance
– We’re also Boost.Geometry contributors!
• InnoDB R-tree based spatial indexes
– Full ACID, MVCC, & transactional support
– Index records contain minimum bounding box
• GeoHash
• GeoJSON
• Helper functions such as ST_Distance_Sphere() and ST_MakeEnvelope()
MySQL 5.7: GIS Improvements
18

• Native Partitioning
– Eliminates previous limitations
– Eliminates resource usage problems
– Transportable tablespace support
• Native Full-Text Search
– Including full CJK support!
• Native Spatial Indexes
• Transparent page compression
• Support for 32K and 64K pages
– Use with transparent page compression for
very high compression ratios
• General TABLESPACEsupport
– Store multiple tables in user defined shared
tablespaces
• Support for MySQL Group Replication
– High priority transactions
• Improvedsupport for cache preloading
– Load your hottest data loaded at startup
• Configurable fill-factor
– Allows for improvements in storage footprint
• Improvedbulk-dataload performance
• Resize the InnoDB Buffer Pool online
MySQL 5.7: InnoDB Improvements
19

MySQL 5.7: InnoDB Compression
Thank you, SanDisk Fusion-io
• Transparent Page Level Compression
– Happens transparently in background threads
– Managed entirely within the IO layer
– Uses sparse file and "hole punching" support in OS kernels and File Systems
• Reduces IO
– ImprovesMySQL performance
– Improvesstorage efficiency
– Reduces write cycles,thus increasing SSD lifespan
• Applies to all InnoDB data, including the system tablespace and UNDO logs
20

• AES 256 Encryption now the default
• Password rotation policies
– Can be set globally, and at the user level
• Deployment:enable secure unattended
install by default
– Random password set on install
– Remove anonymous accounts
– Deployment without test account, schema,
demo files
• Easier instance initialization and setup:
mysqld –initialize
• New detection and support for systemd
• SSL
– Enabled by default
– Auto-detection of existing keys and certs
– Auto generation of keys and certs when needed
– New helper utility: mysql_ssl_rsa_setup
– New --require_secure_transport option to
prevent insecure communications
– Added SSL support to binary log clients
• ExtendedProxy User Support
– Added Built-in Authentication Plugins support
for ProxyUsers
– Allows multiple users to share a single set of
managed privileges
MySQL 5.7: Security Improvements
21

MySQL 5.7: Server-Side Statement Timeouts
Thank you Davi Arnaut!
• Server side statement timeouts
– Global for server, per session, or for individual SELECT statements
• Expanded to Windows and Solaris, restricted by removing USER option
SELECT /*+ MAX_EXECUTION_TIME(1000) */ * FROM my_table;
22

• GTID enhancements
– On-line, phased deployment of GTIDs
– Binarylogging on slave now optional
• Enhanced Semi-synchronousreplication
– Write guaranteed to be received by slave
before being observed by clients of the master
– Option to wait on Acks from multiple slaves
• Multi-SourceReplication
– Consolidate updates from multiple Masters
into one Slave
• Dynamic slave filters
• 8-10x Faster slave throughput
– Often removes slave as a bottleneck; keep pace
with master with 8+ slave threads
– Option to preserve Commit order
– Automatic slave transaction retries
MySQL 5.7: Replication Improvements
23
0%
50%
100%
150%
200%
250%
1 8 24 48
Slave Threads
Slave throughput vs. 96 Thread Master

• Multi-Source Replication
– Consolidateupdates from multiple
Masters into one Slave
• Consolidated view of all shards
• More flexible topologies
• Centralized point for backups
– Compatible with Semi-Sync
Replication & enhanced MTS
• Performance Schema tables for
monitoring slave
• Online Operations: Dynamic
Replication Filters, switch master
MySQL 5.7: Replication Improvements
Slave
24
Binlog
Master 1
Binlog
Master 2
…
…
Binlog
Master N
IO 1
Relay 1
Coordinator
W1 W2 … WX
IO 2
Relay 2
Coordinator
W1 W2 … WX
…
…
Coordinator
W1 W2 … WX
IO N
Relay N
Coordinator
W1 W2 … WX

MySQL Repos
• Distributions
– Oracle, Red Hat, CentOS
– Fedora
– Ubuntu, Debian
– SUSE
• Official MySQL Docker Image from Oracle
• Coming Soon
– Preconfigured Containers
– Improved support for popular DevOps
deployment tools
https://dev.mysql.com/downloads/repo
MySQL on GitHub
• Git for MySQL Engineering
– Fast, flexible and great for a distributed team
– Great tooling
– Largeand vibrant community
• GitHub for MySQL Community
– Easyand fast code availability to the community
and to downstream projects
– Pull Requests
https://github.com/mysql
25

Program Agenda
MySQL 5.7
MySQL 5.7 Upgrade
MySQL 8
1
2
26
3

Why upgrade?
• Performance/Scalability improvements
• New features
• Stay on fully supported release
• Security improvements
27

Performance/Scalability
• MySQL Scales better:
– MySQL5.1 scales to ~4 cores
• Improvements in optimizer
– Better query plans with MRR, BKA and more
– More work in storage engine with ICP
– Tunable cost model in 5.7
28
https://www.mysql.com/why-mysql/benchmarks/

Support Life Cycle
Release GADate PremierSupportEnd ExtendedSupportEnd SustainingSupportEnd
MySQL5.0 Oct2005 Dec 2011 N/A Indefinite
MySQL5.1 Dec 2008 Dec 2013 N/A Indefinite
MySQL5.5 Dec 2010 Dec 2015 Dec 2018 Indefinite
MySQL5.6 Feb2013 Feb2018 Feb2021 Indefinite
MySQL5.7 Oct2015 Oct2020 Oct2023 Indefinite
29
http://www.oracle.com/us/support/library/lifetime-support-technology-069183.pdf

MySQL Security
• SSL enabled by default
• Force root password during install
• Stronger passwords
• MySQL Enterprise Edition:
– MySQLAudit
– MySQLMonitor/Backup
– MySQLFirewall
– MySQLEncryption
30

MySQL Upgrade – Supported Paths
• All minor upgrades like 5.7.9 to 5.7.10 or doing a jump like 5.7.9 to 5.7.11 is
supported.
• Upgrading one major (5.6 - 5.7) release level is supported. It’s
recommended upgrading to the latest 5.6 release before moving to 5.7.
• Doing long-jumps (5.1 – 5.7) is supported if you upgrade one release level
at a time.
• Direct upgrades that skip a release level (for example, upgrading directly
from MySQL 5.5 to 5.7) is not recommended.
31

MySQL Upgrade – Before you start!
• Review the “Release Notes” for target MySQL version
• Review the manual page on upgrade for this MySQL version, take special
care of sections covering:
– “Server and Status Variables and Options Added, Deprecated, or Removed …”
– “CheckingWhether Tables or Indexes Must Be Rebuilt”
– “Changes Affecting Upgrades to MySQL 5.N”
• Create a plan for your upgrade/downgrade path
• Create a backup of your database
32
http://dev.mysql.com/doc/refman/5.7/en/upgrading.html

MySQL Upgrade – In-place Upgrade
1. Do all necessary preparations/plans as explained earlier
2. Do a graceful shutdown of MySQL (innodb_fast_shutdown=0)
3. Do upgrade, replace the old binaries with the new ones
4. Start new MySQL Server using new binaries
5. Run mysql_upgrade binary to upgrade internal mysql repository
6. Done!
34
http://mysqlserverteam.com/upgrading-directly-from-mysql-5-0-to-5-7-using-an-in-place-upgrade/
http://dev.mysql.com/doc/refman/5.7/en/upgrading.html#upgrade-procedure-inplace

MySQL Upgrade – Logical Upgrade
2. Do a dump of only the user databases/schemas (skipping the mysql
system schema, remember stored procedures, views and functions)
3. Do a clean installation of new MySQL version (remember to migrate
important my.cnf options)
4. Load the dump file into the new MySQL server instance
5. Done!
35
http://dev.mysql.com/doc/refman/5.7/en/upgrading.html#upgrade-procedure-logical
http://mysqlserverteam.com/upgrading-directly-from-mysql-5-0-to-5-7-with-mysqldump/

MySQL Upgrade – Using Replication “simplified”
2. Restore backup from production (5.6) database to new slave server (5.7)
3. Start replication from master to slave
4. Once slave server has caught up with master
5. Change application to connect to slave (5.7)
6. Done!
36
MySQL5.6 MySQL5.7
Application
Replication
http://dev.mysql.com/doc/refman/5.7/en/replication.html

MySQL Upgrade – Regression Problems
• Explain is you friend!
– Optimizer_switch can help get back old query plans for slow queries
• Monitor recourses/MySQL before/after to spot changes
– Without proper monitoring you have little chance to find root cause of problem
• Did you change more than just MySQL version?
– Normally we see people refresh HW/OS at the same time with creates many possible
regression scenarios
• Use PERFORMANCE_SCHEMA and SYS schema to spot problems.
37

Upgrading from MySQL 5.1 or 5.5 to MySQL 5.7?
News in MySQL 5.6
• Need version 5.1.20 of Connector/J (due to retire of “SET OPTION”)
• InnoDB “Files per table” now default
• The Query Cache is disabled by default
• 8 new reserved key words have been defined (“get” and “partition”)
• New data format of TIMESTAMP and DATETIME (microseconds)
• Password on command line causes warnings (problematic for scripts)
• New optimizer features may cause different query plans
• Old password are not allowed by default
38
https://dev.mysql.com/doc/relnotes/mysql/5.6/en/

New Servers Default’s in MySQL 5.7
• innodb_strict_mode is “ON”
• InnoDB file format is now Barracuda
• sql_mode option “NO_AUTO_CREATE_USER, STRICT_TRANS_TABLES,
ERROR_FOR_DIVISION_BY_ZERO, NO_ZERO_DATE, NO_ZERO_IN_DATE” is
now default
• More changes to optimizer_switch, may cause changed query plans!
– Use explain to identify problem and set optimizer_switch to solve problem
• MySQL accounts expire after 360 days. The default-password-lifetime is 360
• … and many more, see link below for more details
39
http://mysqlserverteam.com/improved-server-defaults-in-5-7/

Removed Features in MySQL 5.7
• Old password are not allowed in MySQL 5.7
• Remove IGNORE for ALTER TABLE in 5.7, WL#7395
• Remove of INSERT DELAYED, WL#6073
• Remove YEAR(2) support, WL#6263
• Deprecate Unique Option Prefixes, WL#6978
• Deprecate skip-innodb in 5.6 and remove it in 5.7, WL#7976
• For complete list: http://mysqlserverteam.com/removal-and-deprecation-
in-mysql-5-7/
40

MySQL 5.7 – Installation changes
• mysql_install_db is removed
– Replaced by mysqld --initialize
• MySQL is more Secure by default:
– TLS/SSLenabled by default and for some packages keys are created by default
– No creation of test database
– No creation of anonymous users
– Only one root user and password is set at installation time
– All user accounts will automatically have their passwords rotated after 360 days
– New mysql_ssl_rsa_setup tool
• Use mysqld --initialize-insecure to get old behavior
41

Program Agenda
MySQL 5.7
MySQL 5.7 Upgrade
MySQL 8
1
2
42
3

• Transactional (InnoDB) Data Dictionary
• SQL roles
• INVISIBLE indexes (ALTER TABLE Country ALTER INDEX c INVISIBLE)
• Persistent runtime configuration changes (SET PERSIST)
• MySQL privilege tables are now InnoDB
• Auto-increment counter value will now persist across server restarts!
• Indexing of PS tables
• UTF8MB4 as the default character set
MySQL 8 (DM1)
43
http://dev.mysql.com/doc/relnotes/mysql/8.0/en/news-8-0-0.html
http://mysqlserverteam.com/the-mysql-8-0-0-milestone-release-is-available/

Copyright © 2015, Oracleand/orits affiliates. Allrights reserved. | 44
MySQL 8.0: Common Table Expressions CTE
• “With queries”
• Both Recursive and Non-Recursive Forms
• Simplifies writing complex SQL:
WITH t1 AS (SELECT * FROM tblA WHERE a=‘b’)
SELECT * FROM t1;
Feature Request
fromDevelopers

Copyright © 2015, Oracleand/orits affiliates. Allrights reserved. | 45
MySQL 8.0: Invisible Indexes
• Indexes are “hidden” to the MySQL Optimizer
– Not the same as “disabled indexes”
– Contentsare fully up to date and maintained by DML
• Two use cases:
– Soft Delete (Recycle Bin)
– Staged Rollout
Feature Request
fromDBAs

MySQL 8.0: Security Roles
• Fully Function, Flexible, Properly Architected Roles
• Create and Drop Roles, Grant to Roles
• Grant Roles to Roles, Grant Roles to Users
• Limit Hosts that can use roles, Define Default Roles
• Decide what roles are applicable during a session
• And even visualize Roles with SQL function ROLES_GRAPHML()

• New! Source code documentedwith
Doxygen
• New! Plugin Infrastructure!
• Expanded GIS Support
• Expanded Query Hints Support
• ImprovedScan Query Performance
• ImprovedBLOB Storage
• ImprovedMemcached Interface
• ScalabilityImprovements
• Parser Refactoring
• New! Document Store
• Improvementsto Temporary Tables
• C++11 and Toolchain Improvements
• Replication Applier Thread Progress
Reports
• GTID_PURGEDalways settable
47
MySQL 8.0: All this and more….

MySQL: Additional Info
• http://mysqlserverteam.com/whats-new-in-mysql-5-7-first-release-candidate/
• http://mysqlserverteam.com/json-labs-release-overview/
• http://mysqlserverteam.com/category/performance/optimizer/
• http://mysqlserverteam.com/category/innodb/
• http://mysqlserverteam.com/category/mysql/performance-schema/
• http://mysqlserverteam.com/category/gis/
• http://mysqlserverteam.com/category/full-text-search/
• http://mysqlserverteam.com/category/docstore/
• http://dev.mysql.com/doc/refman/5.7/en/upgrading.html
48

Thank you!

Upgrade to MySQL 5.7 and latest news planned for MySQL 8

More Related Content

What's hot (20)

Viewers also liked (9)

Similar to Upgrade to MySQL 5.7 and latest news planned for MySQL 8 (20)

More from Ted Wennmark (10)

Recently uploaded (20)

Upgrade to MySQL 5.7 and latest news planned for MySQL 8