MySQL performance tuning

MySQL
Performance
Tuning
Anurag Srivastava

Where to Analyze ?
● Development to catch bad queries early
● Production to catch good queries starting to go bad

Query
Performance
always shows it
● Workload Changes
● Data Size
● Changing Query Plans
● Upgrading MySQL Version
● Changing MySQL Settings
● Hardware Changes

Queries
The IN clause in MySQL is very fast!
● Select ... Where idx IN(1,23,345,456)
Keep column alone on left side of condition
● Select ... Where func(idx) = 20 [index ignored]
● Select .. Where idx = otherfunc(20) [may use index]
Avoid % at the start of LIKE on an index
● Select ... Where idx LIKE(‘ABC%’) can use index
● Select ... Where idx LIKE(‘%XYZ’) must do full table scan

Queries
Use EXPLAIN to see the execution plan for the query
● Check the index usage
● Check rows scanned
If column used in ORDER BY clause are indexed they help with
performance.
Try to convert <> operator to = operator
● = operator increases chances of index usage
Avoid using SELECT *
● Forces full table scan
● Wastes network bandwidth

Index
● Database structure that can be
associated to a table
● Indexes are usually used to search
table faster
● Without indexes it would be needed
to scan the whole table
● Table may have multiple indexes

When to use Indexes
● Do not start using indexes before you need them
● Only large tables need indexes
● Large table: Multiply the record count by the field's
length
● Monitor the slow query logs

What is the
slow query
log ?
● Show information of queries that take long time
to execute
● Enable with these lines to your server my.cnf
file
log_slow_queries = 1;
slow_query_log_file = <some file name>;
● Use mysqldumpslow command to get
summaries of slow queries

Index
● In MyISAM data pointers point to physical
offset in the data file
○ All indexes are essentially equivalent
● In Innodb
○ PRIMARY KEY (Explicit or Implicit) - stores data in leaf
pages of the index, not pointer
○ Secondary Indexes - store primary key as data pointer.

Index - Tricky with Multiple Columns
Index(A,B,C) - Order of columns matters
Will use Index for lookup (All listed keyparts)
● A>5
● A=2 AND B>6
● A=2 AND B=6 AND C=7
● A=5 AND B IN (2,3) AND C>5
Will not use Index
● B>5
● B=6 AND C=7
Will Use Part of Index
● A>5 AND B=4 - range on first column, only use this key part
● A=5 AND B>6 AND C=2 - range on second column, use 2 parts

Index-join
Indexes speed up joins
● SELECT X.A, Y.B FROM X,Y WHERE X.C = ‘FL’ and Y.A
= X.A ;
The Filter is on column C of table X
● Table X needs an index on column C for the filter
Table Y is joined to table X by column A
● Table Y needs an index on column A

Index-sort
select * from players order by score desc limit 10
● Will use index on score column
● Without index MySQL will do filesort (external sort) which
is very expensive)
Often combined with using index for lookup
Select * from players where country = “INDIA” order by score
desc limit 10
● Best served by Index on (country, score)

Don't want
MySQL
filesort?
By default, MySQL sorts all GROUP BY col1, col2,
… queries as if you specified ORDER BY col1,
col2, … in the query as well.
If a query includes GROUP BY but you want to
avoid the overhead of sorting the result, you can
suppress sorting by specifying ORDER BY NULL.
SELECT a, COUNT(*) FROM bar GROUP BY a ORDER BY NULL;

Index - multi column efficient sorting
KEY(A,B)
Will use Index for sorting
● ORDER BY A - Sorting by leading column
● A=5 ORDER BY B - EQ filtering by 1st and sorting by 2nd
● ORDER BY A DESC, B DESC - Sorting by 2 columns in same order
● A>5 ORDER BY A - Range and sorting on the 1st column
Will NOT use Index for sorting
● ORDER BY B - Sorting by second column in the index
● A>5 ORDER BY B - Range on first column, sorting by second
● A IN (1,2) ORDER BY B - In-Range on first column

Covering
Index
● Applies to index use for specific query, not type
of index.
Reading Index ONLY and not accessing the data
SELECT status from orders where
customer_id = 123
KEY(customer_id, status)
● Index is typically smaller than data
● Access is a lot more sequential

Overhead of
Indexing ● Indexes are costly. Do not add more than you
need.
○ In most cases extending index is better that adding
new one
● Writes
○ Updating indexes is often major cost of database writes
● Reads
○ Wasted space on disk and in memory.
○ Additional overhead during query optimization

Tools ● Explain
● Performance_Schema (MySQL 5.5+)
● MySQL sys Schema (MySQL 5.7.7+)
● pt-query-digest (Percona Toolkit)
● pt-index-usage (Percona Toolkit)
● mysqltuner

Explain
EXPLAIN is one of the most powerful
tools at your disposal for understanding
and optimizing troublesome MySQL
queries

Explain
Cost: 239*4145*1 = 990655

Explain
Alter table City add index c1 (Name)
Alter table Country add index c2 (Name)

Explain
Old Cost: 239*4145*1 = 990655 New Cost: 1*1*1 = 1

Performance Schema
Performance Schema is a mechanism to give user an
insight of what is happening behind the scene when
MySQL server is running.

Performance
Schema
● Introduced in MySQL 5.5
● New storage engine : Performance
Schema
● New Database : Performance Schema
● SQL user interface

Performance Schema - Instruments
● Name of the monitored activity
● Tree like structure separated by ‘/’.
● Left to right: More generic to more specific.
statement/sql/create_table
statement/sql/select
● 1000+ instruments in MySQL 5.7

Performance Schema - Instruments
TABLE SETUP_INSTRUMENTS
NAME ENABLED TIMED
statement/sql/select YES YES
statement/sql/create_table YES NO
statement/com/Create DB NO NO
…... ... ...

Performance Schema - use cases 1
● Multiple queries running for long on MySQL Server
● Few long running query (taking lots of time)
● No idea which one
● No idea why
● What to do ?

SELECT digest_text , count_star, avg_timer_wait FROM
events_statements_summary_by_digest ORDER BY avg_timer_wait DESC
LIMIT 1G;

Statement giving errors (or warning)
SELECT DIGEST_TEXT, SCHEMA_NAME, COUNT_STAR, SUM_ERRORS, SUM_WARNINGS FROM
performance_schema.event_statements_summary_by_digest WHERE SUM_ERRORS >0 ORDER BY
SUM_ERRORS DESC limit 1G;
select digest_text, schema_name, count_star, sum_errors, sum_warnings, first_seen, last_seen from
performance_schema.events_statements_summary_by_digest where digest =
'47d2c74d34a16ad5d2193a3706374c3e' G;

Problem Statement
● Multithreaded environment
● Session stuck
● Why
● What to do ??

Diagnosis
● What t1 is waiting
select * from events_waits_current where thread_id = t1;
Say t1 is waiting for mutex1 (column: object_instance_begin)
● Lets see who has taken this mutex1
select * from mutex_instances where object_instance_begin = mutex1;
So thread t2 is holding mutex1 (column: locked_by_thread_id)
● Find out what thread t2 is waiting for
select * from events_waits_current where thread_id = t2;

Mysql sys Schema
● A set of objects that helps DBAs and developers interpret data collected by
the Performance Schema.
● Views that summarize Performance Schema data into more easily
understandable form.

Mysql sys Schema
Find out all unused indexes:
select * from sys.schema_unused_indexes;
Queries with errors or warnings:
select * from sys.statements_with_errors_or_warnings;

Pt-query-digest
● Command Line tool from Percona
Toolkit
● Process Log Manually
● Powerful Filtering
● Storing History

pt-query-digest
# 3.4s user time, 20ms system time, 37.16M rss, 99.46M vsz
# Current date: Wed Mar 29 13:58:09 2017
# Hostname: KELLGGNLPTP0129
# Files: /home/user/Downloads/SlaveDB-slow.log
# Overall: 7.25k total, 57 unique, 0.01 QPS, 0.80x concurrency ___________
# Time range: 2017-02-22 12:17:48 to 2017-03-02 12:15:43
# Attribute total min max avg 95% stddev median
# ============ ======= ======= ======= ======= ======= ======= =======
# Exec time 551459s 10s 83865s 76s 167s 1479s 19s
# Lock time 10s 0 6s 1ms 568us 65ms 348us
# Rows sent 56.58M 0 1.40M 7.99k 9.33k 25.57k 4.71k
# Rows examine 67.63G 0 1.10G 9.55M 46.53M 18.98M 3.68M
# Rows affecte 0 0 0 0 0 0 0
# Bytes sent 8.41G 0 43.24M 1.19M 2.49M 1.72M 1.03M
# Query size 13.89M 29 22.27k 1.96k 2.89k 874.90 2.06k

pt-query-digest
TOP QUERIES
# Profile
# Rank Query ID Response time Calls R/Call V/M Item
# ==== ================== ================= ===== ========== ===== =======
# 1 0x478608453385DE52 245460.6392 44.5% 8 30682.5799 35... SELECT stats_family_geo_mapping
beneficiary_mother users vhnd_pw_details delivery_details cronstats_child_immunization death_details
# 2 0x178ACC79136F8827 150300.4869 27.3% 906 165.8946 0.17 SELECT checklist_answer_summary
checklist_answers
# 3 0x621A20DE43CAD79C 53892.1029 9.8% 1625 33.1644 0.10 SELECT beneficiaries beneficiary_childs
Infant_details delivery_details users
# 4 0x18AD3F125971674C 19376.2565 3.5% 1368 14.1639 0.19 SELECT beneficiaries beneficiary_childs
Infant_details delivery_details users districts blocks sub_centers
# 5 0xA5AF83E712E06BB9 14995.0588 2.7% 940 15.9522 0.74 SELECT beneficiaries families users districts
blocks sub_centers villages
# 6 0x774412AFA6FD952C 10953.0720 2.0% 411 26.6498 1.11 SELECT stats_family_geo_mapping users
beneficiaries beneficiary_mother vhnd_pw_details delivery_details
# 7 0x8DA9B283102403CC 10652.1384 1.9% 8 1331.5173 0.75 SELECT delivery_details
stats_mother_hbnc_count families districts blocks users sub_centers stats_pnc_first_visit

pt-query-digest
QUERY DETAILS
# Query 1: 0.00 QPS, 0.43x concurrency, ID 0x478608453385DE52 at byte 9080661
# This item is included in the report because it matches --limit.
# Scores: V/M = 35585.94
# Time range: 2017-02-22 12:17:48 to 2017-03-01 01:10:30
# Attribute pct total min max avg 95% stddev median
# ============ === ======= ======= ======= ======= ======= ======= =======
# Count 0 8
# Exec time 44 245461s 29s 83865s 30683s 81750s 33043s 39323s
# Lock time 1 115ms 200us 112ms 14ms 110ms 36ms 366us
# Rows sent 0 6 0 1 0.75 0.99 0.43 0.99
# Rows examine 0 22.81M 1.47M 3.56M 2.85M 3.50M 841.46k 3.42M
# Rows affecte 0 0 0 0 0 0 0 0
# Bytes sent 0 414 0 69 51.75 65.89 28.53 65.89
# Query size 0 6.38k 817 817 817 817 0 817

pt-index-usage
● Read queries from a log and analyze how they use indexes.
● This tool connects to a MySQL database server, reads through a query log,
and uses EXPLAIN to ask MySQL how it will use each query.
● When it is finished, it prints out a report on indexes that the queries didn’t
use.
sudo pt-index-usage --ask-pass ~/Downloads/SlaveDB-slow.log

pt-index-usage
ALTER TABLE `msehat`.`sub_centers` DROP KEY `fk_village`; -- type:non-unique
ALTER TABLE `msehat`.ùsers` DROP KEY `block`, DROP KEY `device_imei`, DROP KEY
`fk_district_id`, DROP KEY `fk_villaged_id`, DROP KEY ìdx_device_imei_secondary`, DROP KEY
ìdx_parent_id`, DROP KEY `policy_id`, DROP KEY `sub_center`; -- type:non-unique
user@KELLGGNLPTP0129:/var/lib$ sudo pt-index-usage --help
pt-index-usage reads queries from logs and analyzes how they use indexes. For
more details, please use the --help option, or try 'perldoc
/usr/bin/pt-index-usage' for complete documentation.

MySQLTuner
MySQLTuner is a script written in Perl that allows you to
review a MySQL installation quickly and make
adjustments to increase performance and stability. The
current configuration variables and status data is
retrieved and presented in a brief format along with
some basic performance suggestions.

MySQLTuner
>> MySQLTuner 1.6.9 - Major Hayden <major@mhtx.net>
>> Bug reports, feature requests, and downloads at http://mysqltuner.com/
>> Run with '--help' for additional options and output filtering
[OK] Logged in using credentials from debian maintenance account.
[--] Skipped version check for MySQLTuner script
[OK] Currently running supported MySQL version 5.7.13-0ubuntu0.16.04.2
[OK] Operating on 64-bit architecture
-------- Storage Engine Statistics -------------------------------------------
[--] Status: +ARCHIVE +BLACKHOLE +CSV -FEDERATED +InnoDB +MEMORY +MRG_MYISAM
+MyISAM +PERFORMANCE_SCHEMA
[--] Data in InnoDB tables: 160M (Tables: 379)
[--] Data in MyISAM tables: 4M (Tables: 39)
[!!] Total fragmented tables: 180
-------- Security Recommendations -------------------------------------------
[OK] There are no anonymous accounts for any database users

Efficiency
Right Queries
Executed Most
Efficiently

Practical
Metrics How many rows are being scanned vs Sent ?
How much IO are we doing ?
Is there avoidable overhead ?
● Filesorts ?
● TMP Tables ?

Areas of Improvement
Architecture
Hardware
Schema and Queries
MySQL version
MySQL Configuration

MySQL performance tuning

More Related Content

What's hot (20)

Similar to MySQL performance tuning (20)

Recently uploaded (20)

MySQL performance tuning