Why Your Database Queries Stink -SeaGl.org November 11th, 2016

Why Your
Database
Queries Stink!
SeaGL November 11th 2016

Roughly 2% of developers have any training in Structured Query Language
but are mystified why their database queries do not perform. They get stuck
with the N+1 problem, denormalize their data, add object relational
mappers that add layers of complexity, move to NoSQL system that do not
scale or allow complex logical searchers, and end up frustrated. This
sessions covers how to use Venn Diagrams to develop joins, how to
efficiently think in sets, how to use the database to do the heavy lifting, the
smart use of indexing, how query plans are developed, query re-write
plugins, and more. If you use a relational database to store data and are not
100% confident in your queries you need to be in this session.
2

Hello!I am Dave Stokes
MySQL Community Manager
david.stokes@oracle.com @Stoker elephantanddolphin.blooger.com
Slideshare.net/davidmstokes
3

21 Years Old
MySQL has been part of
Oracle’s family of databases
for six years.
MySQL 8
MySQl 5.7 is the current release
but the next version will be
MySQL 8. Big feature is real
time data dictionary
Group Replication
Active master-master
replication.
JSON
A new native JSON datatype to
store documents in a column of
a table
Document Store
Programmers not know SQL
but need a database? X Devapi
allows them to use RDMS from
language of choice
Encryption
Use Oracle Key Vault to
encrypt your data at rest.
4

1.
So your write a
query
Then what?You have produced
a statement in SQL
for processing by
a database
5

Structured Query Language is a
special-purpose programming
language designed for managing
data held in a relational database
management system (RDBMS), or
for stream processing in a
relational data stream
management system. --
Wikipedia
6

× Efficient storage of data, minimal duplication
× Been around for decades
× Based on set theory and relational theory
Need normalized data where components are
divided up into logical components -> Requires
data to be architected and data integrity rules are
enforced
7

Big
Roughly 2% of developers receive any formal training
in SQL, relational theory, etc.
And they wonder why their queries perform poorly.
8

Goal:
Using the MySQL World
Database find the name
of all cities and their
corresponding country
name
Query:
SELECT City.Name,
Country.Name
FROM City
JOIN Country ON
(County.Code =
City.CountryCode);
9

Query:
SELECT City.Name,
Country.Name
FROM City
JOIN Country ON
(County.Code =
City.CountryCode);
Previously someone has
split the data for cities
and countries into two
tables. And they
established a link
between the two tables
use three character
country codes to link the
two together.
10

A fair amount of developers see the database
server as an arcane, messy, dank factory
spewing smoke and mess.
And they may be right.
11

$query = “SELECT City.Name, Country.Name
FROM City JOIN Country On (County.Code =
City.CountryCode”;
$result = mysqli_query($link,$query);
12

One:
Is your system
allowed to connect
to server?
Two:
Are you using a valid
account? Are there
limits to this
account?
Three:
Do you have proper
permission to
access the data you
seek?
15

Login
Okay
System
Okay
Permissions
Good
16

One:
Is your system
allowed to connect
to server?
Two:
Are you using a valid
account? Are there
limits to this
account?
Three:
Do you have proper
permission to
access the data you
seek?
17

IS the Syntax correct?Checking the basics
Build query planFind the best way to assemble the data
Parse the queryFind the pieces needed
18

Cost ModelHow do we return the needed data the least
expensive way?
19

Reading from disk
is slow, 100,000
times slower than
reading from
memory.
Note: Many vendors are
looking at changing the
cost model to
accommodate new
technologies, mixes of
hardware technologies,
and latencies.
See mysql.server_cost and mysql.engine_cost tables for details
20

The optimizer knows from stores statistics
just what is has had to do on similar
queries in the past and guesstimates
What it will take to get this query done.
22

{
"query_block": {
"select_id": 1,
"cost_info": {
"query_cost": "5132.14"
},
"nested_loop": [
{
"table": {
"table_name": "Country",
"access_type": "ALL",
"possible_keys": [
"PRIMARY"
],
"rows_examined_per_scan": 239,
"rows_produced_per_join": 239,
"filtered": "100.00",
"cost_info": {
"read_cost": "6.00",
"eval_cost": "47.80",
"prefix_cost": "53.80",
"data_read_per_join": "61K"
},
"used_columns": [
"Code",
"Name"
]
}
},
{
"table": {
"table_name": "City",
"access_type": "ref",
"possible_keys": [
"CountryCode"
],
"key": "CountryCode",
"used_key_parts": [
"CountryCode"
],
"key_length": "3",
"ref": [
"world.Country.Code"
],
"filtered": "100.00",
"cost_info": {
"read_cost": "4231.95",
"eval_cost": "846.39",
},
"used_columns": [
"Name",
"CountryCode"
]
}
}
]
}
}
23

"nested_loop": [
{
"table": {
"table_name": "Country",
"access_type": "ALL",
"possible_keys": [
"PRIMARY"
],
"filtered": "100.00",
"cost_info": {
"read_cost": "6.00",
"eval_cost": "47.80",
},
"used_columns": [
"Code",
"Name"
]
Nested loop join for Country Table
We DO have an index for the JOIN :-)
Statistics
24

Every new column sought
adds a factorial of
complexity to getting the
data. Adding a third column
would take our simple
query from two to six
possibilities!!!!!!
25

Some Databases allow
locking query plans so
once you have it optimized
you can get consistent
results.
MySQL does not do this!!!!
26

28
MySQL Workbench
(Our second most
popular FREE download)
has the ability to
graphically display the
output of the EXPLAIN
command. This is the
same query as the last
page.

Use SHOW
PROCESS_LIST();
to find a running query on
the server.
30
Use EXPLAIN FOR
CONNENCTION N to
see output.

Indexes let you go to
the exact record(s)
desired. They do
have some overhead
and will slow down
bulk inserts.
Without indexes the
entire table/file has
to be read, AKA a FULL
TABLE SCAN -- to be
avoided if your goal
is not to read the
entire table.
Compound indexes
let you use multiple
columns like YEAR-
MONTH-DATE and allow
YEAR-MONTH-DATE, YEAR-
MONTH, and YEAR
searches.
The Optimizer wants to use indexes as much as possible!
31

Your data is then
packaged up and sent
to your application
And that is the basics.
32

And that is how a query
Is supposed to be
processed
34

Common
problems
And how to avoid them
35

N+1 ProblemBad practice, may come from your ORM
Can be hard to catchLots of fussy little queries
Avoid by thinking in setsLet database do heavy lifting
36

You chain a set of queries
together to answer a
question - Look up
employees, then the ones
who live near you, and
then the ones who have a
parking permit so you
can get a ride to work
Each dive into the
database has a cost.
Databases can handle
one BIG request better
than a lot of little. So One
query for someone who
lives near you with
parking permit.
37

foreach (sale_emp in
sales_employees)
$pay = $pay * .20;
UPDATE employees
SET pay_rate = pay_rate * .20
WHERE department = ‘sales’;
Your boss asks you to give all the sales
staff a 20% pay increase
38

foreach (sale_emp in
sales_employees)
$pay = $pay * .20;
START TRANSACTION;
UPDATE employees
SET pay_rate = pay_rate * .20
WHERE department = ‘sales’;
COMMIT;
A transaction does all the work at the same
time, can be rolled back!!
39

SELECT City.Name as City,
Country.name as Country
FROM City
JOIN ON
(City.CountryCode =
Country.code);
FROM City
JOIN ON
(City.CountryCode =
Country.code)
LIMIT 5;
42

FROM City
JOIN ON
(City.CountryCode =
Country.code)
ORDER BY City.Population;
FROM City
JOIN ON
(City.CountryCode =
Country.code)
GROUP BY Country.Name;
43

You can not tell without looking at the
And using explain on a query. There is no
Way to tell if a query is good or bad
Just by looking.
44

AGE:
Years
Months
Days
HHMMSS
Century
Neolithic
45
Measurement:
Imperial, Metric
Significance
Rounding
single/pair/dozen/pallet
Gender:
Male
Female
NULL*
Null = No Data

NYC Commission on
Human Rights - 31 Gender
Identities:
1. Bi-gendered
2. Cross-dresser
3. Drag King
4. Drag Queen
5. Femme Queen
6. Female-to-Male
7. FTM
8. Gender Bender
9. Genderqueer
10. Male-to-Female
11. MTF
12. Non-Op
13. HIJRA
14. Pangender
15. Transexual/Transsexual
16. Trans Person
17. Woman
18. Man
19. Butch
20. Two-Spirit
46
Facebook UK has 71 Gender Identities!
And your DBA has to update what was once a
simple binary column (1 byte of data) and
then try to plan for the future!!!

Databases have many
useful functions for
maximums, minimums,
averages, standard
deviations, and more. No
Need to crunch numbers
in your app.
47

MySQL and other RDBMS
offer a JSON data type.
You can store a JSON
document without
schema. Great for JSON
formatted data and
those who just don’t care!
This works but is not as
fast as normalized data,
no rigor is applied to the
data, the standard is still
being worked on, and this
frustrates old timers,
dag-nabbit!!
48

MySQL offers the X Devapi
and acts as a docstore
from your language of
choice for CRUD (Create,
Replace, Update, Delete)
so you do not need
knowledge of SQL.
Works very well but you
may need a
DBA/Architect to later
normalize some of the
data with generated
columns for performance.
49

Data
The first step in getting great
performance out of your
database is proper data
normalization.
Indexes
Indexes greatly speed searches
but take overhead. General
rule: Index primary keys and
columns used on right of
WHERE clause
Heavy Lifting
Databases are great at
handling big chunks of data
and can perform transactions.
Try to maximize effects of each
query.
Disks
Do not go cheap on disks. SSDs
do pay for themselves and
move heavy uses to separate
disks/controllers
Slow Query Log
Turn on the slow query log and
pay attention. Note: Some
slow queries are there because
they run a long time and just
not slow
Sys Schema
The Sys Schema was designed
to let you peer into the heart of
your instances and answer
questions like which indexes are
not used, who is hogging I/O
50

SQL Antipatterns
Bill Karwin
SQL and Relational
Theory
CJ Date
51

THANKS!Any questions?
You can find me at @stoker & david.stokes@oracle.com
Slides can be found at http://slideshare.net/davidmstokes
52

Why Your Database Queries Stink -SeaGl.org November 11th, 2016

More Related Content

What's hot (20)

Viewers also liked (18)

Similar to Why Your Database Queries Stink -SeaGl.org November 11th, 2016 (20)

More from Dave Stokes (17)

Recently uploaded (20)

Why Your Database Queries Stink -SeaGl.org November 11th, 2016