SlideShare a Scribd company logo

Fun with SQL

E
eggyknap

The document focuses on SQL operations and best practices, emphasizing the importance of using the database for data management and SQL for efficient querying. It discusses different types of joins, subqueries, set operations, and advanced techniques like common table expressions and recursion. The key takeaway is that databases excel at handling data operations, while applications should manage business logic.

Technology
1 of 58
Downloaded 171 times
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
Fun With SQL Joshua Tolley End Point Corporation Why and Why Not Joins CROSS JOIN Fun With SQL INNER JOIN OUTER JOIN NATURAL JOIN Self Joins Other Useful Joshua Tolley Operations Subqueries End Point Corporation Set Operations Common Operations Advanced Operations Common Table October 5, 2009 Expressions Window Functions Real, Live Queries Something Simple Something Fun Key Points
Fun With SQL Joshua Tolley End Point Corporation Why and Why Not Joins CROSS JOIN INNER JOIN OUTER JOIN NATURAL JOIN ”The degree of normality in a database is inversely Self Joins proportional to that of its DBA.” - Anon, twitter Other Useful Operations Subqueries Set Operations Common Operations Advanced Operations Common Table Expressions Window Functions Real, Live Queries Something Simple Something Fun Key Points
Fun With SQL Why Not Do Stuff in SQL Joshua Tolley End Point Databases are harder to replicate, if you really need to Corporation scale out Why and Why Not Often, one complex SQL query is more efficient than Joins several simple ones CROSS JOIN Sometimes, indeed, it’s useful to reduce the load on the INNER JOIN OUTER JOIN database by moving logic into the application. Be NATURAL JOIN careful doing this Self Joins Other Useful c.f. Premature Optimization Operations Subqueries More complex queries are harder to write and debug Set Operations Common Operations True. But so is more complex programming. Advanced Operations More complex queries are harder for the next guy to Common Table Expressions maintain Window Functions Also, good DBAs are often more expensive than good Real, Live Queries Something Simple programmers Something Fun These are both true. But complex programming is also Key Points hard for the next guy to maintain Of all the reasons not to write fluent SQL, this is probably the most widely applicable
Fun With SQL Why do stuff in SQL? Joshua Tolley End Point Corporation Why and Why Not The database is more efficient than your application for Joins processing big chunks of data CROSS JOIN INNER JOIN ...especially if your code is in an interpreted language OUTER JOIN NATURAL JOIN The database is better tested than your application Self Joins Applications trying to do what SQL should be doing Other Useful Operations often get big and complex quickly Subqueries Set Operations ...and also buggy quickly Common Operations That’s what the database is there for Advanced Operations SQL is designed to express relations and conditions on Common Table Expressions Window Functions them. Your application’s language isn’t. Real, Live Queries A better understanding of SQL allows you to write Something Simple Something Fun queries that perform better Key Points
Fun With SQL Why do stuff in SQL? Joshua Tolley End Point Corporation Why and Why Not Joins CROSS JOIN INNER JOIN OUTER JOIN NATURAL JOIN Self Joins In short, the database exists to manage data, and your Other Useful Operations application exists to handle business logic. Write software Subqueries accordingly. Set Operations Common Operations Advanced Operations Common Table Expressions Window Functions Real, Live Queries Something Simple Something Fun Key Points
Fun With SQL Joshua Tolley End Point Corporation Why and Why Not Joins CROSS JOIN INNER JOIN OUTER JOIN NATURAL JOIN Self Joins So let’s get started... Other Useful Operations Subqueries Set Operations Common Operations Advanced Operations Common Table Expressions Window Functions Real, Live Queries Something Simple Something Fun Key Points
Fun With SQL Tables we’ll use Joshua Tolley End Point Corporation Why and Why Not Joins CROSS JOIN # SELECT * FROM a; # SELECT * FROM b; INNER JOIN OUTER JOIN NATURAL JOIN id | value id | value Self Joins ----+------- ----+------- Other Useful Operations 1 | a1 5 | b5 Subqueries Set Operations 2 | a2 4 | b4 Common Operations 3 | a3 3 | b3 Advanced Operations 4 | a4 6 | b6 Common Table Expressions (4 rows) (4 rows) Window Functions Real, Live Queries Something Simple Something Fun Key Points
Fun With SQL JOINs Joshua Tolley End Point Corporation Why and Why Not Joins CROSS JOIN INNER JOIN OUTER JOIN NATURAL JOIN If you want data from multiple tables, you probably Self Joins Other Useful want a join Operations ...but see also Subqueries, later on Subqueries Set Operations Common Operations There are several different kinds of joins Advanced Operations Common Table Expressions Window Functions Real, Live Queries Something Simple Something Fun Key Points
Fun With SQL JOINs Joshua Tolley End Point Corporation Why and Why Not Joins CROSS JOIN INNER JOIN <table1> [alias1] OUTER JOIN [ [ [NATURAL] [ [FULL | RIGHT | LEFT] [OUTER] | NATURAL JOIN Self Joins INNER] ] | CROSS ] JOIN Other Useful Operations <table2> [alias2] Subqueries Set Operations [USING (...) | Common Operations ON (<value1> <op> <value2> Advanced Operations [,<value3> <op> <value4>...] ) ] Common Table Expressions Window Functions Real, Live Queries Something Simple Something Fun Key Points
Fun With SQL CROSS JOIN Joshua Tolley End Point Corporation Why and Why Not Joins SELECT <... >FROM table1 JOIN table2 CROSS JOIN INNER JOIN With no explicit join type and no join qualifiers (an ON OUTER JOIN NATURAL JOIN clause, WHERE clause involving both relations, etc.) Self Joins this is a CROSS JOIN Other Useful Operations Equivalent to Subqueries Set Operations Common Operations SELECT <... >FROM table1, table2 Advanced SELECT <... >FROM table1 CROSS JOIN table2 Operations Common Table ”Cartesian product” of the two relations Expressions Window Functions Combines every row of table1 with every row of table2 Real, Live Queries Makes LOTS of rows, and can thus be very slow Something Simple Something Fun Key Points
Fun With SQL CROSS JOIN Joshua Tolley End Point Corporation # SELECT * FROM a, b; Why and Why Not id | value | id | value Joins CROSS JOIN ----+-------+----+------- INNER JOIN OUTER JOIN 1 | a1 | 5 | b5 NATURAL JOIN Self Joins 1 | a1 | 4 | b4 Other Useful <snip> Operations Subqueries 3 | a3 | 3 | b3 Set Operations Common Operations 3 | a3 | 6 | b6 Advanced Operations 4 | a4 | 5 | b5 Common Table Expressions 4 | a4 | 4 | b4 Window Functions 4 | a4 | 3 | b3 Real, Live Queries Something Simple 4 | a4 | 6 | b6 Something Fun (16 rows) Key Points
Fun With SQL INNER JOIN Joshua Tolley End Point Corporation Why and Why Not Joins CROSS JOIN INNER JOIN OUTER JOIN NATURAL JOIN SELECT <...>FROM table1 INNER JOIN table2 ON Self Joins Other Useful (table1.field = table2.field ...) Operations Subqueries Only returns rows satisfying the ON condition Set Operations Common Operations Equivalent to a CROSS JOIN with a WHERE clause Advanced Operations Common Table Expressions Window Functions Real, Live Queries Something Simple Something Fun Key Points
Fun With SQL INNER JOIN Joshua Tolley End Point Corporation Why and Why Not Joins CROSS JOIN INNER JOIN OUTER JOIN # SELECT * FROM a INNER JOIN b USING (id); NATURAL JOIN Self Joins id | value | value Other Useful ----+-------+------- Operations Subqueries 3 | a3 | b3 Set Operations Common Operations 4 | a4 | b4 Advanced (2 rows) Operations Common Table Expressions Window Functions Real, Live Queries Something Simple Something Fun Key Points
Fun With SQL OUTER JOIN Joshua Tolley End Point Corporation Why and Why Not Joins Return all rows from one or both relations CROSS JOIN INNER JOIN OUTER JOIN LEFT: Return all rows from the relation on the left NATURAL JOIN Self Joins RIGHT: Return all rows from the relation on the right Other Useful Operations FULL: Return all rows from both relations Subqueries Set Operations Returns nulls for values from one relation when it Common Operations Advanced contains to match with the other relation Operations Common Table The OUTER keyword is redundant Expressions Window Functions Requires ON or USING clause Real, Live Queries Something Simple Something Fun Key Points
Fun With SQL LEFT JOIN Joshua Tolley End Point Corporation Why and Why Not Joins CROSS JOIN # SELECT * FROM a LEFT JOIN b USING (id); INNER JOIN OUTER JOIN id | value | value NATURAL JOIN Self Joins ----+-------+------- Other Useful 1 | a1 | Operations Subqueries 2 | a2 | Set Operations Common Operations 3 | a3 | b3 Advanced Operations 4 | a4 | b4 Common Table Expressions (4 rows) Window Functions Real, Live Queries Something Simple Something Fun Key Points
Fun With SQL RIGHT JOIN Joshua Tolley End Point Corporation Why and Why Not Joins CROSS JOIN # SELECT * FROM a RIGHT JOIN b USING (id); INNER JOIN OUTER JOIN id | value | value NATURAL JOIN Self Joins ----+-------+------- Other Useful 3 | a3 | b3 Operations Subqueries 4 | a4 | b4 Set Operations Common Operations 5 | | b5 Advanced Operations 6 | | b6 Common Table Expressions (4 rows) Window Functions Real, Live Queries Something Simple Something Fun Key Points
Fun With SQL FULL JOIN Joshua Tolley End Point Corporation Why and Why Not Joins # select * from a full join b using (id); CROSS JOIN id | value | value INNER JOIN OUTER JOIN ----+-------+------- NATURAL JOIN Self Joins 1 | a1 | Other Useful Operations 2 | a2 | Subqueries Set Operations 3 | a3 | b3 Common Operations 4 | a4 | b4 Advanced Operations 5 | | b5 Common Table Expressions 6 | | b6 Window Functions (6 rows) Real, Live Queries Something Simple Something Fun Key Points
Fun With SQL Applications Joshua Tolley End Point Corporation Why and Why Not Find rows with no match in table b: Joins CROSS JOIN INNER JOIN OUTER JOIN # SELECT * FROM a LEFT JOIN b USING (id) NATURAL JOIN Self Joins WHERE b.value IS NULL; Other Useful id | value | value Operations Subqueries ----+-------+------- Set Operations Common Operations 1 | a1 | Advanced Operations 2 | a2 | Common Table Expressions (2 rows) Window Functions Real, Live Queries Something Simple Something Fun Key Points
Fun With SQL NATURAL JOIN Joshua Tolley End Point Corporation Why and Why Not Joins CROSS JOIN INNER JOIN OUTER JOIN NATURAL JOIN Self Joins Other Useful NATURAL is syntactic sugar to match all columns with Operations the same name Subqueries Set Operations Common Operations Advanced Operations Common Table Expressions Window Functions Real, Live Queries Something Simple Something Fun Key Points
Fun With SQL NATURAL JOIN Joshua Tolley End Point Corporation # SELECT * FROM a NATURAL FULL JOIN b; id | value Why and Why Not Joins ----+------- CROSS JOIN 1 | a1 INNER JOIN OUTER JOIN 2 | a2 NATURAL JOIN Self Joins 3 | a3 Other Useful Operations 3 | b3 Subqueries Set Operations 4 | a4 Common Operations 4 | b4 Advanced Operations 5 | b5 Common Table Expressions 6 | b6 Window Functions (8 rows) Real, Live Queries Something Simple Something Fun Key Points This looked for matches in both the id and value columns, so no rows matched. It returned all rows of both relations because it’s a FULL JOIN.
Fun With SQL Self Joins Joshua Tolley End Point Corporation Why and Why Not Joins CROSS JOIN INNER JOIN OUTER JOIN NATURAL JOIN Self Joins ”Self joins” are particularly counterintuitive Other Useful Operations Joins one table to itself Subqueries Set Operations It helps to give the table two different aliases Common Operations Advanced Operations Common Table Expressions Window Functions Real, Live Queries Something Simple Something Fun Key Points
Fun With SQL Self Joins Joshua Tolley Find all employees’ names, and each employee’s manager End Point Corporation SELECT Why and Why Not e.first || ’ ’ || e.last, Joins (SELECT CROSS JOIN INNER JOIN m.first || ’ ’ || m.last OUTER JOIN NATURAL JOIN FROM employee m Self Joins Other Useful WHERE m.id = e.manager); Operations Subqueries Set Operations Common Operations ... will generally be much faster rewritten as ... Advanced Operations Common Table SELECT Expressions Window Functions e.first || ’ ’ || e.last, Real, Live Queries m.first || ’ ’ || m.last Something Simple Something Fun FROM Key Points employee e JOIN employee m ON (e.manager = m.id)
Fun With SQL Joshua Tolley End Point Corporation Why and Why Not Joins CROSS JOIN INNER JOIN OUTER JOIN NATURAL JOIN Self Joins More useful operations... Other Useful Operations Subqueries Set Operations Common Operations Advanced Operations Common Table Expressions Window Functions Real, Live Queries Something Simple Something Fun Key Points
Fun With SQL Subqueries Joshua Tolley End Point Corporation Why and Why Not Joins Embeds one query within another CROSS JOIN INNER JOIN OUTER JOIN Examples (some bad, some good) NATURAL JOIN Self Joins SELECT id FROM table WHERE field = (SELECT Other Useful MAX(field) FROM table) Operations SELECT id, (SELECT COUNT(*) FROM table2 Subqueries Set Operations WHERE id = table1.id) FROM table1 Common Operations SELECT a, b FROM (SELECT a, COUNT(*) AS c Advanced Operations FROM table1) t1 JOIN (SELECT b, COUNT(*) AS c Common Table Expressions FROM table2) t2 on (t1.c = t2.c) Window Functions You can join subqueries just like you’d join tables Real, Live Queries Something Simple Something Fun Key Points
Fun With SQL Set Operations Joshua Tolley End Point Corporation INTERSECT Why and Why Not Returns the intersection of two sets Joins Doesn’t exist in MySQL CROSS JOIN INNER JOIN SELECT (SELECT a, b FROM table1) INTERSECT OUTER JOIN NATURAL JOIN (SELECT c, d FROM table2) Self Joins UNION Other Useful Operations Appends one set of rows to another set with matching Subqueries Set Operations column types Common Operations SELECT a FROM table1 UNION SELECT b FROM Advanced Operations table2 Common Table Expressions EXCEPT Window Functions Real, Live Queries Returns rows in one SELECT that aren’t in another Something Simple SELECT Something Fun SELECT a FROM table1 EXCEPT SELECT b FROM Key Points table2
Fun With SQL Common Operations Joshua Tolley End Point Corporation Why and Why Not Joins CROSS JOIN COALESCE(a, b) INNER JOIN OUTER JOIN If a is null, return b, else return a NATURAL JOIN Self Joins SELECT COALESCE(first, ’<NULL>’) FROM table Other Useful Oracle calls this NVL() Operations Subqueries CASE...WHEN Set Operations Common Operations Conditional operation Advanced SELECT CASE WHEN langused IN (’Lisp’, ’OCaml’, Operations Common Table ’Haskell’) THEN ’Functional’ ELSE ’Imperative’ AS Expressions Window Functions langtype FROM software Real, Live Queries Something Simple Something Fun Key Points
Fun With SQL Series Generation Joshua Tolley End Point Corporation generate series() in PostgreSQL; might be something else in other databases Why and Why Not Joins Returns a series of numbers CROSS JOIN INNER JOIN Can be used like a for loop (example given later) OUTER JOIN NATURAL JOIN Self Joins # SELECT * FROM generate_series(1, 5); Other Useful Operations generate_series Subqueries Set Operations ----------------- Common Operations Advanced 1 Operations 2 Common Table Expressions Window Functions 3 Real, Live Queries 4 Something Simple Something Fun 5 Key Points (5 rows)
Fun With SQL Common Table Expressions Joshua Tolley End Point Corporation Why and Why Not Joins CROSS JOIN Abbreviated CTEs INNER JOIN OUTER JOIN Fairly advanced; not available in all databases NATURAL JOIN Self Joins Not in PostgreSQL before v. 8.4, or any version of Other Useful Operations MySQL Subqueries Set Operations It’s just like defining a one-time view for your query Common Operations One major benefit: CTEs allow recursion Advanced Operations Recursing with CTEs is much more efficent than Common Table Expressions processing recursive data in your application Window Functions Real, Live Queries Something Simple Something Fun Key Points
Fun With SQL A Simple CTE Example Joshua Tolley End Point Corporation Why and Why Not # SELECT * FROM GENERATE_SERIES(1,3) Joins CROSS JOIN CROSS JOIN INNER JOIN (SELECT * FROM GENERATE_SERIES(8,9)) AS f; OUTER JOIN NATURAL JOIN generate_series | generate_series Self Joins -----------------+----------------- Other Useful Operations 1 | 8 Subqueries Set Operations 1 | 9 Common Operations 2 | 8 Advanced Operations 2 | 9 Common Table Expressions 3 | 8 Window Functions Real, Live Queries 3 | 9 Something Simple Something Fun (6 rows) Key Points
Fun With SQL A Simple CTE Example Joshua Tolley End Point Corporation # WITH t AS ( Why and Why Not SELECT * FROM GENERATE_SERIES(8,9) Joins ) CROSS JOIN INNER JOIN SELECT * FROM GENERATE_SERIES(1,3) OUTER JOIN NATURAL JOIN CROSS JOIN t; Self Joins Other Useful generate_series | generate_series Operations -----------------+----------------- Subqueries Set Operations 1 | 8 Common Operations Advanced 1 | 9 Operations Common Table 2 | 8 Expressions Window Functions 2 | 9 Real, Live Queries 3 | 8 Something Simple Something Fun 3 | 9 Key Points (6 rows)
Fun With SQL Joshua Tolley End Point Corporation Why and Why Not Joins CROSS JOIN INNER JOIN OUTER JOIN NATURAL JOIN That last example was a bit cheesy, but the technique can Self Joins Other Useful be useful for complex queries in several parts Operations Subqueries Set Operations Common Operations Advanced Operations Common Table Expressions Window Functions Real, Live Queries Something Simple Something Fun Key Points
Fun With SQL Recursion Joshua Tolley End Point Corporation Start with this: Why and Why Not Joins # SELECT * FROM employee; CROSS JOIN INNER JOIN first | last | id | manager OUTER JOIN NATURAL JOIN --------+----------+----+--------- Self Joins john | doe | 1 | Other Useful Operations fred | rogers | 2 | 1 Subqueries Set Operations speedy | gonzales | 3 | 1 Common Operations carly | fiorina | 4 | 1 Advanced Operations hans | reiser | 5 | 2 Common Table Expressions johnny | carson | 6 | 5 Window Functions Real, Live Queries martha | stewart | 7 | 3 Something Simple (7 rows) Something Fun Key Points
Fun With SQL Recursion Joshua Tolley Recursive CTE to retrieve management hierarchy: End Point Corporation # WITH RECURSIVE t (id, managernames) AS ( Why and Why Not SELECT e.id, first || ’ ’ || last Joins AS managernames CROSS JOIN INNER JOIN FROM employee e WHERE manager IS NULL OUTER JOIN NATURAL JOIN UNION ALL Self Joins Other Useful SELECT e.id, Operations Subqueries first || ’ ’ || last || ’, ’ || managernames Set Operations Common Operations AS managernames Advanced FROM employee e Operations Common Table JOIN t ON (e.manager = t.id) Expressions Window Functions WHERE manager IS NOT NULL Real, Live Queries ) Something Simple Something Fun SELECT e.id, first || ’ ’ || last AS name, Key Points managernames FROM employee e JOIN t ON (e.id = t.id);
Fun With SQL Recursion Joshua Tolley End Point Corporation ...and get this... Why and Why Not id | name | managernames Joins ----+-----------------+-----------------------------CROSS JOIN INNER JOIN 1 | john doe | john doe OUTER JOIN NATURAL JOIN 2 | fred rogers | fred rogers, john doe Self Joins 3 | speedy gonzales | speedy gonzales, john doe Other Useful Operations 4 | carly fiorina | carly fiorina, john doe Subqueries Set Operations 5 | hans reiser | hans reiser, fred rogers, Common Operations Advanced | | john doe Operations 6 | johnny carson | johnny carson, hans reiser, Common Table Expressions Window Functions | | fred rogers, john doe Real, Live Queries 7 | martha stewart | martha stewart, speedy Something Simple Something Fun | | gonzales, john doe Key Points (7 rows)
Fun With SQL Fractals in SQL Joshua Tolley End Point Corporation WITH RECURSIVE x(i) AS (VALUES(0) UNION ALL SELECT i + 1 FROM x WHERE i < 101), Z(Ix, Iy, Cx, Cy, X, Y, I) Why and Why Not AS ( SELECT Ix, Iy, X::float, Y::float, X::float, Y::float, 0 Joins FROM (SELECT -2.2 + 0.031 * i, i FROM x) AS xgen(x,ix) CROSS JOIN CROSS JOIN INNER JOIN (SELECT -1.5 + 0.031 * i, i FROM x) AS ygen(y,iy) OUTER JOIN UNION ALL NATURAL JOIN Self Joins SELECT Ix, Iy, Cx, Cy, X * X - Y * Y + Cx AS X, Other Useful Y * X * 2 + Cy, I + 1 Operations FROM Z Subqueries WHERE X * X + Y * Y < 16.0 AND I < 27), Set Operations Zt (Ix, Iy, I) AS ( Common Operations SELECT Ix, Iy, MAX(I) AS I FROM Z GROUP BY Iy, Ix Advanced ORDER BY Iy, Ix Operations ) Common Table SELECT array_to_string( Expressions array_agg( Window Functions SUBSTRING(’ .,,,-----++++%%%%@@@@#### ’, Real, Live Queries GREATEST(I,1), 1) Something Simple ),’’ Something Fun ) FROM Zt GROUP BY Iy ORDER BY Iy; Key Points (yes, this query is SQL-spec compliant)
Fun With SQL Joshua Tolley End Point Corporation Why and Why Not Joins CROSS JOIN INNER JOIN OUTER JOIN NATURAL JOIN Self Joins Other Useful Operations Subqueries Set Operations Common Operations Advanced Operations Common Table Expressions Window Functions Real, Live Queries Something Simple Something Fun Key Points
Fun With SQL Window Functions Joshua Tolley End Point Corporation Why and Why Not Joins CROSS JOIN INNER JOIN OUTER JOIN Like CTEs, these are quite advanced NATURAL JOIN Self Joins Also unavailable in MySQL, and PostgreSQL before 8.4 Other Useful Operations Allow ranking, moving averages Subqueries Set Operations Like a set-returning aggregate function. Window Common Operations Advanced functions return results for each row based on a Operations ”window” of related rows Common Table Expressions Window Functions Real, Live Queries Something Simple Something Fun Key Points
Fun With SQL Window Functions Joshua Tolley End Point Corporation If our employee table had department and salary information... Why and Why Not Joins # SELECT first, last, salary, department CROSS JOIN INNER JOIN FROM employee; OUTER JOIN NATURAL JOIN first | last | salary | department Self Joins Other Useful --------+----------+--------+---------------- Operations fred | rogers | 97000 | sales Subqueries Set Operations carly | fiorina | 95000 | sales Common Operations Advanced johnny | carson | 89000 | sales Operations speedy | gonzales | 96000 | development Common Table Expressions Window Functions hans | reiser | 93000 | development Real, Live Queries martha | stewart | 90000 | development Something Simple Something Fun john | doe | 99000 | administration Key Points (7 rows)
Fun With SQL Window Functions Example Joshua Tolley End Point Corporation Why and Why Not Joins Rank employees in each department by salary CROSS JOIN INNER JOIN OUTER JOIN NATURAL JOIN SELECT first, last, salary, department, Self Joins RANK() OVER ( Other Useful Operations PARTITION BY department Subqueries Set Operations ORDER BY salary DESC Common Operations ) Advanced Operations FROM employee Common Table Expressions Window Functions Real, Live Queries Something Simple Something Fun Key Points
Fun With SQL Window Functions Example Joshua Tolley End Point Corporation Why and Why Not ... and get this: Joins CROSS JOIN INNER JOIN first | last | salary | department | rank OUTER JOIN --------+----------+--------+----------------+------ NATURAL JOIN Self Joins john | doe | 99000 | administration | 1 Other Useful speedy | gonzales | 96000 | development | 1 Operations hans | reiser | 93000 | development | 2 Subqueries Set Operations martha | stewart | 90000 | development | 3 Common Operations fred | rogers | 97000 | sales | 1 Advanced carly | fiorina | 95000 | sales | 2 Operations Common Table johnny | carson | 89000 | sales | 3 Expressions Window Functions (7 rows) Real, Live Queries Something Simple Something Fun Key Points
Fun With SQL Joshua Tolley End Point Corporation Why and Why Not Joins CROSS JOIN INNER JOIN OUTER JOIN NATURAL JOIN Self Joins Real, live queries Other Useful Operations Subqueries Set Operations Common Operations Advanced Operations Common Table Expressions Window Functions Real, Live Queries Something Simple Something Fun Key Points
Fun With SQL Something Simple Joshua Tolley End Point Corporation Why and Why Not The slow version: Joins CROSS JOIN SELECT DISTINCT(sync) FROM bucardo.bucardo_rate INNER JOIN OUTER JOIN ORDER BY 1 NATURAL JOIN Self Joins Other Useful Operations The fast version: Subqueries Set Operations Common Operations SELECT name FROM sync WHERE EXISTS ( Advanced Operations SELECT 1 FROM bucardo_rate Common Table Expressions WHERE sync = name LIMIT 1) Window Functions Real, Live Queries ORDER BY 1 Something Simple Something Fun Key Points
Fun With SQL Something Simple Joshua Tolley End Point Corporation Why and Why Not Joins CROSS JOIN INNER JOIN OUTER JOIN The bucardo rate table is huge, with few distinct values NATURAL JOIN Self Joins finding ”DISTINCT sync” requires a long table scan Other Useful Operations The sync table contains a list of all possible values in Subqueries Set Operations the bucardo rate.sync column Common Operations Advanced So instead of a big table scan, we scan the small table, Operations and filter out values can’t find in bucardo rate Common Table Expressions Window Functions Real, Live Queries Something Simple Something Fun Key Points
Fun With SQL Joshua Tolley End Point Corporation Why and Why Not Joins CROSS JOIN INNER JOIN OUTER JOIN NATURAL JOIN Self Joins Something Fun Other Useful Operations Subqueries Set Operations Common Operations Advanced Operations Common Table Expressions Window Functions Real, Live Queries Something Simple Something Fun Key Points
Fun With SQL Joshua Tolley SELECT End Point id, idname, Corporation COALESCE(ROUND(AVG(synctime)::NUMERIC, 1), 0) AS avgtime, COALESCE(SUM(total), 0) AS count FROM ( Why and Why Not SELECT slavecommit, EXTRACT(EPOCH FROM slavecommit - mastercommit) AS synctime, Joins total CROSS JOIN FROM bucardo.bucardo_rate INNER JOIN WHERE sync = ’RO_everything’ AND OUTER JOIN mastercommit > (NOW() - (15 + 1) * INTERVAL ’1 HOUR’) NATURAL JOIN ) i Self Joins RIGHT JOIN ( Other Useful SELECT id, idname, Operations TO_TIMESTAMP(start - start::INTEGER % 3600) AS start, Subqueries TO_TIMESTAMP(stop - stop::INTEGER % 3600) AS stop Set Operations FROM ( Common Operations SELECT id, TO_CHAR(NOW() - id * INTERVAL ’1 HOUR’, Advanced ’Dy Mon DD HH:MI AM’) AS idname, Operations EXTRACT(EPOCH FROM NOW() - id * INTERVAL ’1 HOUR’) AS start, Common Table EXTRACT(EPOCH FROM NOW() - (id - 1) * INTERVAL ’1 HOUR’) AS stop Expressions FROM ( Window Functions SELECT GENERATE_SERIES(1, 15) AS id Real, Live Queries ) f ) g Something Simple Something Fun ) h ON (slavecommit BETWEEN start AND stop) GROUP BY id, idname Key Points ORDER BY id DESC;
Fun With SQL Something Fun Joshua Tolley End Point Corporation Why and Why Not Joins CROSS JOIN INNER JOIN OUTER JOIN The table contains replication data NATURAL JOIN Self Joins Time of commit on master Other Useful Time of commit on slave Operations Subqueries Number of rows replicated Set Operations Common Operations The user wants a graph of replication speed over time, Advanced given a user-determined range of time Operations Common Table Expressions Window Functions Real, Live Queries Something Simple Something Fun Key Points
Fun With SQL Something Fun Joshua Tolley End Point We want to average replication times over a series of Corporation buckets. The first part of our query creates those buckets, Why and Why Not based on generate series(). Here we create buckets for Joins 15 hours CROSS JOIN INNER JOIN OUTER JOIN SELECT NATURAL JOIN Self Joins id, Other Useful Operations TO_CHAR(NOW() - id * INTERVAL ’1 HOUR’, Subqueries ’Dy Mon DD HH:MI AM’) AS idname, Set Operations Common Operations EXTRACT(EPOCH FROM NOW() - id * Advanced Operations INTERVAL ’1 HOUR’) AS start, Common Table Expressions EXTRACT(EPOCH FROM NOW() - (id - 1) * Window Functions INTERVAL ’1 HOUR’) AS stop Real, Live Queries Something Simple FROM ( Something Fun SELECT GENERATE_SERIES(1, 15) AS id Key Points ) f
Fun With SQL Something Fun Joshua Tolley End Point Corporation Why and Why Not Joins CROSS JOIN This gives us: INNER JOIN OUTER JOIN NATURAL JOIN id | idname | start | stop Self Joins ----+---------------------+------------------+------------------ Other Useful 1 | Sat Mar 14 10:23 PM | 1237091036.95657 | 1237094636.95657 Operations 2 | Sat Mar 14 09:23 PM | 1237087436.95657 | 1237091036.95657 Subqueries Set Operations 3 | Sat Mar 14 08:23 PM | 1237083836.95657 | 1237087436.95657 Common Operations 4 | Sat Mar 14 07:23 PM | 1237080236.95657 | 1237083836.95657 Advanced ... Operations Common Table Expressions Window Functions Real, Live Queries Something Simple Something Fun Key Points
Fun With SQL Something Fun Joshua Tolley End Point Corporation Why and Why Not Make the buckets end on nice time boundaries: Joins CROSS JOIN INNER JOIN SELECT id, idname, OUTER JOIN NATURAL JOIN TO_TIMESTAMP(start - start::INTEGER % 3600) Self Joins Other Useful AS start, Operations TO_TIMESTAMP(stop - stop::INTEGER % 3600) Subqueries Set Operations AS stop Common Operations Advanced FROM ( Operations -- The bucket query, shown earlier, goes here Common Table Expressions Window Functions ) g Real, Live Queries Something Simple Something Fun Key Points
Fun With SQL Something Fun Joshua Tolley End Point Corporation Why and Why Not Joins CROSS JOIN INNER JOIN OUTER JOIN That gives us this: NATURAL JOIN Self Joins id | idname | start | stop ----+---------------------+-------------------------------+------------------------------- Other Useful 1 | Sat Mar 14 10:23 PM | 2009-03-14 21:59:59.956568-06 | 2009-03-14 22:59:59.956568-06 Operations 2 | Sat Mar 14 09:23 PM | 2009-03-14 20:59:59.956568-06 | 2009-03-14 21:59:59.956568-06 Subqueries 3 | Sat Mar 14 08:23 PM | 2009-03-14 19:59:59.956568-06 | 2009-03-14 20:59:59.956568-06 Set Operations 4 | Sat Mar 14 07:23 PM | 2009-03-14 18:59:59.956568-06 | 2009-03-14 19:59:59.956568-06 Common Operations Advanced Operations Common Table Expressions Window Functions Real, Live Queries Something Simple Something Fun Key Points
Fun With SQL Something Fun Joshua Tolley End Point Corporation In an different subquery, select everything from the table of Why and Why Not the right time period and right sync. Call this the ”stats” Joins query: CROSS JOIN INNER JOIN OUTER JOIN SELECT NATURAL JOIN Self Joins slavecommit, Other Useful EXTRACT(EPOCH FROM slavecommit - mastercommit) Operations Subqueries AS synctime, Set Operations Common Operations total Advanced FROM bucardo.bucardo_rate Operations Common Table WHERE Expressions Window Functions sync = ’RO_everything’ AND Real, Live Queries Something Simple mastercommit > (NOW() - (15 + 1) * Something Fun INTERVAL ’1 HOUR’) Key Points
Fun With SQL Something Fun Joshua Tolley End Point Corporation Why and Why Not ...which gives us this: Joins CROSS JOIN INNER JOIN slavecommit | synctime | total OUTER JOIN -------------------------------+------------------+-------- NATURAL JOIN Self Joins 2009-03-14 07:32:00.103759-06 | 5.65614098310471 | 1 Other Useful 2009-03-14 07:32:04.31508-06 | 5.25827997922897 | 3 Operations 2009-03-14 07:32:04.31508-06 | 5.25827997922897 | 5 Subqueries Set Operations 2009-03-14 07:32:08.700184-06 | 7.71899098157883 | 1 Common Operations 2009-03-14 07:32:08.700184-06 | 8.22490698099136 | 1 Advanced 2009-03-14 07:32:12.675518-06 | 7.85176599025726 | 6 Operations Common Table 2009-03-14 07:32:12.675518-06 | 7.15798497200012 | 6 Expressions Window Functions ... Real, Live Queries Something Simple Something Fun Key Points
Fun With SQL Something Fun Joshua Tolley End Point Corporation Now, join the two queries: Why and Why Not SELECT Joins CROSS JOIN id, idname, INNER JOIN OUTER JOIN COALESCE(ROUND(AVG(synctime)::NUMERIC, 1), 0) NATURAL JOIN Self Joins AS avgtime, Other Useful COALESCE(SUM(total), 0) AS count Operations Subqueries FROM ( Set Operations Common Operations <STATS QUERY> Advanced ) RIGHT JOIN ( Operations Common Table <CALENDAR QUERY> Expressions Window Functions ) ON (slavecommit BETWEEN start AND stop) Real, Live Queries Something Simple GROUP BY id, idname Something Fun ORDER BY id DESC; Key Points
Fun With SQL Something Fun Joshua Tolley End Point ...and get this: Corporation Why and Why Not id | idname | avgtime | count Joins ----+---------------------+---------+------- CROSS JOIN INNER JOIN 15 | Sat Mar 14 08:35 AM | 7.9 | 14219 OUTER JOIN NATURAL JOIN 14 | Sat Mar 14 09:35 AM | 6.9 | 16444 Self Joins 13 | Sat Mar 14 10:35 AM | 6.5 | 62100 Other Useful Operations 12 | Sat Mar 14 11:35 AM | 6.2 | 47349 Subqueries Set Operations 11 | Sat Mar 14 12:35 PM | 0 | 0 Common Operations Advanced 10 | Sat Mar 14 01:35 PM | 4.6 | 21348 Operations Common Table Expressions Window Functions This is the average replication time and total replicated rows Real, Live Queries Something Simple per hour. Note that this correctly returns zeroes when no Something Fun rows are replicated, and still returns a value for that time Key Points slot. This prevents some amount of application-side processing.
Fun With SQL Something Fun Joshua Tolley End Point Corporation Why and Why Not Joins CROSS JOIN INNER JOIN OUTER JOIN NATURAL JOIN Self Joins Other Useful Operations That query again: Subqueries Set Operations Common Operations Advanced Operations Common Table Expressions Window Functions Real, Live Queries Something Simple Something Fun Key Points
Fun With SQL Joshua Tolley SELECT End Point id, idname, Corporation COALESCE(ROUND(AVG(synctime)::NUMERIC, 1), 0) AS avgtime, COALESCE(SUM(total), 0) AS count FROM ( Why and Why Not SELECT slavecommit, EXTRACT(EPOCH FROM slavecommit - mastercommit) AS synctime, Joins total CROSS JOIN FROM bucardo.bucardo_rate INNER JOIN WHERE sync = ’RO_everything’ AND OUTER JOIN mastercommit > (NOW() - (15 + 1) * INTERVAL ’1 HOUR’) NATURAL JOIN ) i Self Joins RIGHT JOIN ( Other Useful SELECT id, idname, Operations TO_TIMESTAMP(start - start::INTEGER % 3600) AS start, Subqueries TO_TIMESTAMP(stop - stop::INTEGER % 3600) AS stop Set Operations FROM ( Common Operations SELECT id, TO_CHAR(NOW() - id * INTERVAL ’1 HOUR’, Advanced ’Dy Mon DD HH:MI AM’) AS idname, Operations EXTRACT(EPOCH FROM NOW() - id * INTERVAL ’1 HOUR’) AS start, Common Table EXTRACT(EPOCH FROM NOW() - (id - 1) * INTERVAL ’1 HOUR’) AS stop Expressions FROM ( Window Functions SELECT GENERATE_SERIES(1, 15) AS id Real, Live Queries ) f ) g Something Simple Something Fun ) h ON (slavecommit BETWEEN start AND stop) GROUP BY id, idname Key Points ORDER BY id DESC;
Fun With SQL Key Points Joshua Tolley End Point Corporation Why and Why Not Joins CROSS JOIN INNER JOIN OUTER JOIN NATURAL JOIN Understand join types, and use them Self Joins Other Useful Know what functions and set operations your database Operations Subqueries provides Set Operations Common Operations Build large queries piece by piece Advanced Operations Common Table Expressions Window Functions Real, Live Queries Something Simple Something Fun Key Points
Fun With SQL Joshua Tolley End Point Corporation Why and Why Not Joins CROSS JOIN INNER JOIN OUTER JOIN NATURAL JOIN Self Joins Questions? Other Useful Operations Subqueries Set Operations Common Operations Advanced Operations Common Table Expressions Window Functions Real, Live Queries Something Simple Something Fun Key Points

More Related Content

PPTX
SQLite - Overview
Emanuele Bartolesi
 
PPTX
Basic sql Commands
MUHAMMED MASHAHIL PUKKUNNUMMAL
 
PDF
Oracle db performance tuning
Simon Huang
 
PPT
Lecture2 oracle ppt
Hitesh Kumar Markam
 
PPTX
OLAP v/s OLTP
ahsan irfan
 
PPT
PL/SQL Introduction and Concepts
Bharat Kalia
 
PPT
Sql ppt
Anuja Lad
 
PDF
Chapter 4 Structured Query Language
Eddyzulham Mahluzydde
 
SQLite - Overview
Emanuele Bartolesi
 
Basic sql Commands
MUHAMMED MASHAHIL PUKKUNNUMMAL
 
Oracle db performance tuning
Simon Huang
 
Lecture2 oracle ppt
Hitesh Kumar Markam
 
OLAP v/s OLTP
ahsan irfan
 
PL/SQL Introduction and Concepts
Bharat Kalia
 
Sql ppt
Anuja Lad
 
Chapter 4 Structured Query Language
Eddyzulham Mahluzydde
 

What's hot (20)

PPT
SQL
Harshad Umredkar
 
PPTX
introdution to SQL and SQL functions
farwa waqar
 
PPTX
Stored procedure
baabtra.com - No. 1 supplier of quality freshers
 
ODP
Ms sql-server
Md.Mojibul Hoque
 
PPTX
Explain the explain_plan
Maria Colgan
 
PPT
SQL Tutorial - Basic Commands
1keydata
 
PDF
Oracle SQL Basics
Dhananjay Goel
 
PPT
Sq lite database
AYESHA JAVED
 
PPTX
sql function(ppt)
Ankit Dubey
 
PPT
Introduction to sql
VARSHAKUMARI49
 
PPTX
introduction to NOSQL Database
nehabsairam
 
DOC
SQL
Shunya Ram
 
PPTX
Sql commands
Pooja Dixit
 
PPTX
SQL Functions
ammarbrohi
 
PPTX
Sql subquery
Raveena Thakur
 
PPTX
Snowflake Data Loading.pptx
Parag860410
 
PPT
Constraints In Sql
Anurag
 
ODP
Partitioning
Reema Gajjar
 
PPTX
5. stored procedure and functions
Amrit Kaur
 
PDF
Sql tutorial
Rumman Ansari
 
SQL
Harshad Umredkar
 
introdution to SQL and SQL functions
farwa waqar
 
Stored procedure
baabtra.com - No. 1 supplier of quality freshers
 
Ms sql-server
Md.Mojibul Hoque
 
Explain the explain_plan
Maria Colgan
 
SQL Tutorial - Basic Commands
1keydata
 
Oracle SQL Basics
Dhananjay Goel
 
Sq lite database
AYESHA JAVED
 
sql function(ppt)
Ankit Dubey
 
Introduction to sql
VARSHAKUMARI49
 
introduction to NOSQL Database
nehabsairam
 
SQL
Shunya Ram
 
Sql commands
Pooja Dixit
 
SQL Functions
ammarbrohi
 
Sql subquery
Raveena Thakur
 
Snowflake Data Loading.pptx
Parag860410
 
Constraints In Sql
Anurag
 
Partitioning
Reema Gajjar
 
5. stored procedure and functions
Amrit Kaur
 
Sql tutorial
Rumman Ansari
 
Ad

Viewers also liked (7)

PDF
Various entity relation with dynamics CRM
Aegis Softtech
 
PPTX
types of SQL Joins
vikram rajpurohit
 
PDF
Schwachstellen In Sap Web Anwendungen (OWASP Germany 2009)
sebastianschinzel
 
PPT
Sql joins
Berkeley
 
PPT
Types Of Join In Sql Server - Join With Example In Sql Server
programmings guru
 
PDF
SQL Joins and Query Optimization
Brian Gallagher
 
PDF
10 SQL Tricks that You Didn't Think Were Possible
Lukas Eder
 
Various entity relation with dynamics CRM
Aegis Softtech
 
types of SQL Joins
vikram rajpurohit
 
Schwachstellen In Sap Web Anwendungen (OWASP Germany 2009)
sebastianschinzel
 
Sql joins
Berkeley
 
Types Of Join In Sql Server - Join With Example In Sql Server
programmings guru
 
SQL Joins and Query Optimization
Brian Gallagher
 
10 SQL Tricks that You Didn't Think Were Possible
Lukas Eder
 
Ad

Similar to Fun with SQL (20)

PDF
Funwithsqlnopause 091010085359-phpapp01
sagaroceanic11
 
PDF
Optimize sql server queries with these advanced tuning techniques tech repu
Kaing Menglieng
 
PPTX
Databases for Storage Engineers
Thomas Kejser
 
PDF
Sql basics joi ns and common commands (1)
johnnygoodman
 
PPTX
SQL Server 2008 Overview
David Chou
 
PDF
C net code tips to speed up sql query
Kaing Menglieng
 
PDF
10 sql tips to speed up your database cats whocode.com
Kaing Menglieng
 
PDF
Improving Performance with Better Indexes
MYXPLAIN
 
PDF
Interaction
Saikou Marong
 
PPTX
Performance By Design
Guy Harrison
 
PDF
15 ways to optimize your sql queries hungred dot com
Kaing Menglieng
 
PDF
Sql Commands
Sachin MK
 
PDF
Sql server common interview questions and answers page 5
Kaing Menglieng
 
PDF
MySQL Optimizer Overview
MYXPLAIN
 
PPTX
Mis04
Lee Gomez
 
PPT
Drill / SQL / Optiq
Julian Hyde
 
PDF
NoSQL and SQL - blending the best of both worlds
Andrew Morgan
 
PPTX
SQL Server Development Tools & Processes Using Visual Studio 2010
Ayman El-Hattab
 
PPTX
SQL_Part1
Rick Perry
 
PPTX
TechDays Tunisia - Visual Studio & SQL Server, Better Together - Ayman El-Hattab
Ayman El-Hattab
 
Funwithsqlnopause 091010085359-phpapp01
sagaroceanic11
 
Optimize sql server queries with these advanced tuning techniques tech repu
Kaing Menglieng
 
Databases for Storage Engineers
Thomas Kejser
 
Sql basics joi ns and common commands (1)
johnnygoodman
 
SQL Server 2008 Overview
David Chou
 
C net code tips to speed up sql query
Kaing Menglieng
 
10 sql tips to speed up your database cats whocode.com
Kaing Menglieng
 
Improving Performance with Better Indexes
MYXPLAIN
 
Interaction
Saikou Marong
 
Performance By Design
Guy Harrison
 
15 ways to optimize your sql queries hungred dot com
Kaing Menglieng
 
Sql Commands
Sachin MK
 
Sql server common interview questions and answers page 5
Kaing Menglieng
 
MySQL Optimizer Overview
MYXPLAIN
 
Mis04
Lee Gomez
 
Drill / SQL / Optiq
Julian Hyde
 
NoSQL and SQL - blending the best of both worlds
Andrew Morgan
 
SQL Server Development Tools & Processes Using Visual Studio 2010
Ayman El-Hattab
 
SQL_Part1
Rick Perry
 
TechDays Tunisia - Visual Studio & SQL Server, Better Together - Ayman El-Hattab
Ayman El-Hattab
 

Recently uploaded (20)

PPTX
Cloud computing and distributed systems.
kkkkkhan
 
PDF
Dropbox Q2 2025 Financial Results & Investor Presentation
Dropbox
 
PDF
MIND Revenue Release Quarter 2 2025 Press Release
MIND CTI
 
PDF
Per capita expenditure prediction using model stacking based on satellite ima...
IAESIJAI
 
PDF
Profit Center Accounting in SAP S/4HANA, S4F28 Col11
Libreria ERP
 
PDF
KodekX | Application Modernization Development
KodekX
 
PPTX
KOM of Painting work and Equipment Insulation REV00 update 25-dec.pptx
muhammadmuneebnaeem7
 
PDF
Mobile App Security Testing_ A Comprehensive Guide.pdf
flufftailshop
 
PPT
“AI and Expert System Decision Support & Business Intelligence Systems”
ZubinRadhakrishnan
 
PDF
NewMind AI Weekly Chronicles - August'25 Week I
NewMind AI
 
PDF
Spectral efficient network and resource selection model in 5G networks
IAESIJAI
 
PDF
Blue Purple Modern Animated Computer Science Presentation.pdf.pdf
Akar16
 
PDF
TokAI - TikTok AI Agent : The First AI Application That Analyzes 10,000+ Vira...
SOFTTECHHUB
 
PDF
Encapsulation_ Review paper, used for researhc scholars
gurumoop
 
PDF
Empathic Computing: Creating Shared Understanding
Mark Billinghurst
 
PPT
Teaching material agriculture food technology
LiaRayya
 
DOCX
The AUB Centre for AI in Media Proposal.docx
GAONE9
 
PDF
Machine learning based COVID-19 study performance prediction
IAESIJAI
 
PPTX
Spectroscopy.pptx food analysis technology
nawahassan45
 
PDF
Electronic commerce courselecture one. Pdf
OmerMohamed64
 
Cloud computing and distributed systems.
kkkkkhan
 
Dropbox Q2 2025 Financial Results & Investor Presentation
Dropbox
 
MIND Revenue Release Quarter 2 2025 Press Release
MIND CTI
 
Per capita expenditure prediction using model stacking based on satellite ima...
IAESIJAI
 
Profit Center Accounting in SAP S/4HANA, S4F28 Col11
Libreria ERP
 
KodekX | Application Modernization Development
KodekX
 
KOM of Painting work and Equipment Insulation REV00 update 25-dec.pptx
muhammadmuneebnaeem7
 
Mobile App Security Testing_ A Comprehensive Guide.pdf
flufftailshop
 
“AI and Expert System Decision Support & Business Intelligence Systems”
ZubinRadhakrishnan
 
NewMind AI Weekly Chronicles - August'25 Week I
NewMind AI
 
Spectral efficient network and resource selection model in 5G networks
IAESIJAI
 
Blue Purple Modern Animated Computer Science Presentation.pdf.pdf
Akar16
 
TokAI - TikTok AI Agent : The First AI Application That Analyzes 10,000+ Vira...
SOFTTECHHUB
 
Encapsulation_ Review paper, used for researhc scholars
gurumoop
 
Empathic Computing: Creating Shared Understanding
Mark Billinghurst
 
Teaching material agriculture food technology
LiaRayya
 
The AUB Centre for AI in Media Proposal.docx
GAONE9
 
Machine learning based COVID-19 study performance prediction
IAESIJAI
 
Spectroscopy.pptx food analysis technology
nawahassan45
 
Electronic commerce courselecture one. Pdf
OmerMohamed64
 

Fun with SQL

  • 1. Fun With SQL Joshua Tolley End Point Corporation Why and Why Not Joins CROSS JOIN Fun With SQL INNER JOIN OUTER JOIN NATURAL JOIN Self Joins Other Useful Joshua Tolley Operations Subqueries End Point Corporation Set Operations Common Operations Advanced Operations Common Table October 5, 2009 Expressions Window Functions Real, Live Queries Something Simple Something Fun Key Points
  • 2. Fun With SQL Joshua Tolley End Point Corporation Why and Why Not Joins CROSS JOIN INNER JOIN OUTER JOIN NATURAL JOIN ”The degree of normality in a database is inversely Self Joins proportional to that of its DBA.” - Anon, twitter Other Useful Operations Subqueries Set Operations Common Operations Advanced Operations Common Table Expressions Window Functions Real, Live Queries Something Simple Something Fun Key Points
  • 3. Fun With SQL Why Not Do Stuff in SQL Joshua Tolley End Point Databases are harder to replicate, if you really need to Corporation scale out Why and Why Not Often, one complex SQL query is more efficient than Joins several simple ones CROSS JOIN Sometimes, indeed, it’s useful to reduce the load on the INNER JOIN OUTER JOIN database by moving logic into the application. Be NATURAL JOIN careful doing this Self Joins Other Useful c.f. Premature Optimization Operations Subqueries More complex queries are harder to write and debug Set Operations Common Operations True. But so is more complex programming. Advanced Operations More complex queries are harder for the next guy to Common Table Expressions maintain Window Functions Also, good DBAs are often more expensive than good Real, Live Queries Something Simple programmers Something Fun These are both true. But complex programming is also Key Points hard for the next guy to maintain Of all the reasons not to write fluent SQL, this is probably the most widely applicable
  • 4. Fun With SQL Why do stuff in SQL? Joshua Tolley End Point Corporation Why and Why Not The database is more efficient than your application for Joins processing big chunks of data CROSS JOIN INNER JOIN ...especially if your code is in an interpreted language OUTER JOIN NATURAL JOIN The database is better tested than your application Self Joins Applications trying to do what SQL should be doing Other Useful Operations often get big and complex quickly Subqueries Set Operations ...and also buggy quickly Common Operations That’s what the database is there for Advanced Operations SQL is designed to express relations and conditions on Common Table Expressions Window Functions them. Your application’s language isn’t. Real, Live Queries A better understanding of SQL allows you to write Something Simple Something Fun queries that perform better Key Points
  • 5. Fun With SQL Why do stuff in SQL? Joshua Tolley End Point Corporation Why and Why Not Joins CROSS JOIN INNER JOIN OUTER JOIN NATURAL JOIN Self Joins In short, the database exists to manage data, and your Other Useful Operations application exists to handle business logic. Write software Subqueries accordingly. Set Operations Common Operations Advanced Operations Common Table Expressions Window Functions Real, Live Queries Something Simple Something Fun Key Points
  • 6. Fun With SQL Joshua Tolley End Point Corporation Why and Why Not Joins CROSS JOIN INNER JOIN OUTER JOIN NATURAL JOIN Self Joins So let’s get started... Other Useful Operations Subqueries Set Operations Common Operations Advanced Operations Common Table Expressions Window Functions Real, Live Queries Something Simple Something Fun Key Points
  • 7. Fun With SQL Tables we’ll use Joshua Tolley End Point Corporation Why and Why Not Joins CROSS JOIN # SELECT * FROM a; # SELECT * FROM b; INNER JOIN OUTER JOIN NATURAL JOIN id | value id | value Self Joins ----+------- ----+------- Other Useful Operations 1 | a1 5 | b5 Subqueries Set Operations 2 | a2 4 | b4 Common Operations 3 | a3 3 | b3 Advanced Operations 4 | a4 6 | b6 Common Table Expressions (4 rows) (4 rows) Window Functions Real, Live Queries Something Simple Something Fun Key Points
  • 8. Fun With SQL JOINs Joshua Tolley End Point Corporation Why and Why Not Joins CROSS JOIN INNER JOIN OUTER JOIN NATURAL JOIN If you want data from multiple tables, you probably Self Joins Other Useful want a join Operations ...but see also Subqueries, later on Subqueries Set Operations Common Operations There are several different kinds of joins Advanced Operations Common Table Expressions Window Functions Real, Live Queries Something Simple Something Fun Key Points
  • 9. Fun With SQL JOINs Joshua Tolley End Point Corporation Why and Why Not Joins CROSS JOIN INNER JOIN <table1> [alias1] OUTER JOIN [ [ [NATURAL] [ [FULL | RIGHT | LEFT] [OUTER] | NATURAL JOIN Self Joins INNER] ] | CROSS ] JOIN Other Useful Operations <table2> [alias2] Subqueries Set Operations [USING (...) | Common Operations ON (<value1> <op> <value2> Advanced Operations [,<value3> <op> <value4>...] ) ] Common Table Expressions Window Functions Real, Live Queries Something Simple Something Fun Key Points
  • 10. Fun With SQL CROSS JOIN Joshua Tolley End Point Corporation Why and Why Not Joins SELECT <... >FROM table1 JOIN table2 CROSS JOIN INNER JOIN With no explicit join type and no join qualifiers (an ON OUTER JOIN NATURAL JOIN clause, WHERE clause involving both relations, etc.) Self Joins this is a CROSS JOIN Other Useful Operations Equivalent to Subqueries Set Operations Common Operations SELECT <... >FROM table1, table2 Advanced SELECT <... >FROM table1 CROSS JOIN table2 Operations Common Table ”Cartesian product” of the two relations Expressions Window Functions Combines every row of table1 with every row of table2 Real, Live Queries Makes LOTS of rows, and can thus be very slow Something Simple Something Fun Key Points
  • 11. Fun With SQL CROSS JOIN Joshua Tolley End Point Corporation # SELECT * FROM a, b; Why and Why Not id | value | id | value Joins CROSS JOIN ----+-------+----+------- INNER JOIN OUTER JOIN 1 | a1 | 5 | b5 NATURAL JOIN Self Joins 1 | a1 | 4 | b4 Other Useful <snip> Operations Subqueries 3 | a3 | 3 | b3 Set Operations Common Operations 3 | a3 | 6 | b6 Advanced Operations 4 | a4 | 5 | b5 Common Table Expressions 4 | a4 | 4 | b4 Window Functions 4 | a4 | 3 | b3 Real, Live Queries Something Simple 4 | a4 | 6 | b6 Something Fun (16 rows) Key Points
  • 12. Fun With SQL INNER JOIN Joshua Tolley End Point Corporation Why and Why Not Joins CROSS JOIN INNER JOIN OUTER JOIN NATURAL JOIN SELECT <...>FROM table1 INNER JOIN table2 ON Self Joins Other Useful (table1.field = table2.field ...) Operations Subqueries Only returns rows satisfying the ON condition Set Operations Common Operations Equivalent to a CROSS JOIN with a WHERE clause Advanced Operations Common Table Expressions Window Functions Real, Live Queries Something Simple Something Fun Key Points
  • 13. Fun With SQL INNER JOIN Joshua Tolley End Point Corporation Why and Why Not Joins CROSS JOIN INNER JOIN OUTER JOIN # SELECT * FROM a INNER JOIN b USING (id); NATURAL JOIN Self Joins id | value | value Other Useful ----+-------+------- Operations Subqueries 3 | a3 | b3 Set Operations Common Operations 4 | a4 | b4 Advanced (2 rows) Operations Common Table Expressions Window Functions Real, Live Queries Something Simple Something Fun Key Points
  • 14. Fun With SQL OUTER JOIN Joshua Tolley End Point Corporation Why and Why Not Joins Return all rows from one or both relations CROSS JOIN INNER JOIN OUTER JOIN LEFT: Return all rows from the relation on the left NATURAL JOIN Self Joins RIGHT: Return all rows from the relation on the right Other Useful Operations FULL: Return all rows from both relations Subqueries Set Operations Returns nulls for values from one relation when it Common Operations Advanced contains to match with the other relation Operations Common Table The OUTER keyword is redundant Expressions Window Functions Requires ON or USING clause Real, Live Queries Something Simple Something Fun Key Points
  • 15. Fun With SQL LEFT JOIN Joshua Tolley End Point Corporation Why and Why Not Joins CROSS JOIN # SELECT * FROM a LEFT JOIN b USING (id); INNER JOIN OUTER JOIN id | value | value NATURAL JOIN Self Joins ----+-------+------- Other Useful 1 | a1 | Operations Subqueries 2 | a2 | Set Operations Common Operations 3 | a3 | b3 Advanced Operations 4 | a4 | b4 Common Table Expressions (4 rows) Window Functions Real, Live Queries Something Simple Something Fun Key Points
  • 16. Fun With SQL RIGHT JOIN Joshua Tolley End Point Corporation Why and Why Not Joins CROSS JOIN # SELECT * FROM a RIGHT JOIN b USING (id); INNER JOIN OUTER JOIN id | value | value NATURAL JOIN Self Joins ----+-------+------- Other Useful 3 | a3 | b3 Operations Subqueries 4 | a4 | b4 Set Operations Common Operations 5 | | b5 Advanced Operations 6 | | b6 Common Table Expressions (4 rows) Window Functions Real, Live Queries Something Simple Something Fun Key Points
  • 17. Fun With SQL FULL JOIN Joshua Tolley End Point Corporation Why and Why Not Joins # select * from a full join b using (id); CROSS JOIN id | value | value INNER JOIN OUTER JOIN ----+-------+------- NATURAL JOIN Self Joins 1 | a1 | Other Useful Operations 2 | a2 | Subqueries Set Operations 3 | a3 | b3 Common Operations 4 | a4 | b4 Advanced Operations 5 | | b5 Common Table Expressions 6 | | b6 Window Functions (6 rows) Real, Live Queries Something Simple Something Fun Key Points
  • 18. Fun With SQL Applications Joshua Tolley End Point Corporation Why and Why Not Find rows with no match in table b: Joins CROSS JOIN INNER JOIN OUTER JOIN # SELECT * FROM a LEFT JOIN b USING (id) NATURAL JOIN Self Joins WHERE b.value IS NULL; Other Useful id | value | value Operations Subqueries ----+-------+------- Set Operations Common Operations 1 | a1 | Advanced Operations 2 | a2 | Common Table Expressions (2 rows) Window Functions Real, Live Queries Something Simple Something Fun Key Points
  • 19. Fun With SQL NATURAL JOIN Joshua Tolley End Point Corporation Why and Why Not Joins CROSS JOIN INNER JOIN OUTER JOIN NATURAL JOIN Self Joins Other Useful NATURAL is syntactic sugar to match all columns with Operations the same name Subqueries Set Operations Common Operations Advanced Operations Common Table Expressions Window Functions Real, Live Queries Something Simple Something Fun Key Points
  • 20. Fun With SQL NATURAL JOIN Joshua Tolley End Point Corporation # SELECT * FROM a NATURAL FULL JOIN b; id | value Why and Why Not Joins ----+------- CROSS JOIN 1 | a1 INNER JOIN OUTER JOIN 2 | a2 NATURAL JOIN Self Joins 3 | a3 Other Useful Operations 3 | b3 Subqueries Set Operations 4 | a4 Common Operations 4 | b4 Advanced Operations 5 | b5 Common Table Expressions 6 | b6 Window Functions (8 rows) Real, Live Queries Something Simple Something Fun Key Points This looked for matches in both the id and value columns, so no rows matched. It returned all rows of both relations because it’s a FULL JOIN.
  • 21. Fun With SQL Self Joins Joshua Tolley End Point Corporation Why and Why Not Joins CROSS JOIN INNER JOIN OUTER JOIN NATURAL JOIN Self Joins ”Self joins” are particularly counterintuitive Other Useful Operations Joins one table to itself Subqueries Set Operations It helps to give the table two different aliases Common Operations Advanced Operations Common Table Expressions Window Functions Real, Live Queries Something Simple Something Fun Key Points
  • 22. Fun With SQL Self Joins Joshua Tolley Find all employees’ names, and each employee’s manager End Point Corporation SELECT Why and Why Not e.first || ’ ’ || e.last, Joins (SELECT CROSS JOIN INNER JOIN m.first || ’ ’ || m.last OUTER JOIN NATURAL JOIN FROM employee m Self Joins Other Useful WHERE m.id = e.manager); Operations Subqueries Set Operations Common Operations ... will generally be much faster rewritten as ... Advanced Operations Common Table SELECT Expressions Window Functions e.first || ’ ’ || e.last, Real, Live Queries m.first || ’ ’ || m.last Something Simple Something Fun FROM Key Points employee e JOIN employee m ON (e.manager = m.id)
  • 23. Fun With SQL Joshua Tolley End Point Corporation Why and Why Not Joins CROSS JOIN INNER JOIN OUTER JOIN NATURAL JOIN Self Joins More useful operations... Other Useful Operations Subqueries Set Operations Common Operations Advanced Operations Common Table Expressions Window Functions Real, Live Queries Something Simple Something Fun Key Points
  • 24. Fun With SQL Subqueries Joshua Tolley End Point Corporation Why and Why Not Joins Embeds one query within another CROSS JOIN INNER JOIN OUTER JOIN Examples (some bad, some good) NATURAL JOIN Self Joins SELECT id FROM table WHERE field = (SELECT Other Useful MAX(field) FROM table) Operations SELECT id, (SELECT COUNT(*) FROM table2 Subqueries Set Operations WHERE id = table1.id) FROM table1 Common Operations SELECT a, b FROM (SELECT a, COUNT(*) AS c Advanced Operations FROM table1) t1 JOIN (SELECT b, COUNT(*) AS c Common Table Expressions FROM table2) t2 on (t1.c = t2.c) Window Functions You can join subqueries just like you’d join tables Real, Live Queries Something Simple Something Fun Key Points
  • 25. Fun With SQL Set Operations Joshua Tolley End Point Corporation INTERSECT Why and Why Not Returns the intersection of two sets Joins Doesn’t exist in MySQL CROSS JOIN INNER JOIN SELECT (SELECT a, b FROM table1) INTERSECT OUTER JOIN NATURAL JOIN (SELECT c, d FROM table2) Self Joins UNION Other Useful Operations Appends one set of rows to another set with matching Subqueries Set Operations column types Common Operations SELECT a FROM table1 UNION SELECT b FROM Advanced Operations table2 Common Table Expressions EXCEPT Window Functions Real, Live Queries Returns rows in one SELECT that aren’t in another Something Simple SELECT Something Fun SELECT a FROM table1 EXCEPT SELECT b FROM Key Points table2
  • 26. Fun With SQL Common Operations Joshua Tolley End Point Corporation Why and Why Not Joins CROSS JOIN COALESCE(a, b) INNER JOIN OUTER JOIN If a is null, return b, else return a NATURAL JOIN Self Joins SELECT COALESCE(first, ’<NULL>’) FROM table Other Useful Oracle calls this NVL() Operations Subqueries CASE...WHEN Set Operations Common Operations Conditional operation Advanced SELECT CASE WHEN langused IN (’Lisp’, ’OCaml’, Operations Common Table ’Haskell’) THEN ’Functional’ ELSE ’Imperative’ AS Expressions Window Functions langtype FROM software Real, Live Queries Something Simple Something Fun Key Points
  • 27. Fun With SQL Series Generation Joshua Tolley End Point Corporation generate series() in PostgreSQL; might be something else in other databases Why and Why Not Joins Returns a series of numbers CROSS JOIN INNER JOIN Can be used like a for loop (example given later) OUTER JOIN NATURAL JOIN Self Joins # SELECT * FROM generate_series(1, 5); Other Useful Operations generate_series Subqueries Set Operations ----------------- Common Operations Advanced 1 Operations 2 Common Table Expressions Window Functions 3 Real, Live Queries 4 Something Simple Something Fun 5 Key Points (5 rows)
  • 28. Fun With SQL Common Table Expressions Joshua Tolley End Point Corporation Why and Why Not Joins CROSS JOIN Abbreviated CTEs INNER JOIN OUTER JOIN Fairly advanced; not available in all databases NATURAL JOIN Self Joins Not in PostgreSQL before v. 8.4, or any version of Other Useful Operations MySQL Subqueries Set Operations It’s just like defining a one-time view for your query Common Operations One major benefit: CTEs allow recursion Advanced Operations Recursing with CTEs is much more efficent than Common Table Expressions processing recursive data in your application Window Functions Real, Live Queries Something Simple Something Fun Key Points
  • 29. Fun With SQL A Simple CTE Example Joshua Tolley End Point Corporation Why and Why Not # SELECT * FROM GENERATE_SERIES(1,3) Joins CROSS JOIN CROSS JOIN INNER JOIN (SELECT * FROM GENERATE_SERIES(8,9)) AS f; OUTER JOIN NATURAL JOIN generate_series | generate_series Self Joins -----------------+----------------- Other Useful Operations 1 | 8 Subqueries Set Operations 1 | 9 Common Operations 2 | 8 Advanced Operations 2 | 9 Common Table Expressions 3 | 8 Window Functions Real, Live Queries 3 | 9 Something Simple Something Fun (6 rows) Key Points
  • 30. Fun With SQL A Simple CTE Example Joshua Tolley End Point Corporation # WITH t AS ( Why and Why Not SELECT * FROM GENERATE_SERIES(8,9) Joins ) CROSS JOIN INNER JOIN SELECT * FROM GENERATE_SERIES(1,3) OUTER JOIN NATURAL JOIN CROSS JOIN t; Self Joins Other Useful generate_series | generate_series Operations -----------------+----------------- Subqueries Set Operations 1 | 8 Common Operations Advanced 1 | 9 Operations Common Table 2 | 8 Expressions Window Functions 2 | 9 Real, Live Queries 3 | 8 Something Simple Something Fun 3 | 9 Key Points (6 rows)
  • 31. Fun With SQL Joshua Tolley End Point Corporation Why and Why Not Joins CROSS JOIN INNER JOIN OUTER JOIN NATURAL JOIN That last example was a bit cheesy, but the technique can Self Joins Other Useful be useful for complex queries in several parts Operations Subqueries Set Operations Common Operations Advanced Operations Common Table Expressions Window Functions Real, Live Queries Something Simple Something Fun Key Points
  • 32. Fun With SQL Recursion Joshua Tolley End Point Corporation Start with this: Why and Why Not Joins # SELECT * FROM employee; CROSS JOIN INNER JOIN first | last | id | manager OUTER JOIN NATURAL JOIN --------+----------+----+--------- Self Joins john | doe | 1 | Other Useful Operations fred | rogers | 2 | 1 Subqueries Set Operations speedy | gonzales | 3 | 1 Common Operations carly | fiorina | 4 | 1 Advanced Operations hans | reiser | 5 | 2 Common Table Expressions johnny | carson | 6 | 5 Window Functions Real, Live Queries martha | stewart | 7 | 3 Something Simple (7 rows) Something Fun Key Points
  • 33. Fun With SQL Recursion Joshua Tolley Recursive CTE to retrieve management hierarchy: End Point Corporation # WITH RECURSIVE t (id, managernames) AS ( Why and Why Not SELECT e.id, first || ’ ’ || last Joins AS managernames CROSS JOIN INNER JOIN FROM employee e WHERE manager IS NULL OUTER JOIN NATURAL JOIN UNION ALL Self Joins Other Useful SELECT e.id, Operations Subqueries first || ’ ’ || last || ’, ’ || managernames Set Operations Common Operations AS managernames Advanced FROM employee e Operations Common Table JOIN t ON (e.manager = t.id) Expressions Window Functions WHERE manager IS NOT NULL Real, Live Queries ) Something Simple Something Fun SELECT e.id, first || ’ ’ || last AS name, Key Points managernames FROM employee e JOIN t ON (e.id = t.id);
  • 34. Fun With SQL Recursion Joshua Tolley End Point Corporation ...and get this... Why and Why Not id | name | managernames Joins ----+-----------------+-----------------------------CROSS JOIN INNER JOIN 1 | john doe | john doe OUTER JOIN NATURAL JOIN 2 | fred rogers | fred rogers, john doe Self Joins 3 | speedy gonzales | speedy gonzales, john doe Other Useful Operations 4 | carly fiorina | carly fiorina, john doe Subqueries Set Operations 5 | hans reiser | hans reiser, fred rogers, Common Operations Advanced | | john doe Operations 6 | johnny carson | johnny carson, hans reiser, Common Table Expressions Window Functions | | fred rogers, john doe Real, Live Queries 7 | martha stewart | martha stewart, speedy Something Simple Something Fun | | gonzales, john doe Key Points (7 rows)
  • 35. Fun With SQL Fractals in SQL Joshua Tolley End Point Corporation WITH RECURSIVE x(i) AS (VALUES(0) UNION ALL SELECT i + 1 FROM x WHERE i < 101), Z(Ix, Iy, Cx, Cy, X, Y, I) Why and Why Not AS ( SELECT Ix, Iy, X::float, Y::float, X::float, Y::float, 0 Joins FROM (SELECT -2.2 + 0.031 * i, i FROM x) AS xgen(x,ix) CROSS JOIN CROSS JOIN INNER JOIN (SELECT -1.5 + 0.031 * i, i FROM x) AS ygen(y,iy) OUTER JOIN UNION ALL NATURAL JOIN Self Joins SELECT Ix, Iy, Cx, Cy, X * X - Y * Y + Cx AS X, Other Useful Y * X * 2 + Cy, I + 1 Operations FROM Z Subqueries WHERE X * X + Y * Y < 16.0 AND I < 27), Set Operations Zt (Ix, Iy, I) AS ( Common Operations SELECT Ix, Iy, MAX(I) AS I FROM Z GROUP BY Iy, Ix Advanced ORDER BY Iy, Ix Operations ) Common Table SELECT array_to_string( Expressions array_agg( Window Functions SUBSTRING(’ .,,,-----++++%%%%@@@@#### ’, Real, Live Queries GREATEST(I,1), 1) Something Simple ),’’ Something Fun ) FROM Zt GROUP BY Iy ORDER BY Iy; Key Points (yes, this query is SQL-spec compliant)
  • 36. Fun With SQL Joshua Tolley End Point Corporation Why and Why Not Joins CROSS JOIN INNER JOIN OUTER JOIN NATURAL JOIN Self Joins Other Useful Operations Subqueries Set Operations Common Operations Advanced Operations Common Table Expressions Window Functions Real, Live Queries Something Simple Something Fun Key Points
  • 37. Fun With SQL Window Functions Joshua Tolley End Point Corporation Why and Why Not Joins CROSS JOIN INNER JOIN OUTER JOIN Like CTEs, these are quite advanced NATURAL JOIN Self Joins Also unavailable in MySQL, and PostgreSQL before 8.4 Other Useful Operations Allow ranking, moving averages Subqueries Set Operations Like a set-returning aggregate function. Window Common Operations Advanced functions return results for each row based on a Operations ”window” of related rows Common Table Expressions Window Functions Real, Live Queries Something Simple Something Fun Key Points
  • 38. Fun With SQL Window Functions Joshua Tolley End Point Corporation If our employee table had department and salary information... Why and Why Not Joins # SELECT first, last, salary, department CROSS JOIN INNER JOIN FROM employee; OUTER JOIN NATURAL JOIN first | last | salary | department Self Joins Other Useful --------+----------+--------+---------------- Operations fred | rogers | 97000 | sales Subqueries Set Operations carly | fiorina | 95000 | sales Common Operations Advanced johnny | carson | 89000 | sales Operations speedy | gonzales | 96000 | development Common Table Expressions Window Functions hans | reiser | 93000 | development Real, Live Queries martha | stewart | 90000 | development Something Simple Something Fun john | doe | 99000 | administration Key Points (7 rows)
  • 39. Fun With SQL Window Functions Example Joshua Tolley End Point Corporation Why and Why Not Joins Rank employees in each department by salary CROSS JOIN INNER JOIN OUTER JOIN NATURAL JOIN SELECT first, last, salary, department, Self Joins RANK() OVER ( Other Useful Operations PARTITION BY department Subqueries Set Operations ORDER BY salary DESC Common Operations ) Advanced Operations FROM employee Common Table Expressions Window Functions Real, Live Queries Something Simple Something Fun Key Points
  • 40. Fun With SQL Window Functions Example Joshua Tolley End Point Corporation Why and Why Not ... and get this: Joins CROSS JOIN INNER JOIN first | last | salary | department | rank OUTER JOIN --------+----------+--------+----------------+------ NATURAL JOIN Self Joins john | doe | 99000 | administration | 1 Other Useful speedy | gonzales | 96000 | development | 1 Operations hans | reiser | 93000 | development | 2 Subqueries Set Operations martha | stewart | 90000 | development | 3 Common Operations fred | rogers | 97000 | sales | 1 Advanced carly | fiorina | 95000 | sales | 2 Operations Common Table johnny | carson | 89000 | sales | 3 Expressions Window Functions (7 rows) Real, Live Queries Something Simple Something Fun Key Points
  • 41. Fun With SQL Joshua Tolley End Point Corporation Why and Why Not Joins CROSS JOIN INNER JOIN OUTER JOIN NATURAL JOIN Self Joins Real, live queries Other Useful Operations Subqueries Set Operations Common Operations Advanced Operations Common Table Expressions Window Functions Real, Live Queries Something Simple Something Fun Key Points
  • 42. Fun With SQL Something Simple Joshua Tolley End Point Corporation Why and Why Not The slow version: Joins CROSS JOIN SELECT DISTINCT(sync) FROM bucardo.bucardo_rate INNER JOIN OUTER JOIN ORDER BY 1 NATURAL JOIN Self Joins Other Useful Operations The fast version: Subqueries Set Operations Common Operations SELECT name FROM sync WHERE EXISTS ( Advanced Operations SELECT 1 FROM bucardo_rate Common Table Expressions WHERE sync = name LIMIT 1) Window Functions Real, Live Queries ORDER BY 1 Something Simple Something Fun Key Points
  • 43. Fun With SQL Something Simple Joshua Tolley End Point Corporation Why and Why Not Joins CROSS JOIN INNER JOIN OUTER JOIN The bucardo rate table is huge, with few distinct values NATURAL JOIN Self Joins finding ”DISTINCT sync” requires a long table scan Other Useful Operations The sync table contains a list of all possible values in Subqueries Set Operations the bucardo rate.sync column Common Operations Advanced So instead of a big table scan, we scan the small table, Operations and filter out values can’t find in bucardo rate Common Table Expressions Window Functions Real, Live Queries Something Simple Something Fun Key Points
  • 44. Fun With SQL Joshua Tolley End Point Corporation Why and Why Not Joins CROSS JOIN INNER JOIN OUTER JOIN NATURAL JOIN Self Joins Something Fun Other Useful Operations Subqueries Set Operations Common Operations Advanced Operations Common Table Expressions Window Functions Real, Live Queries Something Simple Something Fun Key Points
  • 45. Fun With SQL Joshua Tolley SELECT End Point id, idname, Corporation COALESCE(ROUND(AVG(synctime)::NUMERIC, 1), 0) AS avgtime, COALESCE(SUM(total), 0) AS count FROM ( Why and Why Not SELECT slavecommit, EXTRACT(EPOCH FROM slavecommit - mastercommit) AS synctime, Joins total CROSS JOIN FROM bucardo.bucardo_rate INNER JOIN WHERE sync = ’RO_everything’ AND OUTER JOIN mastercommit > (NOW() - (15 + 1) * INTERVAL ’1 HOUR’) NATURAL JOIN ) i Self Joins RIGHT JOIN ( Other Useful SELECT id, idname, Operations TO_TIMESTAMP(start - start::INTEGER % 3600) AS start, Subqueries TO_TIMESTAMP(stop - stop::INTEGER % 3600) AS stop Set Operations FROM ( Common Operations SELECT id, TO_CHAR(NOW() - id * INTERVAL ’1 HOUR’, Advanced ’Dy Mon DD HH:MI AM’) AS idname, Operations EXTRACT(EPOCH FROM NOW() - id * INTERVAL ’1 HOUR’) AS start, Common Table EXTRACT(EPOCH FROM NOW() - (id - 1) * INTERVAL ’1 HOUR’) AS stop Expressions FROM ( Window Functions SELECT GENERATE_SERIES(1, 15) AS id Real, Live Queries ) f ) g Something Simple Something Fun ) h ON (slavecommit BETWEEN start AND stop) GROUP BY id, idname Key Points ORDER BY id DESC;
  • 46. Fun With SQL Something Fun Joshua Tolley End Point Corporation Why and Why Not Joins CROSS JOIN INNER JOIN OUTER JOIN The table contains replication data NATURAL JOIN Self Joins Time of commit on master Other Useful Time of commit on slave Operations Subqueries Number of rows replicated Set Operations Common Operations The user wants a graph of replication speed over time, Advanced given a user-determined range of time Operations Common Table Expressions Window Functions Real, Live Queries Something Simple Something Fun Key Points
  • 47. Fun With SQL Something Fun Joshua Tolley End Point We want to average replication times over a series of Corporation buckets. The first part of our query creates those buckets, Why and Why Not based on generate series(). Here we create buckets for Joins 15 hours CROSS JOIN INNER JOIN OUTER JOIN SELECT NATURAL JOIN Self Joins id, Other Useful Operations TO_CHAR(NOW() - id * INTERVAL ’1 HOUR’, Subqueries ’Dy Mon DD HH:MI AM’) AS idname, Set Operations Common Operations EXTRACT(EPOCH FROM NOW() - id * Advanced Operations INTERVAL ’1 HOUR’) AS start, Common Table Expressions EXTRACT(EPOCH FROM NOW() - (id - 1) * Window Functions INTERVAL ’1 HOUR’) AS stop Real, Live Queries Something Simple FROM ( Something Fun SELECT GENERATE_SERIES(1, 15) AS id Key Points ) f
  • 48. Fun With SQL Something Fun Joshua Tolley End Point Corporation Why and Why Not Joins CROSS JOIN This gives us: INNER JOIN OUTER JOIN NATURAL JOIN id | idname | start | stop Self Joins ----+---------------------+------------------+------------------ Other Useful 1 | Sat Mar 14 10:23 PM | 1237091036.95657 | 1237094636.95657 Operations 2 | Sat Mar 14 09:23 PM | 1237087436.95657 | 1237091036.95657 Subqueries Set Operations 3 | Sat Mar 14 08:23 PM | 1237083836.95657 | 1237087436.95657 Common Operations 4 | Sat Mar 14 07:23 PM | 1237080236.95657 | 1237083836.95657 Advanced ... Operations Common Table Expressions Window Functions Real, Live Queries Something Simple Something Fun Key Points
  • 49. Fun With SQL Something Fun Joshua Tolley End Point Corporation Why and Why Not Make the buckets end on nice time boundaries: Joins CROSS JOIN INNER JOIN SELECT id, idname, OUTER JOIN NATURAL JOIN TO_TIMESTAMP(start - start::INTEGER % 3600) Self Joins Other Useful AS start, Operations TO_TIMESTAMP(stop - stop::INTEGER % 3600) Subqueries Set Operations AS stop Common Operations Advanced FROM ( Operations -- The bucket query, shown earlier, goes here Common Table Expressions Window Functions ) g Real, Live Queries Something Simple Something Fun Key Points
  • 50. Fun With SQL Something Fun Joshua Tolley End Point Corporation Why and Why Not Joins CROSS JOIN INNER JOIN OUTER JOIN That gives us this: NATURAL JOIN Self Joins id | idname | start | stop ----+---------------------+-------------------------------+------------------------------- Other Useful 1 | Sat Mar 14 10:23 PM | 2009-03-14 21:59:59.956568-06 | 2009-03-14 22:59:59.956568-06 Operations 2 | Sat Mar 14 09:23 PM | 2009-03-14 20:59:59.956568-06 | 2009-03-14 21:59:59.956568-06 Subqueries 3 | Sat Mar 14 08:23 PM | 2009-03-14 19:59:59.956568-06 | 2009-03-14 20:59:59.956568-06 Set Operations 4 | Sat Mar 14 07:23 PM | 2009-03-14 18:59:59.956568-06 | 2009-03-14 19:59:59.956568-06 Common Operations Advanced Operations Common Table Expressions Window Functions Real, Live Queries Something Simple Something Fun Key Points
  • 51. Fun With SQL Something Fun Joshua Tolley End Point Corporation In an different subquery, select everything from the table of Why and Why Not the right time period and right sync. Call this the ”stats” Joins query: CROSS JOIN INNER JOIN OUTER JOIN SELECT NATURAL JOIN Self Joins slavecommit, Other Useful EXTRACT(EPOCH FROM slavecommit - mastercommit) Operations Subqueries AS synctime, Set Operations Common Operations total Advanced FROM bucardo.bucardo_rate Operations Common Table WHERE Expressions Window Functions sync = ’RO_everything’ AND Real, Live Queries Something Simple mastercommit > (NOW() - (15 + 1) * Something Fun INTERVAL ’1 HOUR’) Key Points
  • 52. Fun With SQL Something Fun Joshua Tolley End Point Corporation Why and Why Not ...which gives us this: Joins CROSS JOIN INNER JOIN slavecommit | synctime | total OUTER JOIN -------------------------------+------------------+-------- NATURAL JOIN Self Joins 2009-03-14 07:32:00.103759-06 | 5.65614098310471 | 1 Other Useful 2009-03-14 07:32:04.31508-06 | 5.25827997922897 | 3 Operations 2009-03-14 07:32:04.31508-06 | 5.25827997922897 | 5 Subqueries Set Operations 2009-03-14 07:32:08.700184-06 | 7.71899098157883 | 1 Common Operations 2009-03-14 07:32:08.700184-06 | 8.22490698099136 | 1 Advanced 2009-03-14 07:32:12.675518-06 | 7.85176599025726 | 6 Operations Common Table 2009-03-14 07:32:12.675518-06 | 7.15798497200012 | 6 Expressions Window Functions ... Real, Live Queries Something Simple Something Fun Key Points
  • 53. Fun With SQL Something Fun Joshua Tolley End Point Corporation Now, join the two queries: Why and Why Not SELECT Joins CROSS JOIN id, idname, INNER JOIN OUTER JOIN COALESCE(ROUND(AVG(synctime)::NUMERIC, 1), 0) NATURAL JOIN Self Joins AS avgtime, Other Useful COALESCE(SUM(total), 0) AS count Operations Subqueries FROM ( Set Operations Common Operations <STATS QUERY> Advanced ) RIGHT JOIN ( Operations Common Table <CALENDAR QUERY> Expressions Window Functions ) ON (slavecommit BETWEEN start AND stop) Real, Live Queries Something Simple GROUP BY id, idname Something Fun ORDER BY id DESC; Key Points
  • 54. Fun With SQL Something Fun Joshua Tolley End Point ...and get this: Corporation Why and Why Not id | idname | avgtime | count Joins ----+---------------------+---------+------- CROSS JOIN INNER JOIN 15 | Sat Mar 14 08:35 AM | 7.9 | 14219 OUTER JOIN NATURAL JOIN 14 | Sat Mar 14 09:35 AM | 6.9 | 16444 Self Joins 13 | Sat Mar 14 10:35 AM | 6.5 | 62100 Other Useful Operations 12 | Sat Mar 14 11:35 AM | 6.2 | 47349 Subqueries Set Operations 11 | Sat Mar 14 12:35 PM | 0 | 0 Common Operations Advanced 10 | Sat Mar 14 01:35 PM | 4.6 | 21348 Operations Common Table Expressions Window Functions This is the average replication time and total replicated rows Real, Live Queries Something Simple per hour. Note that this correctly returns zeroes when no Something Fun rows are replicated, and still returns a value for that time Key Points slot. This prevents some amount of application-side processing.
  • 55. Fun With SQL Something Fun Joshua Tolley End Point Corporation Why and Why Not Joins CROSS JOIN INNER JOIN OUTER JOIN NATURAL JOIN Self Joins Other Useful Operations That query again: Subqueries Set Operations Common Operations Advanced Operations Common Table Expressions Window Functions Real, Live Queries Something Simple Something Fun Key Points
  • 56. Fun With SQL Joshua Tolley SELECT End Point id, idname, Corporation COALESCE(ROUND(AVG(synctime)::NUMERIC, 1), 0) AS avgtime, COALESCE(SUM(total), 0) AS count FROM ( Why and Why Not SELECT slavecommit, EXTRACT(EPOCH FROM slavecommit - mastercommit) AS synctime, Joins total CROSS JOIN FROM bucardo.bucardo_rate INNER JOIN WHERE sync = ’RO_everything’ AND OUTER JOIN mastercommit > (NOW() - (15 + 1) * INTERVAL ’1 HOUR’) NATURAL JOIN ) i Self Joins RIGHT JOIN ( Other Useful SELECT id, idname, Operations TO_TIMESTAMP(start - start::INTEGER % 3600) AS start, Subqueries TO_TIMESTAMP(stop - stop::INTEGER % 3600) AS stop Set Operations FROM ( Common Operations SELECT id, TO_CHAR(NOW() - id * INTERVAL ’1 HOUR’, Advanced ’Dy Mon DD HH:MI AM’) AS idname, Operations EXTRACT(EPOCH FROM NOW() - id * INTERVAL ’1 HOUR’) AS start, Common Table EXTRACT(EPOCH FROM NOW() - (id - 1) * INTERVAL ’1 HOUR’) AS stop Expressions FROM ( Window Functions SELECT GENERATE_SERIES(1, 15) AS id Real, Live Queries ) f ) g Something Simple Something Fun ) h ON (slavecommit BETWEEN start AND stop) GROUP BY id, idname Key Points ORDER BY id DESC;
  • 57. Fun With SQL Key Points Joshua Tolley End Point Corporation Why and Why Not Joins CROSS JOIN INNER JOIN OUTER JOIN NATURAL JOIN Understand join types, and use them Self Joins Other Useful Know what functions and set operations your database Operations Subqueries provides Set Operations Common Operations Build large queries piece by piece Advanced Operations Common Table Expressions Window Functions Real, Live Queries Something Simple Something Fun Key Points
  • 58. Fun With SQL Joshua Tolley End Point Corporation Why and Why Not Joins CROSS JOIN INNER JOIN OUTER JOIN NATURAL JOIN Self Joins Questions? Other Useful Operations Subqueries Set Operations Common Operations Advanced Operations Common Table Expressions Window Functions Real, Live Queries Something Simple Something Fun Key Points