SlideShare a Scribd company logo

Pro PostgreSQL, OSCon 2008

Download as ODP, PDF
Robert Treat, profile picture
Robert Treat

This document provides an overview and introduction to PostgreSQL for new users. It covers getting started with PostgreSQL, including installing it, configuring authentication and logging, upgrading to new versions, routine maintenance tasks, hardware recommendations, availability and scalability options, and query tuning and optimization. The document is presented as a slide deck with different sections labeled by letters (e.g. K-0, S-0, U-0).

Technology
1 of 106
Downloaded 275 times
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
Pro PostgreSQL Robert Treat omniti.com brighterlamp.org
Who Am I? (Why Listen To Me) O-0 PostgreSQL User Since 6.5.x DBA of High Traffic / Large PostgreSQL Instances Long Time Contributor to PostgreSQL Project Contribute / Maintain Several Open Source Projects Co-Author Beginning PHP & PostgreSQL 8 (Apress)
Outline O-1 What you need to know about the project Getting started Upgrading Configuring your server Hardware Availability Scalability Query tuning Tablespaces Partitioning Stuff you should know about
K-0 Know Your Way Around The Project
Know Your Way Around The Project K-1 www.postgresql.org downloads documentation bug reports security alerts wiki support companies rss –> news – events - versions
Know Your Way Around The Project K-2 www.pgfoundry.org projects.postgresql.org Modules Programs Resources URI Type CIText SkyTools Npgsql Pl/Proxy pg_bulkload plpgsql-debugger sample databases
Know Your Way Around The Project K-3 www.planetpostgresql.org Project News Community News Helpful Tips / Examples
Know Your Way Around The Project K-4 archives.postgresql.org mailing list archives back to 1997 full text search via postgtresql 8.3 keyword search suggestions lists for users, developers, regional, user groups
Know Your Way Around The Project K-5 #postgresql irc.freenode.net real time help rtfm_please - ??help
Know Your Way Around The Project K-6 project management core team committers -hackers roadmap web team
S-0 Get Off To A Good Start
S-1 Get Off To A Good Start Use package management Consistent Standardized Simple
Different across systems Upgrades are an issue Trust your packager? S-2 Get Off To A Good Start Use package management
Different across systems Upgrades are an issue Trust your packager? S-2 Get Off To A Good Start Use package management Don't Be Afraid To Roll Your Own
S-4 Get Off To A Good Start $PGDATA/pg_log /var/log/pgsql when in doubt... (postgresql.conf) separate disk Configure Logging Logging is often overlooked, but is the first step toward troubleshooting!
S-5 Get Off To A Good Start most systems have different defaults firewalls/ selinux (FATAL) rtfm (pg_hba.conf, grant, revoke) Configure Authentication
S-6 Get Off To A Good Start TRUST md5 IDENT Authentication Methods
S-7 Get Off To A Good Start trust these more than your own code package dependent use different schemas (when able) pgcrypto pgstatstuple, pg_buffercache, pg_freespacemap /contrib
S-8 Get Off To A Good Start package dependent some are non-core (plruby, plr, plphp) varying functionality varying levels of trust don't be afraid, test! procedural languages
U-0 Let's Talk About Upgrades
U-1 Let's Talk About Upgrades Versioning First Digit ( 7 .4.16 -> 8 .2.0) Second Digit (8.2.4 -> 8.3.0) Third Digit (8.3.0 -> 8.3.1)
U-2 Let's Talk About Upgrades Versioning First Digit (7.4.16 -> 8.2.0) Second Digit (8. 2 .4 -> 8. 3 .0) Third Digit (8.3.0 -> 8.3.1)
U-3 Let's Talk About Upgrades Versioning First Digit (7.4.16 -> 8.2.0) Second Digit (8.2.4 -> 8.3.0) Third Digit (8.3. 0 -> 8.3. 1 )
U-4 Let's Talk About Upgrades Achtung!! Make Backups! Read the Release Notes!
U-5 Let's Talk About Upgrades pg_dump/pg_restore simple -Fc is your friend dump with new version of pg_dump pitfalls (time, hdd)
U-6 Let's Talk About Upgrades the slony method not simple create slave on new version switchover (switch back?) pitfalls (initial synch, compatibility)
U-7 Let's Talk About Upgrades pg_migrator in place upgrades rewrites system catalog info no way to go back (fs snapshots) still new, under-flux 8.1 -> 8.2 only (for now)
U-8 Let's Talk About Upgrades upgrading older db <= 7.3 is no longer supported (upgrade now!) pg_dump 8.2 has issues with <= 7.2 you can upgrade to 7.3 first use adddepends on 7.3 install slony requires 7.3 (or 7.4) (or 8.3) or newer pg_migrator (lol)
C-0 Figure Your Configure
C-1 Figure Your Configure the basics : performance effective_cache_size shared_buffers default_statistics_target work_mem checkpoint_segments
C-2 Figure Your Configure the basics : logging stderr/pg_log vs. syslog/eventlog log_min_error_statement (error!) log_min_duration_statement log_line_prefix (%d, %p, %t)
C-3 Figure Your Configure other stuff worth looking at maintenance_work_mem max_prepared_transactions update_process_title max_fsm_pages synchronous_commit
M-0 Routine Maintenance
M-1 Routine Maintenance a word about vacuum reclaim usable space update table stats avoid xid wraparound
M-2 Routine Maintenance autovacuum : just do it! autovacuum track_activities track_counts autovacuum_max_freeze_age pg_autovacuum ?
M-3 Routine Maintenance other stuff worth looking at reindexing logfiles backups failover
P-0 Hardware For Software People
P-1 Hardware For Software People you can be cheap on the licensing, don't be cheap on the hardware Cram the (ecc) RAM Redundant Disks (Software|Hardware) RAID Prefer SCSI, Accept SATA RAID Z (JBOD)
P-2 Hardware For Software People cpu and you Multiple CPU's work wonders, up to 8 processors (16 OK) Opteron, Xeon, Core 2 Duo http://tweakers.net/
P-3 Hardware For Software People disk do's Put WAL on it's own disk (RAID 1) Put DATA directory on it's own disk (RAID 10) More Spindles is Good More Controllers Even Gooder (tablespaces!) Battery Backed Controllers, Write cache enabled
P-4 Hardware For Software People disk dont's NFS == Not Fully Safe! RAID 5 Beware disks / controllers that Lie
A-0 Availability (Not Scalability)
A-1 Availability what do we mean by availability? not backups (exactly) not replication (necessarily) not clustering (even less so)
A-2 Availability what do we mean by availability? if (kablooy) then (ok) not backups (exactly) not replication (necessarily) not clustering (even less so)
A-3 Availability pg_dump traditionally used for backups send dump to another server constantly run restore process large time, i/o constraints
A-4 Availability filesystem snapshots zfs, lvm built on top of pitr backup commands traditionally used for backups copy snapshot to another server cumbersome time, i/o constraints
A-5 Availability pitr create second, standby server ship wal logs to new server less time/io than pg_dump 8.1 -> cold standby 8.2 -> warm standby 8.4 -> hot standby ?
A-6 Availability slony asynchronous, master-slave replication controlled switchover, failover low i/o, time constraints other benefits (upgrades, scaling)
A-7 Availability bucardo asynchronous, multi-master replication also does master-slave low i/o, time constraints other benefits (upgrades, scaling)
A-8 Availability shared disk one copy of PGDATA on shared storage standby takes over akin to db crash shared disk is point of failure (raid) STONITH
A-9 Availability filesystem replication drbd filesystem mirrored between servers synchronized, ordered writes single disk system?
A-10 Availability pgpool dual-master, statement based little caveats (random(),now(),sequences) bigger caveats (security, password, pg_hba) pgpool becomes failure point
A-11 Availability postgres-r multi-master, synchronous just open sourced this month! small community not proven
H-0 Scalability
H-1 Scalability what is scaling? “ How well a solution to some problem will work when the relative size of the problem increases” - Theo Schlossnagle
H-2 Scalability bigger, better, faster, more! postgresql scales up pretty well more disks (tablespaces) more cpu's, more ram connection pooling 1000+ connections, TB+ data
H-3 Scalability pgpool dual-master, statement based little caveats (random(),now(),sequences) bigger caveats (security, password, pg_hba) pgpool becomes failure point
H-4 Scalability pg_bouncer simple connection pooler 10/1 -> 40/1 caveats (prepared statements, temp tables) skype, myyearbook.com
H-5 Scalability slony asynchronous, master-slave replication multiple, cascading slaves scales read operations other benefits (upgrades, scaling) solid user base
H-6 Scalability bucardo asynchronous, multi-master replication also does master-slave low i/o, time constraints other benefits (upgrades, scaling)
H-7 Scalability pgpool-II single db over multiple machines scales read operations replication, load balance, parallel query green technology
H-8 Scalability pgcluster synchronous multi-master replication significant complexity scales read operations other uses (failover abilities) green technology
H-9 Scalability postgres-r multi-master, synchronous just open source this month! small community other uses (failover abilities) not proven
H-10 Scalability pitr read-only slaves based on pitr, warm standby operation core team officially supporting development 8.4 -> synchronous wal shipping 8.? -> read only slaves
J-0 Query Your Queries
J-1 Query Your Queries finding slow queries: log_min_duration_statement -1, 0 , n superuser only alter user LOG: duration: 5005.273 ms statement: select pg_sleep(5);
J-2 Query Your Queries finding slow queries: pgfouine / pqa log analyzers command line, generate reports i/o load http://pgfouine.projects.postgresql.org/reports.html http://pqa.projects.postgresql.org/example.html
J-3 Query Your Queries finding slow queries: pg_stat_all_tables pagila=# \d pg_stat_all_tables View &quot;pg_catalog.pg_stat_all_tables&quot; Column | Type | ------------------+-------------+ relid | oid | schemaname | name | relname | name | seq_scan | bigint | seq_tup_read | bigint | idx_scan | bigint | idx_tup_fetch | bigint | n_tup_ins | bigint | n_tup_upd | bigint | n_tup_del | bigint | n_tup_hot_upd | bigint | n_live_tup | bigint | n_dead_tup | bigint | last_vacuum | timestamptz | last_autovacuum | timestamptz | last_analyze | timestamptz | last_autoanalyze | timestamptz |
J-4 Query Your Queries finding slow queries: pg_stat_all_tables pagila=# \d pg_stat_all_tables View &quot;pg_catalog.pg_stat_all_tables&quot; Column | Type | ------------------+-------------+ relid | oid | schemaname | name | relname | name | seq_scan | bigint | seq_tup_read | bigint | idx_scan | bigint | idx_tup_fetch | bigint | n_tup_ins | bigint | n_tup_upd | bigint | n_tup_del | bigint | n_tup_hot_upd | bigint | n_live_tup | bigint | n_dead_tup | bigint | last_vacuum | timestamptz | last_autovacuum | timestamptz | last_analyze | timestamptz | last_autoanalyze | timestamptz |
J-5 Query Your Queries finding slow queries: pg_stat_all_tables pagila=# \d pg_stat_all_tables View &quot;pg_catalog.pg_stat_all_tables&quot; Column | Type | ------------------+-------------+ relid | oid | schemaname | name | relname | name | seq_scan | bigint | seq_tup_read | bigint | idx_scan | bigint | idx_tup_fetch | bigint | n_tup_ins | bigint | n_tup_upd | bigint | n_tup_del | bigint | n_tup_hot_upd | bigint | n_live_tup | bigint | n_dead_tup | bigint | last_vacuum | timestamptz | last_autovacuum | timestamptz | last_analyze | timestamptz | last_autoanalyze | timestamptz |
J-6 Query Your Queries finding slow queries: pg_stat_all_tables pagila=# \d pg_stat_all_tables View &quot;pg_catalog.pg_stat_all_tables&quot; Column | Type | ------------------+-------------+ relid | oid | schemaname | name | relname | name | seq_scan | bigint | seq_tup_read | bigint | idx_scan | bigint | idx_tup_fetch | bigint | n_tup_ins | bigint | n_tup_upd | bigint | n_tup_del | bigint | n_tup_hot_upd | bigint | n_live_tup | bigint | n_dead_tup | bigint | last_vacuum | timestamptz | last_autovacuum | timestamptz | last_analyze | timestamptz | last_autoanalyze | timestamptz |
J-7 Query Your Queries finding slow queries: pg_stat_all_tables pagila=# \d pg_stat_all_tables View &quot;pg_catalog.pg_stat_all_tables&quot; Column | Type | ------------------+-------------+ relid | oid | schemaname | name | relname | name | seq_scan | bigint | seq_tup_read | bigint | idx_scan | bigint | idx_tup_fetch | bigint | n_tup_ins | bigint | n_tup_upd | bigint | n_tup_del | bigint | n_tup_hot_upd | bigint | n_live_tup | bigint | n_dead_tup | bigint | last_vacuum | timestamptz | last_autovacuum | timestamptz | last_analyze | timestamptz | last_autoanalyze | timestamptz |
J-8 Query Your Queries finding slow queries: pg_stat_all_tables pagila=# \d pg_stat_all_tables View &quot;pg_catalog.pg_stat_all_tables&quot; Column | Type | ------------------+-------------+ relid | oid | schemaname | name | relname | name | seq_scan | bigint | seq_tup_read | bigint | idx_scan | bigint | idx_tup_fetch | bigint | n_tup_ins | bigint | n_tup_upd | bigint | n_tup_del | bigint | n_tup_hot_upd | bigint | n_live_tup | bigint | n_dead_tup | bigint | last_vacuum | timestamptz | last_autovacuum | timestamptz | last_analyze | timestamptz | last_autoanalyze | timestamptz |
J-9 Query Your Queries finding slow queries: pg_stat_all_indexes pagila=# \d pg_stat_all_indexes View &quot;pg_catalog.pg_stat_all_indexes&quot; Column | Type | ---------------+--------+ relid | oid | indexrelid | oid | schemaname | name | relname | name | indexrelname | name | idx_scan | bigint | idx_tup_read | bigint | idx_tup_fetch | bigint |
J-10 Query Your Queries finding slow queries: pg_stat_all_indexes pagila=# \d pg_stat_all_indexes View &quot;pg_catalog.pg_stat_all_indexes&quot; Column | Type | ---------------+--------+ relid | oid | indexrelid | oid | schemaname | name | relname | name | indexrelname | name | idx_scan | bigint | idx_tup_read | bigint | idx_tup_fetch | bigint |
J-11 Query Your Queries finding slow queries: pg_statio_all_tables pagila=# \d pg_statio_all_tables View &quot;pg_catalog.pg_statio_all_tables&quot; Column | Type | -----------------+--------+ relid | oid | schemaname | name | relname | name | heap_blks_read | bigint | heap_blks_hit | bigint | idx_blks_read | bigint | idx_blks_hit | bigint | toast_blks_read | bigint | toast_blks_hit | bigint | tidx_blks_read | bigint | tidx_blks_hit | bigint |
J-12 Query Your Queries finding slow queries: pg_statio_all_tables pagila=# \d pg_statio_all_tables View &quot;pg_catalog.pg_statio_all_tables&quot; Column | Type | -----------------+--------+ relid | oid | schemaname | name | relname | name | heap_blks_read | bigint | heap_blks_hit | bigint | idx_blks_read | bigint | idx_blks_hit | bigint | toast_blks_read | bigint | toast_blks_hit | bigint | tidx_blks_read | bigint | tidx_blks_hit | bigint |
J-13 Query Your Queries fixing slow queries: explain analyze universal tool good for specific queries “ explain” for large queries could be it's own talk
J-14 Query Your Queries fixing slow queries: explain analyze universal tool good for specific queries “ explain” for large queries could be it's own talk http://wiki.postgresql.org/Using_EXPLAIN
J-15 Query Your Queries fixing slow queries: indexing (basic) use explain to find large sequential reads use pg_stat_* tables to find numerous reads btree – (gist/gin) enable_indexscan, enable_bitmapscan dual column vs. single column
J-16 Query Your Queries fixing slow queries: indexing (partial) create index address_ba_part_idx on address (district) where district = 'Buenos Aires'; restrain index to rows that matter can give significant speed improvements where clause of index should match where clause of query
J-17 Query Your Queries fixing slow queries: indexing (partial) create index customer_active_part_idx on customer (customer_id) where activebool is true; restrain index to rows that matter can give significant speed improvements where clause of index should match where clause of query
J-18 Query Your Queries fixing slow queries: indexing (functional) some people prefer to call these expressional indexes
J-19 Query Your Queries fixing slow queries: indexing (expressional) create unique index one_true_email_xidx on customer (lower(email)); push expensive functions into your index system sees just WHERE indexedcolumn = 'constant' expression of index should match expression of queries narrow scope, but nice gains
J-20 Query Your Queries fixing slow queries: indexing (expressional) create index fullname_xidx on customer ((first_name||' '||last_name)); push expensive functions into your index system sees just WHERE indexedcolumn = 'constant' expression of index should match expression of queries narrow scope, but nice gains
J-21 Query Your Queries fixing slow queries: full text search uses lexmes and word stemming to find common words replacement for LIKE '%x%', ~* 'x'; supports multiple languages, custom dictionaries special indexing options
J-22 Indexing Options full text indexing gist vs. gin “ old school” slower for queries faster insert / update mature new in 8.2 faster for queries slower insert / update stable
N-0 PostgreSQL Tablespaces
N-1 PostgreSQL Tablespaces tablespaces? define logical locations for object placement point to locations on disk (uses symlinks) size determined by disk size (not pre-ordained) dedicate per db, split db across multiple tblspc
N-2 PostgreSQL Tablespaces tablespaces! split database over separate disks use stat, statio tables to gauge disk access create dedicated storage for workloads disk for read / write disk for read only large, slow disk for archiving disk for indexes
Q-0 PostgreSQL Partitioning
Q-1 PostgreSQL Partitioning partitioning? as table size grows, it becomes unmanageable use inheritance, rules, constraints to split data queries ignore non-relevant partitions could be it's own talk
Q-2 PostgreSQL Partitioning partitioning! as table size grows, it becomes unmanageable use inheritance, rules, constraints to split data queries ignore non-relevant partitions could be it's own talk http://www.pgcon.org/2007/schedule/events/41.en.html
Q-3 PostgreSQL Partitioning partitioning : key points determine list vs. range use triggers rather than rules partition creation vs. data population automate maintenance
I-0 Other Stuff I Should Mention
I-1 Other Stuff I Should Mention pgcrypto cryptography type functions /contrib (export issues) md5, sha1, blowfish, many more
I-2 Other Stuff I Should Mention dblink pg -> pg connections /contrib (still under development?) can have performance issues on large queries make it live in it's own schema
I-3 Other Stuff I Should Mention *-link heterogenous connections for postgresql db specific and db independent options any pl/u language can implement this similar performance issues to dblink dblink-tds, dbi-link, oralink, odbclink http://www.pgfoundry.org/ (db link)
I-4 Other Stuff I Should Mention autonomous logging tool persistent logging for postgresql functions built on top of dblink make it live in it's own schema https://labs.omniti.com/trac/pgsoltools
I-5 Other Stuff I Should Mention snapshot pitr clones full read/write copy of pitr slave static snapshot need solaris (zfs zone mojo) could re-implement on other systems https://labs.omniti.com/trac/pgsoltools
I-6 Other Stuff I Should Mention check_postgres nagios based monitoring script common items for warnings and alerts can be adapted to other uses http://bucardo.org/check_postgres
I-7 Other Stuff I Should Mention reconnoiter monitoring / graphing tool postgres based still pretty green https://labs.omniti.com/trac/reconnoiter
I-8 Other Stuff I Should Mention phpPgAdmin web based gui for postgresql remote administration of multiple servers implements much of postgresql functionality support back to 7.2? http://phppgadmin.sourceforge.net/
I-9 Other Stuff I Should Mention ;-) my book?
I-10 Other Stuff I Should Mention ;-) we're hiring Ops Ninjas Perl Kung-Fu Artists PHP Ninjas Database Samurai http://omniti.com/is/hiring
L-0 El Fin

More Related Content

PDF
Inside PostgreSQL Shared Memory
EDB
 
PDF
PostgreSQL Performance Tuning
elliando dias
 
PDF
Mastering PostgreSQL Administration
EDB
 
PDF
Streaming replication in practice
Alexey Lesovsky
 
PDF
PostgreSQL Extensions: A deeper look
Jignesh Shah
 
PDF
Postgres Toolkit
Uptime Technologies LLC (JP)
 
PDF
High Availability PostgreSQL with Zalando Patroni
Zalando Technology
 
ODP
PostgreSQL Administration for System Administrators
Command Prompt., Inc
 
Inside PostgreSQL Shared Memory
EDB
 
PostgreSQL Performance Tuning
elliando dias
 
Mastering PostgreSQL Administration
EDB
 
Streaming replication in practice
Alexey Lesovsky
 
PostgreSQL Extensions: A deeper look
Jignesh Shah
 
Postgres Toolkit
Uptime Technologies LLC (JP)
 
High Availability PostgreSQL with Zalando Patroni
Zalando Technology
 
PostgreSQL Administration for System Administrators
Command Prompt., Inc
 

What's hot (19)

PPT
A brief introduction to PostgreSQL
Vu Hung Nguyen
 
PDF
PostgreSQL High Availability in a Containerized World
Jignesh Shah
 
PDF
PostgreSQL Streaming Replication Cheatsheet
Alexey Lesovsky
 
PDF
Problems with PostgreSQL on Multi-core Systems with MultiTerabyte Data
Jignesh Shah
 
PDF
Out of the box replication in postgres 9.4
Denish Patel
 
PDF
Introduction to PostgreSQL for System Administrators
Jignesh Shah
 
PPTX
Postgres-BDR with Google Cloud Platform
SungJae Yun
 
PDF
pgDay Asia 2016 - Swapping Pacemaker-Corosync for repmgr (1)
Wei Shan Ang
 
PDF
pgpool-II demonstration
elliando dias
 
PDF
plProxy, pgBouncer, pgBalancer
elliando dias
 
PDF
GitLab PostgresMortem: Lessons Learned
Alexey Lesovsky
 
PDF
pgpool: Features and Development
elliando dias
 
PDF
Out of the box replication in postgres 9.4(pg confus)
Denish Patel
 
PDF
Percona Toolkit for Effective MySQL Administration
Mydbops
 
PPTX
Automating Disaster Recovery PostgreSQL
Nina Kaufman
 
PDF
PostgreSQL High Availability in a Containerized World
Jignesh Shah
 
KEY
PostgreSQL
Reuven Lerner
 
PPTX
Postgresql Database Administration Basic - Day2
PoguttuezhiniVP
 
PPTX
HADOOP 실제 구성 사례, Multi-Node 구성
Young Pyo
 
A brief introduction to PostgreSQL
Vu Hung Nguyen
 
PostgreSQL High Availability in a Containerized World
Jignesh Shah
 
PostgreSQL Streaming Replication Cheatsheet
Alexey Lesovsky
 
Problems with PostgreSQL on Multi-core Systems with MultiTerabyte Data
Jignesh Shah
 
Out of the box replication in postgres 9.4
Denish Patel
 
Introduction to PostgreSQL for System Administrators
Jignesh Shah
 
Postgres-BDR with Google Cloud Platform
SungJae Yun
 
pgDay Asia 2016 - Swapping Pacemaker-Corosync for repmgr (1)
Wei Shan Ang
 
pgpool-II demonstration
elliando dias
 
plProxy, pgBouncer, pgBalancer
elliando dias
 
GitLab PostgresMortem: Lessons Learned
Alexey Lesovsky
 
pgpool: Features and Development
elliando dias
 
Out of the box replication in postgres 9.4(pg confus)
Denish Patel
 
Percona Toolkit for Effective MySQL Administration
Mydbops
 
Automating Disaster Recovery PostgreSQL
Nina Kaufman
 
PostgreSQL High Availability in a Containerized World
Jignesh Shah
 
PostgreSQL
Reuven Lerner
 
Postgresql Database Administration Basic - Day2
PoguttuezhiniVP
 
HADOOP 실제 구성 사례, Multi-Node 구성
Young Pyo
 
Ad

Viewers also liked (19)

PPT
Building a Spatial Database in PostgreSQL
Kudos S.A.S
 
PDF
PostgreSQL Scaling And Failover
John Paulett
 
PDF
My experience with embedding PostgreSQL
Jignesh Shah
 
PDF
Why use PostgreSQL?
Gabriele Bartolini
 
PDF
Android & PostgreSQL
Mark Wong
 
PDF
PostgreSQL Hooks for Fun and Profit
David Fetter
 
PDF
PostgreSQL and RAM usage
Alexey Bashtanov
 
PDF
Get to know PostgreSQL!
Oddbjørn Steffensen
 
PDF
Data Processing Inside PostgreSQL
EDB
 
PDF
Really Big Elephants: PostgreSQL DW
PostgreSQL Experts, Inc.
 
PDF
Howdah - An Application using Pylons, PostgreSQL, Simpycity and Exceptable
Command Prompt., Inc
 
PDF
PostgreSQL Deep Internal
EXEM
 
ODP
PostgreSQL Replication in 10 Minutes - SCALE
PostgreSQL Experts, Inc.
 
PDF
Full Text Search In PostgreSQL
Karwin Software Solutions LLC
 
PPTX
Cours SIG
Riadh Tebourbi (Ai1990)
 
PDF
alphorm.com - Formation PostgreSQL administration
Alphorm
 
PDF
A couple of things about PostgreSQL...
Federico Campoli
 
PDF
Postgresql database administration volume 1
Federico Campoli
 
PDF
5 Steps to PostgreSQL Performance
Command Prompt., Inc
 
Building a Spatial Database in PostgreSQL
Kudos S.A.S
 
PostgreSQL Scaling And Failover
John Paulett
 
My experience with embedding PostgreSQL
Jignesh Shah
 
Why use PostgreSQL?
Gabriele Bartolini
 
Android & PostgreSQL
Mark Wong
 
PostgreSQL Hooks for Fun and Profit
David Fetter
 
PostgreSQL and RAM usage
Alexey Bashtanov
 
Get to know PostgreSQL!
Oddbjørn Steffensen
 
Data Processing Inside PostgreSQL
EDB
 
Really Big Elephants: PostgreSQL DW
PostgreSQL Experts, Inc.
 
Howdah - An Application using Pylons, PostgreSQL, Simpycity and Exceptable
Command Prompt., Inc
 
PostgreSQL Deep Internal
EXEM
 
PostgreSQL Replication in 10 Minutes - SCALE
PostgreSQL Experts, Inc.
 
Full Text Search In PostgreSQL
Karwin Software Solutions LLC
 
Cours SIG
Riadh Tebourbi (Ai1990)
 
alphorm.com - Formation PostgreSQL administration
Alphorm
 
A couple of things about PostgreSQL...
Federico Campoli
 
Postgresql database administration volume 1
Federico Campoli
 
5 Steps to PostgreSQL Performance
Command Prompt., Inc
 
Ad

Similar to Pro PostgreSQL, OSCon 2008 (20)

PPT
Planning for-high-performance-web-application
Nguyễn Duy Nhân
 
PPT
Building tungsten-clusters-with-postgre sql-hot-standby-and-streaming-replica...
Command Prompt., Inc
 
PDF
Linux Desktop Automation
Rui Lapa
 
PDF
Creating customized openSUSE versions with SUSE Studio
elliando dias
 
ODP
Nagios Conference 2012 - Dan Wittenberg - Case Study: Scaling Nagios Core at ...
Nagios
 
PDF
The Accidental DBA
PostgreSQL Experts, Inc.
 
ODP
Extending OpenShift Origin: Build Your Own Cartridge with Bill DeCoste of Red...
OpenShift Origin
 
PPT
Planning For High Performance Web Application
Yue Tian
 
PDF
PGConf.ASIA 2019 Bali - AppOS: PostgreSQL Extension for Scalable File I/O - K...
Equnix Business Solutions
 
PDF
Out of the Box Replication in Postgres 9.4(pgconfsf)
Denish Patel
 
PDF
configuring a warm standby, the easy way
Command Prompt., Inc
 
PDF
Out of the Box Replication in Postgres 9.4(PgCon)
Denish Patel
 
PDF
Out of the Box Replication in Postgres 9.4(PgCon)
Denish Patel
 
PDF
Linux Common Command
Jeff Yang
 
PDF
Hadoop Summit San Jose 2013: Compression Options in Hadoop - A Tale of Tradeo...
Sumeet Singh
 
PDF
Bundling Packages and Deploying Applications with RPM
Alexander Shopov
 
PDF
2009-03-13 Atlanda System z Council Meeting
Shawn Wells
 
PDF
DrupalCampLA 2011: Drupal backend-performance
Ashok Modi
 
PDF
Cómo se diseña una base de datos que pueda ingerir más de cuatro millones de ...
javier ramirez
 
PDF
An Introduction To Linux
Ishan A B Ambanwela
 
Planning for-high-performance-web-application
Nguyễn Duy Nhân
 
Building tungsten-clusters-with-postgre sql-hot-standby-and-streaming-replica...
Command Prompt., Inc
 
Linux Desktop Automation
Rui Lapa
 
Creating customized openSUSE versions with SUSE Studio
elliando dias
 
Nagios Conference 2012 - Dan Wittenberg - Case Study: Scaling Nagios Core at ...
Nagios
 
The Accidental DBA
PostgreSQL Experts, Inc.
 
Extending OpenShift Origin: Build Your Own Cartridge with Bill DeCoste of Red...
OpenShift Origin
 
Planning For High Performance Web Application
Yue Tian
 
PGConf.ASIA 2019 Bali - AppOS: PostgreSQL Extension for Scalable File I/O - K...
Equnix Business Solutions
 
Out of the Box Replication in Postgres 9.4(pgconfsf)
Denish Patel
 
configuring a warm standby, the easy way
Command Prompt., Inc
 
Out of the Box Replication in Postgres 9.4(PgCon)
Denish Patel
 
Out of the Box Replication in Postgres 9.4(PgCon)
Denish Patel
 
Linux Common Command
Jeff Yang
 
Hadoop Summit San Jose 2013: Compression Options in Hadoop - A Tale of Tradeo...
Sumeet Singh
 
Bundling Packages and Deploying Applications with RPM
Alexander Shopov
 
2009-03-13 Atlanda System z Council Meeting
Shawn Wells
 
DrupalCampLA 2011: Drupal backend-performance
Ashok Modi
 
Cómo se diseña una base de datos que pueda ingerir más de cuatro millones de ...
javier ramirez
 
An Introduction To Linux
Ishan A B Ambanwela
 

More from Robert Treat (20)

PDF
Advanced Int->Bigint Conversions
Robert Treat
 
PDF
Explaining Explain
Robert Treat
 
PDF
the-lost-art-of-plpgsql
Robert Treat
 
PDF
Managing Chaos In Production: Testing vs Monitoring
Robert Treat
 
PDF
Managing Databases In A DevOps Environment 2016
Robert Treat
 
PDF
Less Alarming Alerts - SRECon 2016
Robert Treat
 
PDF
What Ops Can Learn From Design
Robert Treat
 
PDF
Postgres 9.4 First Look
Robert Treat
 
PDF
Less Alarming Alerts!
Robert Treat
 
PDF
Past, Present, and Pachyderm - All Things Open - 2013
Robert Treat
 
PDF
Big Bad "Upgraded" Postgres
Robert Treat
 
PDF
Managing Databases In A DevOps Environment
Robert Treat
 
PDF
The Essential PostgreSQL.conf
Robert Treat
 
PDF
Pro Postgres 9
Robert Treat
 
PDF
Advanced WAL File Management With OmniPITR
Robert Treat
 
PDF
Scaling with Postgres (Highload++ 2010)
Robert Treat
 
PDF
Intro to Postgres 9 Tutorial
Robert Treat
 
PDF
Check Please!
Robert Treat
 
PDF
Database Scalability Patterns
Robert Treat
 
PDF
A Guide To PostgreSQL 9.0
Robert Treat
 
Advanced Int->Bigint Conversions
Robert Treat
 
Explaining Explain
Robert Treat
 
the-lost-art-of-plpgsql
Robert Treat
 
Managing Chaos In Production: Testing vs Monitoring
Robert Treat
 
Managing Databases In A DevOps Environment 2016
Robert Treat
 
Less Alarming Alerts - SRECon 2016
Robert Treat
 
What Ops Can Learn From Design
Robert Treat
 
Postgres 9.4 First Look
Robert Treat
 
Less Alarming Alerts!
Robert Treat
 
Past, Present, and Pachyderm - All Things Open - 2013
Robert Treat
 
Big Bad "Upgraded" Postgres
Robert Treat
 
Managing Databases In A DevOps Environment
Robert Treat
 
The Essential PostgreSQL.conf
Robert Treat
 
Pro Postgres 9
Robert Treat
 
Advanced WAL File Management With OmniPITR
Robert Treat
 
Scaling with Postgres (Highload++ 2010)
Robert Treat
 
Intro to Postgres 9 Tutorial
Robert Treat
 
Check Please!
Robert Treat
 
Database Scalability Patterns
Robert Treat
 
A Guide To PostgreSQL 9.0
Robert Treat
 

Recently uploaded (20)

PDF
Architecting across the Boundaries of two Complex Domains - Healthcare & Tech...
Peter Tan
 
PPTX
VMware vSphere Foundation How to Sell Presentation-Ver1.4-2-14-2024.pptx
blackmambaettijean
 
PDF
Dropbox Q2 2025 Financial Results & Investor Presentation
Dropbox
 
PDF
cuic standard and advanced reporting.pdf
SimoneTitaniumTomase
 
PPTX
Cloud computing and distributed systems.
kkkkkhan
 
PDF
Mobile App Security Testing_ A Comprehensive Guide.pdf
flufftailshop
 
PDF
Blue Purple Modern Animated Computer Science Presentation.pdf.pdf
Akar16
 
PDF
Encapsulation_ Review paper, used for researhc scholars
gurumoop
 
PDF
Review of recent advances in non-invasive hemoglobin estimation
IAESIJAI
 
PPTX
KOM of Painting work and Equipment Insulation REV00 update 25-dec.pptx
muhammadmuneebnaeem7
 
PDF
Build a system with the filesystem maintained by OSTree @ COSCUP 2025
Jian-Hong Pan
 
PPTX
20250228 LYD VKU AI Blended-Learning.pptx
DatVoNgoc
 
PPT
“AI and Expert System Decision Support & Business Intelligence Systems”
ZubinRadhakrishnan
 
PDF
TokAI - TikTok AI Agent : The First AI Application That Analyzes 10,000+ Vira...
SOFTTECHHUB
 
PDF
Reach Out and Touch Someone: Haptics and Empathic Computing
Mark Billinghurst
 
PPTX
Effective Security Operations Center (SOC) A Modern, Strategic, and Threat-In...
ReZa AdineH
 
PDF
Empathic Computing: Creating Shared Understanding
Mark Billinghurst
 
PDF
Encapsulation theory and applications.pdf
gurumoop
 
PDF
Advanced methodologies resolving dimensionality complications for autism neur...
IAESIJAI
 
PPT
Teaching material agriculture food technology
LiaRayya
 
Architecting across the Boundaries of two Complex Domains - Healthcare & Tech...
Peter Tan
 
VMware vSphere Foundation How to Sell Presentation-Ver1.4-2-14-2024.pptx
blackmambaettijean
 
Dropbox Q2 2025 Financial Results & Investor Presentation
Dropbox
 
cuic standard and advanced reporting.pdf
SimoneTitaniumTomase
 
Cloud computing and distributed systems.
kkkkkhan
 
Mobile App Security Testing_ A Comprehensive Guide.pdf
flufftailshop
 
Blue Purple Modern Animated Computer Science Presentation.pdf.pdf
Akar16
 
Encapsulation_ Review paper, used for researhc scholars
gurumoop
 
Review of recent advances in non-invasive hemoglobin estimation
IAESIJAI
 
KOM of Painting work and Equipment Insulation REV00 update 25-dec.pptx
muhammadmuneebnaeem7
 
Build a system with the filesystem maintained by OSTree @ COSCUP 2025
Jian-Hong Pan
 
20250228 LYD VKU AI Blended-Learning.pptx
DatVoNgoc
 
“AI and Expert System Decision Support & Business Intelligence Systems”
ZubinRadhakrishnan
 
TokAI - TikTok AI Agent : The First AI Application That Analyzes 10,000+ Vira...
SOFTTECHHUB
 
Reach Out and Touch Someone: Haptics and Empathic Computing
Mark Billinghurst
 
Effective Security Operations Center (SOC) A Modern, Strategic, and Threat-In...
ReZa AdineH
 
Empathic Computing: Creating Shared Understanding
Mark Billinghurst
 
Encapsulation theory and applications.pdf
gurumoop
 
Advanced methodologies resolving dimensionality complications for autism neur...
IAESIJAI
 
Teaching material agriculture food technology
LiaRayya
 

Pro PostgreSQL, OSCon 2008

  • 1. Pro PostgreSQL Robert Treat omniti.com brighterlamp.org
  • 2. Who Am I? (Why Listen To Me) O-0 PostgreSQL User Since 6.5.x DBA of High Traffic / Large PostgreSQL Instances Long Time Contributor to PostgreSQL Project Contribute / Maintain Several Open Source Projects Co-Author Beginning PHP & PostgreSQL 8 (Apress)
  • 3. Outline O-1 What you need to know about the project Getting started Upgrading Configuring your server Hardware Availability Scalability Query tuning Tablespaces Partitioning Stuff you should know about
  • 4. K-0 Know Your Way Around The Project
  • 5. Know Your Way Around The Project K-1 www.postgresql.org downloads documentation bug reports security alerts wiki support companies rss –> news – events - versions
  • 6. Know Your Way Around The Project K-2 www.pgfoundry.org projects.postgresql.org Modules Programs Resources URI Type CIText SkyTools Npgsql Pl/Proxy pg_bulkload plpgsql-debugger sample databases
  • 7. Know Your Way Around The Project K-3 www.planetpostgresql.org Project News Community News Helpful Tips / Examples
  • 8. Know Your Way Around The Project K-4 archives.postgresql.org mailing list archives back to 1997 full text search via postgtresql 8.3 keyword search suggestions lists for users, developers, regional, user groups
  • 9. Know Your Way Around The Project K-5 #postgresql irc.freenode.net real time help rtfm_please - ??help
  • 10. Know Your Way Around The Project K-6 project management core team committers -hackers roadmap web team
  • 11. S-0 Get Off To A Good Start
  • 12. S-1 Get Off To A Good Start Use package management Consistent Standardized Simple
  • 13. Different across systems Upgrades are an issue Trust your packager? S-2 Get Off To A Good Start Use package management
  • 14. Different across systems Upgrades are an issue Trust your packager? S-2 Get Off To A Good Start Use package management Don't Be Afraid To Roll Your Own
  • 15. S-4 Get Off To A Good Start $PGDATA/pg_log /var/log/pgsql when in doubt... (postgresql.conf) separate disk Configure Logging Logging is often overlooked, but is the first step toward troubleshooting!
  • 16. S-5 Get Off To A Good Start most systems have different defaults firewalls/ selinux (FATAL) rtfm (pg_hba.conf, grant, revoke) Configure Authentication
  • 17. S-6 Get Off To A Good Start TRUST md5 IDENT Authentication Methods
  • 18. S-7 Get Off To A Good Start trust these more than your own code package dependent use different schemas (when able) pgcrypto pgstatstuple, pg_buffercache, pg_freespacemap /contrib
  • 19. S-8 Get Off To A Good Start package dependent some are non-core (plruby, plr, plphp) varying functionality varying levels of trust don't be afraid, test! procedural languages
  • 20. U-0 Let's Talk About Upgrades
  • 21. U-1 Let's Talk About Upgrades Versioning First Digit ( 7 .4.16 -> 8 .2.0) Second Digit (8.2.4 -> 8.3.0) Third Digit (8.3.0 -> 8.3.1)
  • 22. U-2 Let's Talk About Upgrades Versioning First Digit (7.4.16 -> 8.2.0) Second Digit (8. 2 .4 -> 8. 3 .0) Third Digit (8.3.0 -> 8.3.1)
  • 23. U-3 Let's Talk About Upgrades Versioning First Digit (7.4.16 -> 8.2.0) Second Digit (8.2.4 -> 8.3.0) Third Digit (8.3. 0 -> 8.3. 1 )
  • 24. U-4 Let's Talk About Upgrades Achtung!! Make Backups! Read the Release Notes!
  • 25. U-5 Let's Talk About Upgrades pg_dump/pg_restore simple -Fc is your friend dump with new version of pg_dump pitfalls (time, hdd)
  • 26. U-6 Let's Talk About Upgrades the slony method not simple create slave on new version switchover (switch back?) pitfalls (initial synch, compatibility)
  • 27. U-7 Let's Talk About Upgrades pg_migrator in place upgrades rewrites system catalog info no way to go back (fs snapshots) still new, under-flux 8.1 -> 8.2 only (for now)
  • 28. U-8 Let's Talk About Upgrades upgrading older db <= 7.3 is no longer supported (upgrade now!) pg_dump 8.2 has issues with <= 7.2 you can upgrade to 7.3 first use adddepends on 7.3 install slony requires 7.3 (or 7.4) (or 8.3) or newer pg_migrator (lol)
  • 29. C-0 Figure Your Configure
  • 30. C-1 Figure Your Configure the basics : performance effective_cache_size shared_buffers default_statistics_target work_mem checkpoint_segments
  • 31. C-2 Figure Your Configure the basics : logging stderr/pg_log vs. syslog/eventlog log_min_error_statement (error!) log_min_duration_statement log_line_prefix (%d, %p, %t)
  • 32. C-3 Figure Your Configure other stuff worth looking at maintenance_work_mem max_prepared_transactions update_process_title max_fsm_pages synchronous_commit
  • 34. M-1 Routine Maintenance a word about vacuum reclaim usable space update table stats avoid xid wraparound
  • 35. M-2 Routine Maintenance autovacuum : just do it! autovacuum track_activities track_counts autovacuum_max_freeze_age pg_autovacuum ?
  • 36. M-3 Routine Maintenance other stuff worth looking at reindexing logfiles backups failover
  • 37. P-0 Hardware For Software People
  • 38. P-1 Hardware For Software People you can be cheap on the licensing, don't be cheap on the hardware Cram the (ecc) RAM Redundant Disks (Software|Hardware) RAID Prefer SCSI, Accept SATA RAID Z (JBOD)
  • 39. P-2 Hardware For Software People cpu and you Multiple CPU's work wonders, up to 8 processors (16 OK) Opteron, Xeon, Core 2 Duo http://tweakers.net/
  • 40. P-3 Hardware For Software People disk do's Put WAL on it's own disk (RAID 1) Put DATA directory on it's own disk (RAID 10) More Spindles is Good More Controllers Even Gooder (tablespaces!) Battery Backed Controllers, Write cache enabled
  • 41. P-4 Hardware For Software People disk dont's NFS == Not Fully Safe! RAID 5 Beware disks / controllers that Lie
  • 42. A-0 Availability (Not Scalability)
  • 43. A-1 Availability what do we mean by availability? not backups (exactly) not replication (necessarily) not clustering (even less so)
  • 44. A-2 Availability what do we mean by availability? if (kablooy) then (ok) not backups (exactly) not replication (necessarily) not clustering (even less so)
  • 45. A-3 Availability pg_dump traditionally used for backups send dump to another server constantly run restore process large time, i/o constraints
  • 46. A-4 Availability filesystem snapshots zfs, lvm built on top of pitr backup commands traditionally used for backups copy snapshot to another server cumbersome time, i/o constraints
  • 47. A-5 Availability pitr create second, standby server ship wal logs to new server less time/io than pg_dump 8.1 -> cold standby 8.2 -> warm standby 8.4 -> hot standby ?
  • 48. A-6 Availability slony asynchronous, master-slave replication controlled switchover, failover low i/o, time constraints other benefits (upgrades, scaling)
  • 49. A-7 Availability bucardo asynchronous, multi-master replication also does master-slave low i/o, time constraints other benefits (upgrades, scaling)
  • 50. A-8 Availability shared disk one copy of PGDATA on shared storage standby takes over akin to db crash shared disk is point of failure (raid) STONITH
  • 51. A-9 Availability filesystem replication drbd filesystem mirrored between servers synchronized, ordered writes single disk system?
  • 52. A-10 Availability pgpool dual-master, statement based little caveats (random(),now(),sequences) bigger caveats (security, password, pg_hba) pgpool becomes failure point
  • 53. A-11 Availability postgres-r multi-master, synchronous just open sourced this month! small community not proven
  • 55. H-1 Scalability what is scaling? “ How well a solution to some problem will work when the relative size of the problem increases” - Theo Schlossnagle
  • 56. H-2 Scalability bigger, better, faster, more! postgresql scales up pretty well more disks (tablespaces) more cpu's, more ram connection pooling 1000+ connections, TB+ data
  • 57. H-3 Scalability pgpool dual-master, statement based little caveats (random(),now(),sequences) bigger caveats (security, password, pg_hba) pgpool becomes failure point
  • 58. H-4 Scalability pg_bouncer simple connection pooler 10/1 -> 40/1 caveats (prepared statements, temp tables) skype, myyearbook.com
  • 59. H-5 Scalability slony asynchronous, master-slave replication multiple, cascading slaves scales read operations other benefits (upgrades, scaling) solid user base
  • 60. H-6 Scalability bucardo asynchronous, multi-master replication also does master-slave low i/o, time constraints other benefits (upgrades, scaling)
  • 61. H-7 Scalability pgpool-II single db over multiple machines scales read operations replication, load balance, parallel query green technology
  • 62. H-8 Scalability pgcluster synchronous multi-master replication significant complexity scales read operations other uses (failover abilities) green technology
  • 63. H-9 Scalability postgres-r multi-master, synchronous just open source this month! small community other uses (failover abilities) not proven
  • 64. H-10 Scalability pitr read-only slaves based on pitr, warm standby operation core team officially supporting development 8.4 -> synchronous wal shipping 8.? -> read only slaves
  • 65. J-0 Query Your Queries
  • 66. J-1 Query Your Queries finding slow queries: log_min_duration_statement -1, 0 , n superuser only alter user LOG: duration: 5005.273 ms statement: select pg_sleep(5);
  • 67. J-2 Query Your Queries finding slow queries: pgfouine / pqa log analyzers command line, generate reports i/o load http://pgfouine.projects.postgresql.org/reports.html http://pqa.projects.postgresql.org/example.html
  • 68. J-3 Query Your Queries finding slow queries: pg_stat_all_tables pagila=# \d pg_stat_all_tables View &quot;pg_catalog.pg_stat_all_tables&quot; Column | Type | ------------------+-------------+ relid | oid | schemaname | name | relname | name | seq_scan | bigint | seq_tup_read | bigint | idx_scan | bigint | idx_tup_fetch | bigint | n_tup_ins | bigint | n_tup_upd | bigint | n_tup_del | bigint | n_tup_hot_upd | bigint | n_live_tup | bigint | n_dead_tup | bigint | last_vacuum | timestamptz | last_autovacuum | timestamptz | last_analyze | timestamptz | last_autoanalyze | timestamptz |
  • 69. J-4 Query Your Queries finding slow queries: pg_stat_all_tables pagila=# \d pg_stat_all_tables View &quot;pg_catalog.pg_stat_all_tables&quot; Column | Type | ------------------+-------------+ relid | oid | schemaname | name | relname | name | seq_scan | bigint | seq_tup_read | bigint | idx_scan | bigint | idx_tup_fetch | bigint | n_tup_ins | bigint | n_tup_upd | bigint | n_tup_del | bigint | n_tup_hot_upd | bigint | n_live_tup | bigint | n_dead_tup | bigint | last_vacuum | timestamptz | last_autovacuum | timestamptz | last_analyze | timestamptz | last_autoanalyze | timestamptz |
  • 70. J-5 Query Your Queries finding slow queries: pg_stat_all_tables pagila=# \d pg_stat_all_tables View &quot;pg_catalog.pg_stat_all_tables&quot; Column | Type | ------------------+-------------+ relid | oid | schemaname | name | relname | name | seq_scan | bigint | seq_tup_read | bigint | idx_scan | bigint | idx_tup_fetch | bigint | n_tup_ins | bigint | n_tup_upd | bigint | n_tup_del | bigint | n_tup_hot_upd | bigint | n_live_tup | bigint | n_dead_tup | bigint | last_vacuum | timestamptz | last_autovacuum | timestamptz | last_analyze | timestamptz | last_autoanalyze | timestamptz |
  • 71. J-6 Query Your Queries finding slow queries: pg_stat_all_tables pagila=# \d pg_stat_all_tables View &quot;pg_catalog.pg_stat_all_tables&quot; Column | Type | ------------------+-------------+ relid | oid | schemaname | name | relname | name | seq_scan | bigint | seq_tup_read | bigint | idx_scan | bigint | idx_tup_fetch | bigint | n_tup_ins | bigint | n_tup_upd | bigint | n_tup_del | bigint | n_tup_hot_upd | bigint | n_live_tup | bigint | n_dead_tup | bigint | last_vacuum | timestamptz | last_autovacuum | timestamptz | last_analyze | timestamptz | last_autoanalyze | timestamptz |
  • 72. J-7 Query Your Queries finding slow queries: pg_stat_all_tables pagila=# \d pg_stat_all_tables View &quot;pg_catalog.pg_stat_all_tables&quot; Column | Type | ------------------+-------------+ relid | oid | schemaname | name | relname | name | seq_scan | bigint | seq_tup_read | bigint | idx_scan | bigint | idx_tup_fetch | bigint | n_tup_ins | bigint | n_tup_upd | bigint | n_tup_del | bigint | n_tup_hot_upd | bigint | n_live_tup | bigint | n_dead_tup | bigint | last_vacuum | timestamptz | last_autovacuum | timestamptz | last_analyze | timestamptz | last_autoanalyze | timestamptz |
  • 73. J-8 Query Your Queries finding slow queries: pg_stat_all_tables pagila=# \d pg_stat_all_tables View &quot;pg_catalog.pg_stat_all_tables&quot; Column | Type | ------------------+-------------+ relid | oid | schemaname | name | relname | name | seq_scan | bigint | seq_tup_read | bigint | idx_scan | bigint | idx_tup_fetch | bigint | n_tup_ins | bigint | n_tup_upd | bigint | n_tup_del | bigint | n_tup_hot_upd | bigint | n_live_tup | bigint | n_dead_tup | bigint | last_vacuum | timestamptz | last_autovacuum | timestamptz | last_analyze | timestamptz | last_autoanalyze | timestamptz |
  • 74. J-9 Query Your Queries finding slow queries: pg_stat_all_indexes pagila=# \d pg_stat_all_indexes View &quot;pg_catalog.pg_stat_all_indexes&quot; Column | Type | ---------------+--------+ relid | oid | indexrelid | oid | schemaname | name | relname | name | indexrelname | name | idx_scan | bigint | idx_tup_read | bigint | idx_tup_fetch | bigint |
  • 75. J-10 Query Your Queries finding slow queries: pg_stat_all_indexes pagila=# \d pg_stat_all_indexes View &quot;pg_catalog.pg_stat_all_indexes&quot; Column | Type | ---------------+--------+ relid | oid | indexrelid | oid | schemaname | name | relname | name | indexrelname | name | idx_scan | bigint | idx_tup_read | bigint | idx_tup_fetch | bigint |
  • 76. J-11 Query Your Queries finding slow queries: pg_statio_all_tables pagila=# \d pg_statio_all_tables View &quot;pg_catalog.pg_statio_all_tables&quot; Column | Type | -----------------+--------+ relid | oid | schemaname | name | relname | name | heap_blks_read | bigint | heap_blks_hit | bigint | idx_blks_read | bigint | idx_blks_hit | bigint | toast_blks_read | bigint | toast_blks_hit | bigint | tidx_blks_read | bigint | tidx_blks_hit | bigint |
  • 77. J-12 Query Your Queries finding slow queries: pg_statio_all_tables pagila=# \d pg_statio_all_tables View &quot;pg_catalog.pg_statio_all_tables&quot; Column | Type | -----------------+--------+ relid | oid | schemaname | name | relname | name | heap_blks_read | bigint | heap_blks_hit | bigint | idx_blks_read | bigint | idx_blks_hit | bigint | toast_blks_read | bigint | toast_blks_hit | bigint | tidx_blks_read | bigint | tidx_blks_hit | bigint |
  • 78. J-13 Query Your Queries fixing slow queries: explain analyze universal tool good for specific queries “ explain” for large queries could be it's own talk
  • 79. J-14 Query Your Queries fixing slow queries: explain analyze universal tool good for specific queries “ explain” for large queries could be it's own talk http://wiki.postgresql.org/Using_EXPLAIN
  • 80. J-15 Query Your Queries fixing slow queries: indexing (basic) use explain to find large sequential reads use pg_stat_* tables to find numerous reads btree – (gist/gin) enable_indexscan, enable_bitmapscan dual column vs. single column
  • 81. J-16 Query Your Queries fixing slow queries: indexing (partial) create index address_ba_part_idx on address (district) where district = 'Buenos Aires'; restrain index to rows that matter can give significant speed improvements where clause of index should match where clause of query
  • 82. J-17 Query Your Queries fixing slow queries: indexing (partial) create index customer_active_part_idx on customer (customer_id) where activebool is true; restrain index to rows that matter can give significant speed improvements where clause of index should match where clause of query
  • 83. J-18 Query Your Queries fixing slow queries: indexing (functional) some people prefer to call these expressional indexes
  • 84. J-19 Query Your Queries fixing slow queries: indexing (expressional) create unique index one_true_email_xidx on customer (lower(email)); push expensive functions into your index system sees just WHERE indexedcolumn = 'constant' expression of index should match expression of queries narrow scope, but nice gains
  • 85. J-20 Query Your Queries fixing slow queries: indexing (expressional) create index fullname_xidx on customer ((first_name||' '||last_name)); push expensive functions into your index system sees just WHERE indexedcolumn = 'constant' expression of index should match expression of queries narrow scope, but nice gains
  • 86. J-21 Query Your Queries fixing slow queries: full text search uses lexmes and word stemming to find common words replacement for LIKE '%x%', ~* 'x'; supports multiple languages, custom dictionaries special indexing options
  • 87. J-22 Indexing Options full text indexing gist vs. gin “ old school” slower for queries faster insert / update mature new in 8.2 faster for queries slower insert / update stable
  • 89. N-1 PostgreSQL Tablespaces tablespaces? define logical locations for object placement point to locations on disk (uses symlinks) size determined by disk size (not pre-ordained) dedicate per db, split db across multiple tblspc
  • 90. N-2 PostgreSQL Tablespaces tablespaces! split database over separate disks use stat, statio tables to gauge disk access create dedicated storage for workloads disk for read / write disk for read only large, slow disk for archiving disk for indexes
  • 92. Q-1 PostgreSQL Partitioning partitioning? as table size grows, it becomes unmanageable use inheritance, rules, constraints to split data queries ignore non-relevant partitions could be it's own talk
  • 93. Q-2 PostgreSQL Partitioning partitioning! as table size grows, it becomes unmanageable use inheritance, rules, constraints to split data queries ignore non-relevant partitions could be it's own talk http://www.pgcon.org/2007/schedule/events/41.en.html
  • 94. Q-3 PostgreSQL Partitioning partitioning : key points determine list vs. range use triggers rather than rules partition creation vs. data population automate maintenance
  • 95. I-0 Other Stuff I Should Mention
  • 96. I-1 Other Stuff I Should Mention pgcrypto cryptography type functions /contrib (export issues) md5, sha1, blowfish, many more
  • 97. I-2 Other Stuff I Should Mention dblink pg -> pg connections /contrib (still under development?) can have performance issues on large queries make it live in it's own schema
  • 98. I-3 Other Stuff I Should Mention *-link heterogenous connections for postgresql db specific and db independent options any pl/u language can implement this similar performance issues to dblink dblink-tds, dbi-link, oralink, odbclink http://www.pgfoundry.org/ (db link)
  • 99. I-4 Other Stuff I Should Mention autonomous logging tool persistent logging for postgresql functions built on top of dblink make it live in it's own schema https://labs.omniti.com/trac/pgsoltools
  • 100. I-5 Other Stuff I Should Mention snapshot pitr clones full read/write copy of pitr slave static snapshot need solaris (zfs zone mojo) could re-implement on other systems https://labs.omniti.com/trac/pgsoltools
  • 101. I-6 Other Stuff I Should Mention check_postgres nagios based monitoring script common items for warnings and alerts can be adapted to other uses http://bucardo.org/check_postgres
  • 102. I-7 Other Stuff I Should Mention reconnoiter monitoring / graphing tool postgres based still pretty green https://labs.omniti.com/trac/reconnoiter
  • 103. I-8 Other Stuff I Should Mention phpPgAdmin web based gui for postgresql remote administration of multiple servers implements much of postgresql functionality support back to 7.2? http://phppgadmin.sourceforge.net/
  • 104. I-9 Other Stuff I Should Mention ;-) my book?
  • 105. I-10 Other Stuff I Should Mention ;-) we're hiring Ops Ninjas Perl Kung-Fu Artists PHP Ninjas Database Samurai http://omniti.com/is/hiring