Trivadis TechEvent 2017 PostgreSQL für die (Orakel) DBA by Ludovico Caldara

BÂLE BERNE BRUGG DUSSELDORF FRANCFORT S.M. FRIBOURG E.BR. GENÈVE
HAMBOURG COPENHAGUE LAUSANNE MUNICH STUTTGART VIENNE ZURICH
PostgreSQL
für die (Orakel) DBA
Ludovico Caldara
Oracle ACE Director
Trivadis AG

PostgreSQL2 26/10/2017
PostgreSQL in the
RDBMS Ecosystem

DB-Engines Ranking of RDBMS (Feb 2017)

Google trends

PostgreSQL awareness…
It is an Open-Source, advanced RDBMS
It supports modern SQL constructs
But … it is much less broaden than Oracle and MySQL
– So why we care about it?

PostgreSQL awareness… (cont.)
More and more customers are upset with Oracle
– High costs (http://www.oracle.com/us/corporate/pricing/technology-price-list-
070617.pdf)
– Lower support quality over the years
– Aggressive License auditing
– On-premises strategy (and roadmap) is not promising
– Unfriendly with 3rd party clouds (https://oracle-base.com/blog/2017/01/28/oracles-
cloud-licensing-change-be-warned/)

PostgreSQL awareness… (cont.)
PostgreSQL is the RDBMS most close to Oracle
– Feature set
– SQL Dialect
– Procedural Language
For a fraction of the price! (or completely free)
Many respected developers endorse it:
– Markus Winand
– Lukas Eder
– Felix Geisendörfer

SQL Standards adherence at a glance
MYSQL 5.7 PostgreSQL 9.6 SQLite DB2 11.1 Oracle 12.1 SQL Server 2016
SQL Standards
SQL:1999 LATERAL
SQL:1999 GROUPING SETS
SQL:1999 WITH
SQL:1999 WITH RECURSIVE
SQL:2003 FILTER
SQL:2003 OVER
SQL:2003 WITHIN GROUP
SQL:2003 TABLESAMPLE
SQL:2008 FETCH FIRST
SQL:2011 OFFSET
SQL:2011 OVER
SQL:2011 TEMPORAL TABLES
SQL:2016 ROW PATTERN MATHING
Source: https://www.slideshare.net/MarkusWinand/modern-sql

Support, License, Cost

EnterpriseDB Standard
https://www.enterprisedb.com/products/subscriptions
Standard PostgreSQL Subscription
– «Almost» vanilla version but pre-packaged
– EDB Postgres Enterprise Manager
– EDB Postgres Failover Manager
– EDB Postgres Replication Server
– EDB Postgres Migration toolkit
– Etc.
Price: 1225€ per core (or vcore) / year (offer 2016)

EnterpriseDB Advanced Server
https://www.enterprisedb.com/products/subscriptions
Commercial Subscription
– All Standard features plus:
– Enterprise Security (password profiles, PL Wrapping, session auditing)
– Enterprise Performance (Partition by, optimizer hints)
– Enterprise Development (see https://www.enterprisedb.com/products/compare-postgres-
databases#securityfeatures for the full list)
– Enterprise Management (Resource manager, extended catalog views)
– Compatibility with Oracle (EDB*OCI, PL/SQL support, PL/SQL debugger)
Price: 1750€ per core (or vcore) / year (offer 2016)

2ndQuadrant
2ndQuadrant maintains and supports additional tools
– Pglogical (replication)
– Postgres-BDR (bi-directional replication)
– Postgres-XL (parallel server scale-out)
– Barman (backup and recovery)
– Repmpr (replication failover)
– Other tools for analytics

Installation

Via Repositories (RPM, DEB)
Standard distribution repositories
– You may encounter unwanted upgrades if you upgrade everything
– Great if a sysadmin need to maintain a single installation
Official PostgreSQL YUM repository
– A single RPM per version/architecture will install the correct YUM repository
– Better control over what you install and when you upgrade
– yum install https://download.postgresql.org/pub/repos/yum/9.6/redhat/rhel-7-
x86_64/pgdg-redhat96-9.6-3.noarch.rpm
Limited to one installation per server/VM

From source code
Quite tedious (libraries dependencies, configure, make…)
Follow the instructions:
https://www.postgresql.org/docs/current/static/installation.html
Not suitable for production, requires deep testing
But great if you want to integrate patches or 3rd parties extentions or plugins

Tarballs
You can find tarballs here:
https://www.enterprisedb.com/download-postgresql-binaries
Great if you want multiple installations
– Use Oracle Flexible Architecture!
Easy: you can script your own installation using tarballs

Installers
PostgreSQL website points to the EnterpriseDB website for installers:
https://www.enterprisedb.com/downloads/postgres-postgresql-downloads
– They require root privileges: non-sense when customers have common security
rules
– Personally I use them as tarballs (no root privilege required)
sudo -u postgres /path/to/postgresql-9.6.2-1-linux-x64.run
--mode unattended
--extract-only yes
--prefix /u01/app/postgres/product/postgresql-9.6.2-1

Initialization
Tarballs and custom builds do not initialize the cluster
– Use this command to do it:
$ initdb --pgdata=$PGDATA --xlogdir=$PGXLOG --auth=md5
--data-checksums [other options]
Basic hardening and parameter configuration is still needed!
– Hardening: we will see it in another chapter. For now:
pg_hba.conf -> defines authentication schemes
pg_ident.conf -> defines user equivalency at OS level
– Parameters: next slides
In postgreSQL a
cluster is the
instance!

Cluster directory structure
Dir / File Content
$PGDATA/postmaster.pid Pid file
$PGDATA/postgresql.conf Config file
$PGDATA/pg_log/ Log files (like alert)
$PGDATA/base/<DB_ID>/<SEG_ID> Physical structures (tables, indexes)
$PGDATA/pg_xlog/ WAL logfiles
Full list of directories: https://www.postgresql.org/docs/current/static/storage-file-layout.html

Basic parameter configuration
$PGDATA/postgresql.conf -> just like init.ora
shared_buffers
– Kind of db_cache, but PostgreSQL uses heavily filesystem cache: different values!
listen_addresses = ‘*’
– To which address it binds
cluster_name
– New in postgresql 9.5, useful to match processes with instances

Basic parameter configuration (cont.)
Max_connections
Wal_*
– WAL files are the after images (redo/archive logs)
– configure them with the wal_* parameters
Archive_*
– WAL files needs to be archived somewhere
– Configure the archiving with the archive_* parameters

Important environment variables
PGPORT
– The port the cluster is listening to (must be unique for each cluster)
PGDATA
– The directory that contains all the cluster data

Operations: Stop/Start

Operations: start
pg_ctl is the main command to stop and start the cluster
pg_ctl start -w -o "-p $PGPORT" -D $PGDATA -l $PGOUTPUT
– -w: wait for the start to complete
– -o: options to be passed to the postgres command
– -D: path to the data (must have permissions 700)
– -l: path to the standard output (will have permissions 700)

Operations: stop
pg_ctl stop –m fast
– -m smart -> waits for disconnects and end of backups (like shutdown normal)
– -m fast -> rolls back the transactions (like shutdown immediate)
– -m immediate -> abort server processes and lead to crash-recovery (like
shutdown abort)
PGDATA must be correctly specified!

Architecture

Cluster, instance, database, what?
A cluster is an instance of postgreSQL containing many databases
– Conceptually similar to MySQL, SQL Server and Oracle Pluggable Databases
Host
Cluster pgclust1 (port 5432) Cluster pgclust2 (port 5433)
postgres template0 template1 postgres template0 template1
userdb1 userdb2 userdb3 userdb4 userdb5 userdb6

Default Databases
Postgres
– Is the «SYSTEM» or «master» database
– It contains system tables, views, procedures, metadata, definition of users and
roles, etc.
Template0
– Read-only, empty database used for creating VERY EMPTY databases (like
PDS$SEED in Oracle)
Template1
– Read-write database, generally empty, might be modified and it is the default
template for new databases (like model in SQL Server)

User processes
(backend)
User processes
(backend)
Background
processes
Background
processes
Overall Architecture: client-server interaction
Server
Cluster
DISK
Client
Application
libpq
Background
processes
User processes
(backend)
1
2
3
Shared memory
Shared Buffers
WAL Sort
OS
Cache
CLOG Other

Overall Architecture: background processes
Server
Cluster instance
DISK
User processes
(backend)
User processes
(backend)
User processes
(backend)
OS
Cache
Shared memory
Shared
Buffers
Sort WAL
CLOG Other
Postgres
daemon *
WAL writer
* Once called «postmaster process»
writer
Check
pointerlogger
archiver
Stats
collector
Autovacuum
launcher
WAL
sender
WAL
receiver
data
WAL
Archived
WAL

Mandatory background processes
Postgres (A.K.A. postmaster)
– Gets connection requests, authenticates them and spawns backend processes
Checkpointer
– Writes dirty pages to disk and registers the checkpoint
Writer
– The «Lazy writer» that writes pages to disk in small increments
WAL Writer
– Flushes the WAL buffer to disk (in WAL files) at every commit

Optional background processes
Logger
– Writes to the «alert log» or whatevrt log is configured
Autovacuum Launcher Process
– Automates VACUUM and ANALYZE commands
Stats Collector
– Collects information about the cluster activity
WAL Sender and Receiver
– Respectively sends WAL to a standby and receives WAL from primary

After Image: Write Ahead Logs
WAL Buffer
Shared Buffers
WAL
File
DATA
File1
read
2
log
3
modify
4
commit
WAL
writer
writer
Check
pointer
5
Commited!
6
write

Before Image: more complicated!
Oracle has Rollback Segments (UNDO mechanism)
– Provides consistent reads while writes happen
– The block is overwritten and the old image is saved in the UNDO tablespace
SQL Server has NO rollback segments for read consistency
– Writes lock reads and viceversa
– Unless a special mechanism is implemented (ALLOW_SNAPSHOT_ISOLATION)
PostgreSQL has NO rollback segments
– But provides consistent reads while writes happen. How?
– The new image is written in a new location while the previous image is left intact

Before Image: explanation
Every time a row is updated, the new version is stored in a new location
Page
Case 1: same page
1,’bb’,test2
2,’cc’,test3
0,’aa’,test1’
Case 2: different page

Page
Case 1: same page
1,’bb’,test2
2,’cc’,test3
0,’aa’,test1’
UPDATE t SET col2=‘test4’ WHERE col0=1;
1,’bb’,test4NEW
OLD

Page
Case 1: same page
1,’bb’,test2
2,’cc’,test3
0,’aa’,test1’
1,’bb’,test4NEW
OLD
Page x
2,’cc’,test3
0,’aa’,test1’
1,’bb’,test2
3,’dd’,test4

Page
Case 1: same page
1,’bb’,test2
2,’cc’,test3
0,’aa’,test1’
1,’bb’,test4NEW
OLD
Page x
2,’cc’,test3
0,’aa’,test1’
Page y
1,’bb’,test2
3,’dd’,test4 1,’bb’,test4
OLD
NEW

Before Images, Indexes and Visibility Maps
Page 0
2,’cc’,test3
0,’aa’,test1’
1,’bb’,test2
3,’dd’,test4
HEAP
HEAP_VM
01
Bt Page 1
[0,4] 2,’cc’
[0,2] 0,’aa’’
[0,3] 1,’bb’
[0,1] 3,’dd’
INDEX ON (ID, COL1)

Page 0
2,’cc’,test3
0,’aa’,test1’
1,’bb’,test2
3,’dd’,test4
HEAP
HEAP_VM
01
Bt Page 1
[0,4] 2,’cc’
[0,2] 0,’aa’’
[0,3] 1,’bb’
[0,1] 3,’dd’
INDEX ON (ID, COL1)
INDEX ONLY SCANS

Page 0
2,’cc’,test3
0,’aa’,test1’
Page 1
1,’bb’,test2
OLD
NEW
HEAP
HEAP_VM
00 00
Bt Page 1
INDEX ON (ID, COL1)
[1,1] 1,’bb’
[0,4] 2,’cc’
[0,2] 0,’aa’’
[0,3] 1,’bb’
[0,1] 3,’dd’
NEW
OLD

Page 0
2,’cc’,test3
0,’aa’,test1’
Page 1
1,’bb’,test2
HEAP
HEAP_VM
00 00
Bt Page 1
INDEX ON (ID, COL1)
[1,1] 1,’bb’
[0,4] 2,’cc’
[0,2] 0,’aa’’
[0,3] 1,’bb’
[0,1] 3,’dd’
INDEX ONLY SCANS

Page 0
2,’cc’,test3
0,’aa’,test1’
Page 1
1,’bb’,test2
OLD
NEW
HEAP
HEAP_VM
01 01
Bt Page 1
INDEX ON (ID, COL1)
[1,1] 1,’bb’
[0,4] 2,’cc’
[0,2] 0,’aa’’
[0,3] 1,’bb’
[0,1] 3,’dd’
NEW
OLD
VACUUM

Pros and Cons of this implementation
Pros (compared to Oracle)
– No need for Rollback Segments, no ORA-01555 for rollback segments too small 
Cons
– High modification == High Fragmentation! VACUUM required
– When a tuple is modified AND changes of page, its address changes!
• Need to update ALL the indexes that point to it, regardless if the modified column
is indexed
• This is the famous «write amplification» problem in PostgreSQL
• The table fillfactor (equivalent of pctused) mitigates the problem
• Vacuum becomes critical for performance

VACUUM
Must be performed regularly in order to
– Reclaim space occupied by old tuple images
– Update data statistics used by the query planner
– Update the visibility map
– Reset the transaction ID of old blocks to prevent wraparound
A default VACUUM is executed regularly by default
– Many parameters help in the VACUUM fine-tuning
Manual VACUUMing is possible

Transaction ID (XID) wraparounds
OLDEST
ACTIVE
TRANSACTION
OLDEST
TRANSACTION ID
XID is 32 bit long: max 4bln XIDs
– 2bln (2^31) in the past
– 2bln (2^31) in the future
XIDs in the future are invisible
(per read consistency)
If an old Tuple wraps around, it
will become invisible: WRONG!
WRAPAROUND
POINT
NOW

Transaction ID (XID) wraparounds - solution
OLDEST
ACTIVE
TRANSACTION
OLDEST
TRANSACTION ID
The VACUUM stamps the tuples
as Frozen after they become
older than the freezing point
Frozen means that the tuples are
visible regardless of the
Transaction ID
Not elegant, but effective solution
VACUUM again plays an
important role
NOW
FREEZING
POINT

The Oversized-Attribute Storage Technique (TOAST)
Support for very large columns
lob_test=# create table t (id integer, toast varchar(1000000));
CREATE TABLE
lob_test=# insert into t values (1, rpad('x',1000000));
INSERT 0 1

Support for very large columns
lob_test=# create table t (id integer, toast varchar(1000000));
CREATE TABLE
lob_test=# insert into t values (1, rpad('x',1000000));
INSERT 0 1
lob_test=# select relfilenode, reltoastrelid
lob_test-# from pg_class where relname='t';
relfilenode | reltoastrelid
-------------+---------------
16853 | 16856
(1 row)
Each table with extended columns has a
dedicated TOAST segment

e.g.: 64k column

e.g.: 64k column
Split in 2048 bytes chunks

Individual chunk compression via lz_compress
e.g.: 64k column

Compressed chunks inserted as rows in the TOAST table
e.g.: 64k column

Compressed chunks inserted as rows in the TOAS table
e.g.: 64k column
Nice
replacement
for LOBs!!

Architectural Oddities #1 : Constraints enforced per row
The doc says constraints are enforced per statement
Actually, they are enforced PER ROW
This lead to unexpected behaviours depending on row placement and access *
pagila=# insert into demo values(2);
INSERT 0 1
INSERT 0 1
pagila=# update demo set n=n+1;
UPDATE 2
INSERT 0 1
INSERT 0 1
pagila=# update demo set n=n+1;
ERROR: duplicate key value violates unique
constraint "demo_pk"
DETAIL: Key (n)=(2) already exists.
pagila=# create table demo (n integer not null);
CREATE TABLE
pagila=# alter table demo add constraint demo_pk primary key(n);
ALTER TABLE
* Unless declared as deferrable

Architectural Oddities #2: Classic Large Objects
The Large Objects are not part of a table (even not logically)
They are just referenced by the columns
As result:
– Large Objects and Tables might have different owners (see next chapter)
– Large Objects might be referenced by multiple rows/tables
– Large Objects are not deleted when the rows are deleted
– Large Objects housekeeping is a routine task! (use vacuumlo binary for this)
Some insights about LOBs here:
– http://www.ludovicocaldara.net/dba/pgsql-lo-space-usage-part-3/

Architectural Oddities #3: Rollback on statement error
With Oracle, in the middle of a transaction an error will rollback just the last
statement
With PostgreSQL, in the middle of a transaction an error will rollback all the
previous statements
– Developers relying on this behavior must be careful
– Using save points might do the job
db=# BEGIN;
BEGIN
db=# INSERT INTO t VALUES (1);
INSERT 0 1
db=# SELECT 0/0;
ERROR: division by zero
db=# select * from t;
ERROR: current transaction is aborted,
commands ignored until end of transaction block
db=# commit;
ROLLBACK

Digging the internals
Open Source: RTFC
(Read That F* Code!)
The source code is complex
but rich of explanations
https://doxygen.postgresql.org/

Authentication and Authorization

Authentication
Authentication != Authorization
A user must authenticate to the server, the user must already exist!
– create role myuser with login;
PostgreSQL supports mixed authentication
Mixed authentication allows nice security features. E.g.
– Active Directory-Based authentication for individuals
– Password authentication for traditional application users
– Client certificate authentication for batches and secure applications

Authentication (cont.)
pge_hba.conf: the first line that matches wins. Real example:
# TYPE DATABASE USER ADDRESS METHOD
host all +ldap_users 0.0.0.0/0 ldap
ldapserver=ldap.trivadis.com ldapport=389 ldapprefix="uid="
ldapsuffix=",ou=people,dc=trivadis,dc=com"
hostssl all +sslcertusers 0.0.0.0/0 trust clientcert=1
local all postgres peer map=dba
host all all 0.0.0.0/0 md5
local
hostnossl
host
hostssl
all
replication
sameuser
samerole
dbname
all
user
+group
@incfile
all
samehost
samenet
IP range
trust
reject
md5
password
gss
sspi
ident
peer
ldap
radius
cert
pam

Authentication maps
With maps, the mapped users can connect with a different username.
– E.g. all the dba can connect locally as postgres:
# TYPE DATABASE USER METHOD
local all postgres peer map=dba
The file ident.conf contains the maps:
#map osuser dbuser
dba luc postgres
dba zam postgres
dba jko postgres
dba bir postgres
dba efu postgres

Authentication - inheritance
Oracle has USERS and ROLES.
PostgreSQL manages everything as a ROLE:
A role can inherit another role:
– A user is a role with login privilege
– A role can be granted to another role -> inheritance, grouping
– A role itself might be granted to login
– A user can set a role and «become» that role
– A schema is not a user or role but an object namespace
– It sounds complicated but it is not (once you get it)
«public» exists but it is actually a hidden role.

Creating a group and a user
postgres=# create role mygroup;
CREATE ROLE
postgres=# du mygroup
List of roles
Role name | Attributes | Member of
-----------+--------------+-----------
mygroup | Cannot login | {}
postgres=# create role myuser with login in group mygroup;
CREATE ROLE
postgres=# du myuser
List of roles
-----------+------------+-----------
myuser | | {mygroup}

A more complex example
# TYPE DATABASE USER ADDRESS METHOD
host all +ldap_users 0.0.0.0/0 ldap
postgres=# create role ldap_users; -- for the LDAP authentication
CREATE ROLE
postgres=# create role pagila with login; -- for the application
CREATE ROLE
postgres=# create role luc with login in group ldap_users, pagila;
CREATE ROLE
postgres=# c pagila
You are now connected to database "pagila" as user "postgres".
pagila=# grant all privileges on schema public to pagila;
GRANT
postgres=#

A more complex example (cont.)
List of roles
-------------+---------------------------+-------------------
ldap_users | Cannot login | {}
pagila | | {}
luc | | {pagila,ldap_users}
# psql –U luc –h $HOSTNAME –p $PGPORT pagila
Password for user luc:
pagila=> set role pagila;
SET
pagila=> select session_user, user;
session_user | current_user
--------------+--------------
luc | pagila
(1 row)

Schemas
ROLES
postgres
user1
user2
user3
user4
DATABASE
PUBLIC SCHEMA1 SCHEMA2
TABLE1
TABLE2
TABLE3 TABLE4
HAS
GRANT
/
OWNS

Schemas: Namespace search
Public schema exists by default
Non-quilified objects are created in the public schema
The default search_path is "$user", public
pagila=# CREATE TABLE t1 (a varchar(50));
CREATE TABLE
pagila=# dt t1
List of relations
Schema | Name | Type | Owner
--------+------+-------+----------
public | t1 | table | postgres
(1 row)
pagila=# select * from t1;
a
---
(0 rows)

Schemas: Namespace search (cont.)
Other schemas must be specified upon object creation:
pagila=# CREATE SCHEMA schema1;
CREATE SCHEMA
pagila =# CREATE TABLE schema1.t2 (a varchar(50));
CREATE TABLE
The schema must be specified to find the object, unless you modify the search space
pagila =# dt t2
No matching relations found.
pagila =# SET search_path=public,schema1;
SET
pagila =# dt t2
List of relations
---------+------+-------+----------
schema1 | t2 | table | postgres
(1 row)
OWNER != SCHEMA

Schema Privileges
Users own schemas or objects in their own or someone else’s schemas
Access to the objects is granted by respective owners
pagila=# GRANT CREATE ON SCHEMA public TO luc;
GRANT
… as luc
pagila=> CREATE TABLE ludovico_table (a varchar(50));
CREATE
pagila=> dt ludovico_table
List of relations
--------+----------------+-------+-------
public | ludovico_table | table | luc
(1 row)
pagila=> GRANT SELECT ON ludovico_table TO ...

Default privileges
The default permissions on a schema might be changed if necessary
ALTER DEFAULT PRIVILEGES
[ FOR { ROLE | USER } target_role [, ...] ]
[ IN SCHEMA schema_name [, ...] ]
GRANT
{ { SELECT | INSERT | UPDATE | DELETE | TRUNCATE | REFERENCES | TRIGGER } [, ...] |
ALL [ PRIVILEGES ] }
ON TABLES TO { [ GROUP ] role_name | PUBLIC } [, ...] [ WITH GRANT OPTION ]

Hardening tips
Believe it or not, public can write in the postgres public schema…
– postgres=# revoke all privileges on schema public from public;
The same applies to the template1… you might want to change it too
– template1=# revoke all privileges on schema public from public;

Audit
No embedded module for auditing
3rd parties module pgaudit exists and it the standard de-facto
– Recommended by US Department of Defense
– https://www.crunchydata.com/postgres-stig/PGSQL-STIG-9.5+.pdf
– shared_preload_libraries ='pgaudit‘
pgaudit.log_catalog='on‘
pgaudit.log_level='log‘
pgaudit.log_parameter='on‘
pgaudit.log_relation='off‘
pgaudit.log_statement_once='off‘
pgaudit.log='ddl, role, write'
– Supplemental work required to analyze data

Pgaudit installation
Download pgaudit-REL9_6_STABLE.zip
unzip pgaudit-REL9_6_STABLE.zip
cd pgaudit-REL9_6_STABLE
set PATH to PostgreSQL binary
make USE_PGXS=1 install
Modify PostgreSQL.conf (previous slide)
Restart the cluster

Operations: Backup/Recovery

Base Backup: it backs up the whole cluster!
With the whole cluster, all the databases are backed up
The WAL files can be applied on a base backup: PITR possible
$ pg_basebackup -D $BACKUP_DEST -Ft -z -l $Label -U postgres
$ ls –l $BACKUP_DEST
-rw-rw---- 1 postgres postgres 5703169 Jul 11 21:15 base.tar.gz
It creates a tar.gz with the necessary files
Fuzzy datafiles: they will need recovery

WAL Backups: WAL introduction
WAL files are a mix between online logs and archive logs
– They contains after images needed for recovery
– If archive mode is OFF, WAL files are recycled
– If archive mode is ON, they are not recycled but they need to be archived
– Archival is ran using the command specified in archive_command parameter
– Impossible to archive: WAL filesystem fills up -> databases hang!
WAL files size is 16MB (fixed, defined at compile time)
Archive_timeout -> similar to archive_lag_target

WAL Backups: parameters
wal_level = replica
archive_mode = on
archive_command = 'test ! -f
/u02/data/postgres/backup/pgts962i/archive/%f && cp %p
/u02/data/postgres/backup/pgts962i/archive/%f'
archive_timeout = 1800
WAL files should be archived on a remote server
You can mimic Oracle by archiving WALs locally and then save asynchronously on
tape
– But then it’s up to YOU to restore them correctly

Restoring a Base Backup
Without point-in-time recover
– Just uncompress the base backup in the original location and start the cluster
– It will restore at the time the backup was taken
With point-in-time recover
– Uncompress the base backup
– Prepare a recovery.conf file in the $PGDATA directory containing:
restore_command='cp
/u02/data/postgres/backup/pgts962i/archive/%f %p‘
recovery_target_time = ‘your time here‘
– Start the cluster
– ATTENTION! Archives not found -> earlier recovery point WITHOUT ERRORS

Database Dump
pg_dump dumps the content of a single database (in text or proprietary format)
Very close to Oracle Exp/Imp and Mysql Dump
It does NOT allow further recovery -> recovery point = time of the dump
Example:
– pg_dump -U postgres --format=c --file=dbname.dump dbname
Pg_dump saves grants but not users definitions or other global objects
To dump users definitions:
– pg_dumpall -g --file=global.dump
Pg_dumpall can be used to dump the cluster content as a whole (including data)

Database Restore
pg_Restore restores a dump created with pg_dump
– pg_restore –U postgres –d dbname dbname.dump
Several options exist. My favorite:
– When target database exists -> drop and recreate it
– When target database dows not exist -> create it empty
Grants on non-existing users will fail (e.g. restore on new cluster)
– Restore the global objects beforehand

Operations: Monitoring

EDB Postgres Enterprise Manager (PEM)
Web based since release 7
Monitors availability, performance
Many checks
Some bugs
Some features are PPAS specific

Other 3rd party Monitoring Solutions
On-premises
– Generic: ManageEngine, SolarWinds, Munin, Zabbix, Cacti
– Specific: Open PostgreSQL Monitoring (OPM), pgwatch, PGObserver
Cloud-based (SaaS)
– Generic: LogicMonitor, NewRelic, DataDog, Okmeter
– Specific: pganalyze, pgAnalytics

Instrumentation
Like every database, monitoring is influenced by instrumentation
PostgreSQL is getting better instrumentation
– Since PostgreSQL 9.6: wait interface
– Waits are documented here:
https://www.postgresql.org/docs/9.6/static/monitoring-stats.html#WAIT-EVENT-
TABLE
– Module pg_stat_statements collects waits and activities at statement level
– Other modules exist, still not comparable to ASH (e.g. pg_statsinfo)

Pg_stat_statements example

SQL Standards and
Programmability

SQL Standards and Programmability
Rich SQL: https://www.postgresql.org/docs/current/static/sql.html
Nice features (e.g. Row Level Security, advanced datatypes (e.g. ipv4))
pg/plsql is «SIMILAR» to PL/SQL

External Plugins

External plugins and additional modules
Open Source -> extendible
Many 3rd party projects make PostgreSQL very powerful
Examples
– PostGIS -> Spatial features for PostgreSQL (http://www.postgis.org/)
– PgRouting -> adds routing algorithms to PostGIS
– Bloom -> Enable index access based on bloom filters
– postgres-fdw -> Enable DB Links to external instances (including other RDBMS)
– Hstore -> key-value data type
– Many others!

Questions?
26/10/2017 PostgreSQL97
Ludovico Caldara
ACE Director, Senior Consultant
Tél. +41 79 909 72 75
ludovico.caldara@trivadis.com

Trivadis TechEvent 2017 PostgreSQL für die (Orakel) DBA by Ludovico Caldara

More Related Content

What's hot (20)

Similar to Trivadis TechEvent 2017 PostgreSQL für die (Orakel) DBA by Ludovico Caldara (20)

More from Trivadis (20)

Recently uploaded (20)

Trivadis TechEvent 2017 PostgreSQL für die (Orakel) DBA by Ludovico Caldara