SlideShare a Scribd company logo
Everything You Wanted to Know about Databases as a Developer
but Were Too Afraid to Ask Your DBA
Keith Fiske | Senior Database Engineer @ Crunchy Data
All Things Open | October 2024
whoami?
Keith Fiske | @keithf4
● Senior Database Engineer
● Principle Engineer for
○ pg_partman
○ pgMonitor
○ Other third-party PG Projects at
Crunchy
● Ansible Automation w/ PostgreSQL and
Patroni
● Bird Enthusiast
Introduction
● Databases are essential to most applications
● Most developers aren’t trained in database administration
● DBAs are grumpy busy people
• Database Architecture
• Users and Roles
• Database Objects
• Database Connections
• Database Operations and Transactions
• WAL
• Documentation
Agenda
• Database Architecture
• Users and Roles
• Database Objects
• Database Connections
• Database Operations and Transactions
• WAL
• Documentation
Agenda
https://www.postgresql.org/docs/current/tutorial-arch.html
Database Architecture
How do I Install PostgreSQL?
How do I Install Postgres?
• Install from source code
https://www.postgresql.org/docs/current/install-procedure.html
• Install package for given platform
https://www.postgresql.org/download/
• Choose a managed service
https://crunchybridge.com/register
• Try out the Postgres Playground (Postgres in your browser using WASM):
https://www.crunchydata.com/developers/tutorials
How do I Install Postgres?
# 1. Configure the PostgreSQL repository
user@my_vm$ sudo sh -c 'echo "deb http://apt.postgresql.org/pub/repos/apt $(lsb_release -cs)-pgdg main"
> /etc/apt/sources.list.d/pgdg.list’
# 2. Import the repository signing key
user@my_vm$ wget --quiet -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc | sudo apt-key add –
# 3. Update the package lists
user@my_vm$ sudo apt-get update
# 3. Install PostgreSQL
user@my_vm$ sudo apt-get -y install postgresql
Ubuntu 20.04 (Focal)
How do I Install Postgres?
# 1. Enable the pgdg repository
user@my_vm$ sudo dnf install -y
https://download.postgresql.org/pub/repos/yum/reporpms/EL-9-x86_64/pgdg-redhat-repo-latest.noarch.rpm
# 2. Disable the built-in PostgreSQL module
user@my_vm$ sudo dnf -qy module disable postgresql
# 3. Install PostgreSQL
user@my_vm$ sudo dnf install -y postgresql14-server
Alma9 (RHEL)
What is a PostgreSQL Cluster?
What is a PostgreSQL Cluster?
https://www.postgresql.org/docs/current/creating-cluster.html data directory
Collection of DBs
managed by single instance
How do I Create a PostgreSQL Cluster?
How do I create a PostgreSQL Cluster?
postgres@my_vm$
How do I create a PostgreSQL Cluster?
postgres@my_vm$ export PATH=$PATH:/usr/pgsql-14/bin
postgres@my_vm$
How do I create a PostgreSQL Cluster?
postgres@my_vm$ export PATH=$PATH:/usr/pgsql-14/bin
postgres@my_vm$ export PGDATA=/var/lib/pgsql/14/data
postgres@my_vm$
How do I create a PostgreSQL Cluster?
https://www.postgresql.org/docs/current/app-initdb.html
postgres@my_vm$ export PATH=$PATH:/usr/pgsql-14/bin
postgres@my_vm$ export PGDATA=/var/lib/pgsql/14/data
postgres@my_vm$ initdb
How do I create a PostgreSQL Cluster?
https://www.postgresql.org/docs/current/app-initdb.html
postgres@my_vm$ export PATH=$PATH:/usr/pgsql-14/bin
postgres@my_vm$ export PGDATA=/var/lib/pgsql/14/data
postgres@my_vm$ initdb
...
Success. You can now start the database server using:
/usr/pgsql-14/bin/pg_ctl -D /var/lib/pgsql/14/data -l logfile start
How do I Start (or Stop) PostgreSQL?
How do I Start Postgres?
postgres@my_vm$
How do I Start Postgres?
https://www.postgresql.org/docs/current/app-pg-ctl.html
postgres@my_vm$ pg_ctl -l logfile start
How do I Start Postgres?
https://www.postgresql.org/docs/current/app-pg-ctl.html
postgres@my_vm$ pg_ctl -l logfile start
waiting for server to start.... done
server started
How do I Start Postgres?
postgres@my_vm$ ps -ef|grep postgres
postgres 4236 1 0 11:59 ? 00:00:00 /usr/pgsql-14/bin/postgres
postgres 4237 4236 0 11:59 ? 00:00:00 postgres: logger
postgres 4239 4236 0 11:59 ? 00:00:00 postgres: checkpointer
postgres 4240 4236 0 11:59 ? 00:00:00 postgres: background writer
postgres 4241 4236 0 11:59 ? 00:00:00 postgres: walwriter
postgres 4242 4236 0 11:59 ? 00:00:00 postgres: autovacuum launcher
postgres 4243 4236 0 11:59 ? 00:00:00 postgres: stats collector
postgres 4244 4236 0 11:59 ? 00:00:00 postgres: logical replication launcher
List the Postgres processes
How do I Stop Postgres?
https://www.postgresql.org/docs/current/app-pg-ctl.html
postgres@my_vm$ pg_ctl stop
How do I Stop Postgres?
https://www.postgresql.org/docs/current/app-pg-ctl.html
postgres@my_vm$ pg_ctl stop
waiting for server to shut down.... done
server stopped
How do I Control Postgres using systemd?
root@my_vm$ systemctl start|stop|restart postgresql-14
How do I Control Postgres using systemd?
root@my_vm$ systemctl start|stop|restart postgresql-14
root@my_vm$ systemctl status postgresql-14
How do I Control Postgres using systemd?
root@my_vm$ systemctl start|stop|restart postgresql-14
root@my_vm$ systemctl status postgresql-14
● postgresql-14.service - PostgreSQL 14 database server
Loaded: loaded (/usr/lib/systemd/system/postgresql-14.service; disabled; vendor preset: disabled)
Active: active (running) since ...
What is a Database?
What is a Database?
https://www.postgresql.org/docs/current/managing-databases.html
● Topmost hierarchical level for organizing database objects
● Databases are isolated from users’ point of view
● Databases are closely bound from an administrative perspective
● Pre-defined databases: postgres, template0, template1
● User-defined databases (optional)
What is a Database?
https://www.postgresql.org/docs/current/managing-databases.html
What is a Database?
https://www.postgresql.org/docs/current/managing-databases.html
What is a Database?
https://www.postgresql.org/docs/current/managing-databases.html
What is a Database?
https://www.postgresql.org/docs/current/managing-databases.html
postgres template0 template1
Pre-defined databases
Aside: What is psql?
Aside: What is psql?
https://www.postgresql.org/docs/current/app-psql.html
● Command line tool
● Execute commands and/or scripts against the database
• SQL
• psql commands: e.g. c to connect to a database d to view table details
● Try it out in your browser at the Crunchy Data Postgres Playground:
https://www.crunchydata.com/developers/tutorials
How do I Create a Database?
How do I Create a Database?
https://www.postgresql.org/docs/current/sql-createdatabase.html
postgres@my_vm$
How do I Create a Database?
https://www.postgresql.org/docs/current/sql-createdatabase.html
postgres@my_vm$ psql
How do I Create a Database?
https://www.postgresql.org/docs/current/sql-createdatabase.html
postgres@my_vm$ psql
psql (14.0)
Type "help" for help.
postgres=#
How do I Create a Database?
https://www.postgresql.org/docs/current/sql-createdatabase.html
postgres@my_vm$ psql
psql (14.0)
Type "help" for help.
postgres=# CREATE DATABASE my_database;
How do I Create a Database?
https://www.postgresql.org/docs/current/sql-createdatabase.html
postgres@my_vm$ psql
psql (14.0)
Type "help" for help.
postgres=# CREATE DATABASE my_database;
CREATE DATABASE
How do I List my Databases?
How do I List my Databases?
postgres=# l
How do I List my Databases?
postgres=# l
List of databases
Name | Owner | Encoding | Collate | Ctype | Access privileges
-------------+----------+----------+-------------+-------------+-----------------------
my_database | postgres | UTF8 | en_US.UTF-8 | en_US.UTF-8 |
postgres | postgres | UTF8 | en_US.UTF-8 | en_US.UTF-8 |
template0 | postgres | UTF8 | en_US.UTF-8 | en_US.UTF-8 | =c/postgres +
| | | | | postgres=CTc/postgres
template1 | postgres | UTF8 | en_US.UTF-8 | en_US.UTF-8 | =c/postgres +
| | | | | postgres=CTc/postgres
(4 rows)
How do I List my Databases?
postgres=# SELECT datname FROM pg_database;
https://www.postgresql.org/docs/current/catalogs.html
How do I List my Databases?
postgres=# SELECT datname FROM pg_database;
datname
-----------
my_database
postgres
template0
template1
(4 rows)
https://www.postgresql.org/docs/current/catalogs.html
What is a Tablespace?
What is a Tablespace?
And why should I create one?
● Physical location of the database objects
● Default tablespace pg_default
● Allows control of the disk layout of a PostgreSQL installation
● Accessible to the entire cluster
How do I Create a Tablespace?
How do I Create a Tablespace?
https://www.postgresql.org/docs/current/sql-createtablespace.html
postgres@my_vm$
How do I Create a Tablespace?
https://www.postgresql.org/docs/current/sql-createtablespace.html
postgres@my_vm$ mkdir -p /my_tablespaces/tbsp_1
postgres@my_vm$
How do I Create a Tablespace?
https://www.postgresql.org/docs/current/sql-createtablespace.html
postgres@my_vm$ mkdir -p /my_tablespaces/tbsp_1
postgres@my_vm$ psql -c "CREATE TABLESPACE tablespace_1 location '/my_tablespaces/tbsp_1'"
How do I Create a Tablespace?
https://www.postgresql.org/docs/current/sql-createtablespace.html
postgres@my_vm$ mkdir -p /my_tablespaces/tbsp_1
postgres@my_vm$ psql -c "CREATE TABLESPACE tablespace_1 location '/my_tablespaces/tbsp_1'"
CREATE TABLESPACE
How do I List my Tablespaces?
How do I List my Tablespaces?
postgres=# db
How do I List my Tablespaces?
postgres=# db
List of tablespaces
Name | Owner | Location
--------------+----------+------------------------
pg_default | postgres |
pg_global | postgres |
tablespace_1 | postgres | /my_tablespaces/tbsp_1
(3 rows)
How do I List my Tablespaces?
postgres=# db+
How do I List my Tablespaces?
postgres=# db+
List of tablespaces
Name | Owner | Location | Access privileges | Options | Size | Description
-----------------+----------+------------------------+-------------------+---------+---------+--------------
pg_default | postgres | | | | 33 MB |
pg_global | postgres | | | | 560 kB |
tablespace_1 | postgres | /my_tablespaces/tbsp_1 | | | 0 bytes |
(3 rows)
How do I List my Tablespaces?
postgres@my_vm$ psql -c "select spcname from pg_tablespace"
https://www.postgresql.org/docs/current/catalogs.html
How do I List my Tablespaces?
postgres@my_vm$ psql -c "select spcname from pg_tablespace"
spcname
-----------------
pg_default
pg_global
tablespace_1
(3 rows)
https://www.postgresql.org/docs/current/catalogs.html
What is a Schema?
What is a Schema?
Collection of objects within the database
● Logical grouping - no impact on physical location of objects
● Schema and owner of objects need not be the same
● Specific to the database in which it is created
● Namespace
How do I Create a Schema?
How do I Create a Schema?
https://www.postgresql.org/docs/current/sql-createschema.html
postgres@my_vm$
How do I Create a Schema?
https://www.postgresql.org/docs/current/sql-createschema.html
postgres@my_vm$ psql my_database -c "CREATE SCHEMA my_schema"
How do I Create a Schema?
https://www.postgresql.org/docs/current/sql-createschema.html
postgres@my_vm$ psql my_database -c "CREATE SCHEMA my_schema"
CREATE SCHEMA
How do I List my Schemas?
my_database=# dn+
List of schemas
Name | Owner | Access privileges | Description
-----------+----------+-----------------------+------------------------
my_schema | postgres | |
public | postgres | postgres=UC/postgres+ | standard public schema
| | =UC/postgres |
(2 rows)
• Database Architecture
• Users and Roles
• Database Objects
• Database Connections
• Database Operations and Transactions
• WAL
• Documentation
Agenda
What’s the Difference
Between a User and a Role?
What’s the Difference Between a User and a Role?
● CREATE USER and CREATE ROLE are synonyms except:
• CREATE USER: LOGIN by default
• CREATE ROLE: NOLOGIN by default
● A role can be considered a user or a group (or both)
● A role is available to the entire cluster
● A role can be granted to another role
● A role can own database objects
How do I Create a Role?
How do I Create a Role?
https://www.postgresql.org/docs/current/sql-createrole.html
postgres@my_vm$
How do I Create a Role?
https://www.postgresql.org/docs/current/sql-createrole.html
postgres@my_vm$ psql -c "CREATE ROLE my_role"
How do I Create a Role?
https://www.postgresql.org/docs/current/sql-createrole.html
postgres@my_vm$ psql -c "CREATE ROLE my_role"
CREATE ROLE
postgres@my_vm$
How do I Create a Role?
https://www.postgresql.org/docs/current/sql-createrole.html
postgres@my_vm$ psql -c "CREATE ROLE my_role"
CREATE ROLE
postgres@my_vm$ psql -d my_database -U my_role
How do I Create a Role?
https://www.postgresql.org/docs/current/sql-createrole.html
postgres@my_vm$ psql -c "CREATE ROLE my_role"
CREATE ROLE
postgres@my_vm$ psql -d my_database -U my_role
psql: error: connection to server on socket "/var/run/postgresql/.s.PGSQL.5432" failed:
FATAL: role "my_role" is not permitted to log in
How do I Create a User?
How do I Create a User?
https://www.postgresql.org/docs/current/sql-createuser.html
postgres@my_vm$
How do I Create a User?
https://www.postgresql.org/docs/current/sql-createuser.html
postgres@my_vm$ psql -c "CREATE USER my_user"
How do I Create a User Role?
https://www.postgresql.org/docs/current/sql-createuser.html
postgres@my_vm$ psql -c "CREATE USER my_user"
CREATE ROLE
How do I Create a User Role?
https://www.postgresql.org/docs/current/sql-createuser.html
postgres@my_vm$ psql -c "CREATE USER my_user"
CREATE ROLE
postgres@my_vm$ psql -d my_database -U my_user
How do I Create a User Role?
https://www.postgresql.org/docs/current/sql-createuser.html
postgres@my_vm$ psql -c "CREATE USER my_user"
CREATE ROLE
postgres@my_vm$ psql -d my_database -U my_user
psql (14.0)
Type "help" for help.
What is a Privilege?
What is a Privilege?
Permission to perform certain action(s) on given object(s)
What is a Privilege?
Permission to perform certain action(s) on given object(s)
● Granted by the owner of the object or by a superuser
● Objects : DATABASE, FUNCTION, SCHEMA, TABLE etc
● Permissions: SELECT, INSERT, UPDATE, DELETE, TRUNCATE, CREATE etc
● ALTER DEFAULT PRIVILEGES
Granting Privileges
https://www.postgresql.org/docs/current/ddl-priv.html
postgres@my_vm$ psql -d my_database
my_database=# GRANT CREATE ON DATABASE my_database TO my_user;
GRANT
my_database=# GRANT CREATE ON SCHEMA my_schema TO my_user;
GRANT
my_database=# GRANT CREATE ON TABLESPACE tablespace_1 TO my_user;
GRANT
• Database Architecture
• Users and Roles
• Database Objects
• Database Connections
• Database Operations and Transactions
• WAL
• Documentation
Agenda
Database Objects
• View
• Materialized View
• Sequence
• Table
• Index
• Constraint
What is a Table?
What is a Table?
• A “relation”
• Data arranged in columns and rows
• Rows are not ordered
How do I Create a Table?
How do I Create a Table?
https://www.postgresql.org/docs/current/sql-createtable.html
postgres@my_vm$ psql -d my_database -U my_user
my_database=>
How do I Create a Table?
https://www.postgresql.org/docs/current/sql-createtable.html
postgres@my_vm$ psql -d my_database -U my_user
my_database=> CREATE TABLE my_schema.dept (
my_database(>
How do I Create a Table?
https://www.postgresql.org/docs/current/sql-createtable.html
postgres@my_vm$ psql -d my_database -U my_user
my_database=> CREATE TABLE my_schema.dept (
my_database(> dept_id integer,
my_database(> dept_name varchar);
my_database->
How do I Create a Table?
https://www.postgresql.org/docs/current/sql-createtable.html
postgres@my_vm$ psql -d my_database -U my_user
my_database=> CREATE TABLE my_schema.dept (
my_database(> dept_id integer,
my_database(> dept_name varchar);
my_database-> TABLESPACE tablespace_1;
How do I Create a Table?
https://www.postgresql.org/docs/current/sql-createtable.html
postgres@my_vm$ psql -d my_database -U my_user
my_database=> CREATE TABLE my_schema.dept (
my_database(> dept_id integer,
my_database(> dept_name varchar);
my_database-> TABLESPACE tablespace_1;
CREATE TABLE
What is a Sequence?
What is a Sequence?
Object that generates a sequence of integers
● Specific to a schema
● Used to generate unique numeric identifiers
● Implemented as a single row table
How do I Create a Sequence?
How do I Create a Sequence?
https://www.postgresql.org/docs/current/sql-createsequence.html
my_database=>
How do I Create a Sequence?
https://www.postgresql.org/docs/current/sql-createsequence.html
my_database=> CREATE SEQUENCE my_schema.s_dept
my_database->
How do I Create a Sequence?
https://www.postgresql.org/docs/current/sql-createsequence.html
my_database=> CREATE SEQUENCE my_schema.s_dept
my_database-> INCREMENT BY 1 MINVALUE 1 NO MAXVALUE;
How do I Create a Sequence?
https://www.postgresql.org/docs/current/sql-createsequence.html
my_database=> CREATE SEQUENCE my_schema.s_dept
my_database-> INCREMENT BY 1 MINVALUE 1 NO MAXVALUE;
CREATE SEQUENCE
How do I Create a Sequence?
https://www.postgresql.org/docs/current/sql-createsequence.html
my_database=> CREATE SEQUENCE my_schema.s_dept
my_database-> INCREMENT BY 1 MINVALUE 1 NO MAXVALUE;
CREATE SEQUENCE
my_database=> ALTER TABLE my_schema.dept
my_database->
How do I Create a Sequence?
https://www.postgresql.org/docs/current/sql-createsequence.html
my_database=> CREATE SEQUENCE my_schema.s_dept
my_database-> INCREMENT BY 1 MINVALUE 1 NO MAXVALUE;
CREATE SEQUENCE
my_database=> ALTER TABLE my_schema.dept
my_database-> ALTER COLUMN dept_id SET DEFAULT nextval('my_schema.s_dept');
How do I Create a Sequence?
https://www.postgresql.org/docs/current/sql-createsequence.html
my_database=> CREATE SEQUENCE my_schema.s_dept
my_database-> INCREMENT BY 1 MINVALUE 1 NO MAXVALUE;
CREATE SEQUENCE
my_database=> ALTER TABLE my_schema.dept
my_database-> ALTER COLUMN dept_id SET DEFAULT nextval('my_schema.s_dept');
ALTER TABLE
How do I Auto Generate an ID Column?
How do I Auto Generate an ID Column?
https://www.postgresql.org/docs/current/sql-createtable.html
my_database=> CREATE TABLE my_schema.emp (
my_database(> emp_id integer,
my_database(> emp_name varchar,
my_database(> dept_id integer)
my_database-> TABLESPACE tablespace_1;
How do I Auto Generate an ID Column?
https://www.postgresql.org/docs/current/sql-createtable.html
my_database=> CREATE TABLE my_schema.emp (
my_database(> emp_id integer generated always as identity,
my_database(> emp_name varchar,
my_database(> dept_id integer)
my_database-> TABLESPACE tablespace_1;
How do I Auto Generate an ID Column?
https://www.postgresql.org/docs/current/sql-createtable.html
my_database=> CREATE TABLE my_schema.emp (
my_database(> emp_id integer generated always as identity,
my_database(> emp_name varchar,
my_database(> dept_id integer)
my_database-> TABLESPACE tablespace_1;
CREATE TABLE
How do I View the Table Definitions?
How do I View the Table Definitions?
my_database=> d my_schema.dept
How do I View the Table Definitions?
my_database=> d my_schema.dept
Table "my_schema.dept"
Column | Type | Collation | Nullable | Default
-----------+-------------------+-----------+----------+------------------------------
dept_id | integer | | | nextval('s_dept'::regclass)
dept_name | character varying | | |
Tablespace: "tablespace_1"
How do I View the Table Definitions?
my_database=> d my_schema.emp
Table "my_schema.emp"
Column | Type | Collation | Nullable | Default
----------+-------------------+-----------+----------+------------------------------
emp_id | integer | | not null | generated always as identity
emp_name | character varying | | |
dept_id | integer | | |
Tablespace: "tablespace_1"
What is an Index?
What is an Index?
Ordered list of entries containing a value and a pointer to the table row
● Can create on one or more columns or expressions
● Can speed up searches based on values of column(s) in the index
● Default type is btree
● Various types available in Postgres
https://www.postgresql.org/docs/current/indexes-types.html
How do I Create an Index?
How do I Create an Index?
https://www.postgresql.org/docs/current/sql-createindex.html
my_database=>
How do I Create an Index?
https://www.postgresql.org/docs/current/sql-createindex.html
my_database=> CREATE INDEX emp_dept_id
my_database->
How do I Create an Index?
https://www.postgresql.org/docs/current/sql-createindex.html
my_database=> CREATE INDEX emp_dept_id
my_database-> ON my_schema.emp (dept_id)
my_database->
How do I Create an Index?
https://www.postgresql.org/docs/current/sql-createindex.html
my_database=> CREATE INDEX emp_dept_id
my_database-> ON my_schema.emp (dept_id)
my_database-> TABLESPACE tablespace_1;
How do I Create an Index?
https://www.postgresql.org/docs/current/sql-createindex.html
my_database=> CREATE INDEX emp_dept_id
my_database-> ON my_schema.emp (dept_id)
my_database-> TABLESPACE tablespace_1;
CREATE INDEX
What is a Constraint?
What is a Constraint?
https://www.postgresql.org/docs/current/ddl-constraints.html
● NOT NULL constraint
● CHECK constraint
● PRIMARY KEY (PK) constraint
● FOREIGN KEY (FK) constraint
What is a NOT NULL Constraint?
● Column must contain a value
What is a NOT NULL Constraint?
● Column must contain a value
What is a CHECK Constraint?
● Value must conform to certain rules
● for example: dept_name must contain only letters and spaces
What is a Primary Key?
What is a Primary Key?
One or more columns that allow a row in a table to be identified uniquely
What is a Primary Key?
One or more columns that allow a row in a table to be identified uniquely
● Enforced through PK constraint + unique index
● A table may have only one PK constraint
● The columns in the PK must be NOT NULL
How do I Create a Primary Key Constraint?
How do I Create a PK Constraint?
https://www.postgresql.org/docs/current/sql-altertable.html
my_database=>
How do I Create a PK Constraint?
https://www.postgresql.org/docs/current/sql-altertable.html
my_database=> ALTER TABLE my_schema.emp
my_database-> ADD CONSTRAINT emp_pk PRIMARY KEY (emp_id)
my_database-> USING INDEX TABLESPACE tablespace_1;
How do I Create a PK Constraint?
https://www.postgresql.org/docs/current/sql-altertable.html
my_database=> ALTER TABLE my_schema.emp
my_database-> ADD CONSTRAINT emp_pk PRIMARY KEY (emp_id)
my_database-> USING INDEX TABLESPACE tablespace_1;
ALTER TABLE
How do I Create a PK Constraint?
https://www.postgresql.org/docs/current/sql-altertable.html
my_database=> ALTER TABLE my_schema.emp
my_database-> ADD CONSTRAINT emp_pk PRIMARY KEY (emp_id)
my_database-> USING INDEX TABLESPACE tablespace_1;
ALTER TABLE
my_database=> d my_schema.emp
How do I Create a PK Constraint?
https://www.postgresql.org/docs/current/sql-altertable.html
my_database=> ALTER TABLE my_schema.emp
my_database-> ADD CONSTRAINT emp_pk PRIMARY KEY (emp_id)
my_database-> USING INDEX TABLESPACE tablespace_1;
ALTER TABLE
my_database=> d my_schema.emp
Table "my_schema.emp"
Column | Type | Collation | Nullable | Default
----------+-------------------+-----------+----------+------------------------------
emp_id | integer | | not null | generated always as identity
emp_name | character varying | | |
dept_id | integer | | |
Indexes:
"emp_pk" PRIMARY KEY, btree (emp_id), tablespace "tablespace_1"
"emp_dept_id" btree (dept_id), tablespace "tablespace_1"
Tablespace: "tablespace_1"
How do I Create a PK Constraint?
https://www.postgresql.org/docs/current/sql-altertable.html
my_database=> ALTER TABLE my_schema.dept
my_database-> ADD CONSTRAINT dept_pk PRIMARY KEY (dept_id)
my_database-> USING INDEX TABLESPACE tablespace_1;
ALTER TABLE
my_database=> d my_schema.dept
Table "my_schema.dept"
Column | Type | Collation | Nullable | Default
-----------+-------------------+-----------+----------+------------------------------
dept_id | integer | | not null | generated always as identity
dept_name | character varying | | |
Indexes:
"dept_pk" PRIMARY KEY, btree (dept_id), tablespace "tablespace_1"
Tablespace: "tablespace_1"
What is a Foreign Key?
What is a Foreign Key?
A relationship between two tables
What is a Foreign Key?
A relationship between a “parent” and a “child” table
● A way to enforce “referential integrity”
● References the primary key or another unique key in the parent table
What is a Foreign Key?
A relationship between a “parent” and a “child” table
● A way to enforce “referential integrity”
● References the primary key or another unique key in the parent table
How do I Create a Foreign Key Constraint?
How do I Create a FK Constraint?
https://www.postgresql.org/docs/current/sql-altertable.html
my_database=>
How do I Create a FK Constraint?
https://www.postgresql.org/docs/current/sql-altertable.html
my_database=> ALTER TABLE my_schema.emp
my_database-> ADD CONSTRAINT emp_dept_fk FOREIGN KEY (dept_id)
my_database-> REFERENCES my_schema.dept(dept_id);
How do I Create a FK Constraint?
https://www.postgresql.org/docs/current/sql-altertable.html
my_database=> ALTER TABLE my_schema.emp
my_database-> ADD CONSTRAINT emp_dept_fk FOREIGN KEY (dept_id)
my_database-> REFERENCES my_schema.dept(dept_id);
ALTER TABLE
How do I Create a FK Constraint?
https://www.postgresql.org/docs/current/sql-altertable.html
my_database=> ALTER TABLE my_schema.emp
my_database-> ADD CONSTRAINT emp_dept_fk FOREIGN KEY (dept_id)
my_database-> REFERENCES my_schema.dept(dept_id);
ALTER TABLE
my_database=> d my_schema.emp
...
Foreign-key constraints:
"emp_dept_fk" FOREIGN KEY (dept_id) REFERENCES my_schema.dept(dept_id)
...
How do I Create a FK Constraint?
https://www.postgresql.org/docs/current/sql-altertable.html
my_database=> ALTER TABLE my_schema.emp
my_database-> ADD CONSTRAINT emp_dept_fk FOREIGN KEY (dept_id)
my_database-> REFERENCES my_schema.dept(dept_id);
ALTER TABLE
my_database=> d my_schema.emp
...
Foreign-key constraints:
"emp_dept_fk" FOREIGN KEY (dept_id) REFERENCES my_schema.dept(dept_id)
...
my_database=> d my_schema.dept
...
Referenced by:
TABLE "emp" CONSTRAINT "emp_dept_fk" FOREIGN KEY (dept_id) REFERENCES dept(dept_id)
...
Aside: What is a Search Path?
Aside: What is a Search Path?
Schema(s) that will be searched if I don’t refer to an object using a fully-qualified object name
my_database=>
Aside: What is a Search Path?
Schema(s) that will be searched if I don’t refer to an object using a fully-qualified object name
my_database=> SHOW search_path;
Aside: What is a Search Path?
Schema(s) that will be searched if I don’t refer to an object using a fully-qualified object name
my_database=> SHOW search_path;
search_path
-----------------
"$user", public
my_database=>
Aside: What is a Search Path?
Schema(s) that will be searched if I don’t refer to an object using a fully-qualified object name
my_database=> SHOW search_path;
search_path
-----------------
"$user", public
my_database=> select * from emp;
Aside: What is a Search Path?
Schema(s) that will be searched if I don’t refer to an object using a fully-qualified object name
my_database=> SHOW search_path;
search_path
-----------------
"$user", public
my_database=> select * from emp;
ERROR: relation "emp" does not exist
LINE 1: select * from emp
^
Aside: What is a Search Path?
Schema(s) that will be searched if I don’t refer to an object using a fully-qualified object name
my_database=> SET search_path = my_schema;
Aside: What is a Search Path?
Schema(s) that will be searched if I don’t refer to an object using a fully-qualified object name
my_database=> SET search_path = my_schema;
SET
my_database=>
Aside: What is a Search Path?
Schema(s) that will be searched if I don’t refer to an object using a fully-qualified object name
my_database=> SET search_path = my_schema;
SET
my_database=> SHOW search_path;
Aside: What is a Search Path?
Schema(s) that will be searched if I don’t refer to an object using a fully-qualified object name
my_database=> SET search_path = my_schema;
SET
my_database=> SHOW search_path;
search_path
-------------
my_schema
(1 row)
my_database=>
Aside: What is a Search Path?
Schema(s) that will be searched if I don’t refer to an object using a fully-qualified object name
my_database=> SET search_path = my_schema;
SET
my_database=> SHOW search_path;
search_path
-------------
my_schema
(1 row)
my_database=> select * from emp;
Aside: What is a Search Path?
Schema(s) that will be searched if I don’t refer to an object using a fully-qualified object name
my_database=> SET search_path = my_schema;
SET
my_database=> SHOW search_path;
search_path
-------------
my_schema
(1 row)
my_database=> select * from emp;
emp_id | emp_name | dept_id
--------+----------+---------
(0 rows)
Aside: What is a Search Path?
I can set the search path (and other options) automatically
postgres@my_vm$ echo 'set search_path to my_schema' >> ~/.psqlrc
postgres@my_vm$
Aside: What is a Search Path?
I can set the search path (and other options) automatically
postgres@my_vm$ echo 'set search_path to my_schema' >> ~/.psqlrc
postgres@my_vm$ psql -d my_database -U my_user
Aside: What is a Search Path?
I can set the search path (and other options) automatically
postgres@my_vm$ echo 'set search_path to my_schema' >> ~/.psqlrc
postgres@my_vm$ psql -d my_database -U my_user
SET
psql (14.0)
Type "help" for help.
my_database=>
Aside: What is a Search Path?
I can set the search path (and other options) automatically
postgres@my_vm$ echo 'set search_path to my_schema' >> ~/.psqlrc
postgres@my_vm$ psql -d my_database -U my_user
SET
psql (14.0)
Type "help" for help.
my_database=> SHOW search_path;
Aside: What is a Search Path?
I can set the search path (and other options) automatically
postgres@my_vm$ echo 'set search_path to my_schema' >> ~/.psqlrc
postgres@my_vm$ psql -d my_database -U my_user
SET
psql (14.0)
Type "help" for help.
my_database=> SHOW search_path;
search_path
-------------
my_schema
(1 row)
How do I Populate my Tables?
How do I Populate my Tables?
https://www.postgresql.org/docs/current/sql-insert.html
my_database=> INSERT INTO dept (dept_name)
my_database-> VALUES ('Sales'),('Consulting'),('Product'),('HR');
INSERT 0 4
How do I Populate my Tables?
https://www.postgresql.org/docs/current/sql-insert.html
my_database=> INSERT INTO dept (dept_name)
my_database-> VALUES ('Sales'),('Consulting'),('Product'),('HR');
INSERT 0 4
my_database=> SELECT * FROM dept ORDER BY dept_id;
dept_id | dept_name
---------+------------
1 | Sales
2 | Consulting
3 | Product
4 | HR
(4 rows)
How do I Populate my Tables?
https://www.postgresql.org/docs/current/sql-insert.html
my_database=> INSERT INTO emp (emp_name, dept_id)
my_database->VALUES ('Ay Bee',2),('Cee Dee',2),('E.F. Gee',1),('Aitch Eye',null),('Jay Kay',4);
INSERT 0 4
How do I Populate my Tables?
https://www.postgresql.org/docs/current/sql-insert.html
my_database=> INSERT INTO emp (emp_name, dept_id)
my_database->VALUES ('Ay Bee',2),('Cee Dee',2),('E.F. Gee',1),('Aitch Eye',null),('Jay Kay',4);
INSERT 0 4
my_database=> SELECT * FROM emp ORDER BY emp_id;
emp_id | emp_name | dept_id
--------+-----------+---------
1 | Ay Bee | 2
2 | Cee Dee | 2
3 | E.F. Gee | 1
4 | Aitch Eye |
5 | Jay Kay | 4
(5 rows)
What if I try to Insert an Invalid dept_id?
https://www.postgresql.org/docs/current/sql-insert.html
my_database=> INSERT INTO emp (emp_name, dept_id)
my_database-> VALUES ('No Good',7);
What if I try to Insert an Invalid dept_id?
https://www.postgresql.org/docs/current/sql-insert.html
my_database=> INSERT INTO emp (emp_name, dept_id)
my_database-> VALUES ('No Good',7);
ERROR: insert or update on table "emp" violates foreign key constraint "emp_dept_fk"
DETAIL: Key (dept_id)=(7) is not present in table "dept".
What is a View?
What is a View?
A virtual table, based on a query
● Does not take up any space
● Executed in real-time
● Shorthand for a long query
● Present just certain data to certain users
How do I Create a View?
How do I Create a View?
https://www.postgresql.org/docs/current/sql-createview.html
my_database=> CREATE VIEW emp_name_view AS
my_database-> SELECT emp_name AS "employee_name" FROM emp ORDER BY emp_name;
How do I Create a View?
https://www.postgresql.org/docs/current/sql-createview.html
my_database=> CREATE VIEW emp_name_view AS
my_database-> SELECT emp_name AS "employee_name" FROM emp ORDER BY emp_name;
CREATE VIEW
How do I Create a View?
https://www.postgresql.org/docs/current/sql-createview.html
my_database=> CREATE VIEW emp_name_view AS
my_database-> SELECT emp_name AS "employee_name" FROM emp ORDER BY emp_name;
CREATE VIEW
my_database=> SELECT * FROM emp_name_view;
employee_name
---------------
Aitch Eye
Ay Bee
Cee Dee
E.F. Gee
Jay Kay
(5 rows)
What is a Materialized View?
What is a Materialized View?
● Query is not executed in real-time
● Can be “refreshed” (query re-executed to gather latest results)
● Useful for aggregating data
● Avoids re-executing long-running/frequently executed queries
A table that contains the results of a query
How do I Create a Materialized View?
How do I Create a Materialized View?
https://www.postgresql.org/docs/current/sql-creatematerializedview.html
my_database=> CREATE MATERIALIZED VIEW emp_mview
my_database-> TABLESPACE tablespace_1 AS
my_database-> SELECT emp_id, emp_name AS "employee_name" FROM emp WHERE dept_id in (1,2,3);
How do I Create a Materialized View?
https://www.postgresql.org/docs/current/sql-creatematerializedview.html
my_database=> CREATE MATERIALIZED VIEW emp_mview
my_database-> TABLESPACE tablespace_1 AS
my_database-> SELECT emp_id, emp_name AS "employee_name" FROM emp WHERE dept_id in (1,2,3);
SELECT 3
How do I Create a Materialized View?
https://www.postgresql.org/docs/current/sql-creatematerializedview.html
my_database=> CREATE MATERIALIZED VIEW emp_mview
my_database-> TABLESPACE tablespace_1 AS
my_database-> SELECT emp_id, emp_name AS "employee_name" FROM emp WHERE dept_id in (1,2,3);
SELECT 3
my_database=> SELECT * FROM emp_mview;
emp_id | employee_name
--------+---------------
1 | Ay Bee
2 | Cee Dee
3 | E.F. Gee
(3 rows)
How do I Refresh a Materialized View?
First, update some data…
my_database=> UPDATE emp SET emp_name = 'CEE DEE' WHERE emp_id = 2;
UPDATE 1
How do I Refresh a Materialized View?
First, update some data…
my_database=> UPDATE emp SET emp_name = 'CEE DEE' WHERE emp_id = 2;
UPDATE 1
my_database=> SELECT emp_name FROM emp WHERE emp_id = 2;
emp_name
----------
CEE DEE
How do I Refresh a Materialized View?
First, update some data…
my_database=> UPDATE emp SET emp_name = 'CEE DEE' WHERE emp_id = 2;
UPDATE 1
my_database=> SELECT emp_name FROM emp WHERE emp_id = 2;
emp_name
----------
CEE DEE
my_database=> SELECT employee_name FROM emp_name_view WHERE employee_name like 'C%';
employee_name
---------------
CEE DEE
(1 row)
How do I Refresh a Materialized View?
First, update some data…
my_database=>SELECT employee_name FROM emp_mview where emp_id = 2;
employee_name
---------------
Cee Dee
(1 row)
How do I Refresh a Materialized View?
https://www.postgresql.org/docs/current/sql-refreshmaterializedview.html
my_database=>SELECT employee_name FROM emp_mview where emp_id = 2;
employee_name
---------------
Cee Dee
(1 row)
my_database=> REFRESH MATERIALIZED VIEW emp_mview;
REFRESH MATERIALIZED VIEW
How do I Refresh a Materialized View?
https://www.postgresql.org/docs/current/sql-refreshmaterializedview.html
my_database=>SELECT employee_name FROM emp_mview where emp_id = 2;
employee_name
---------------
Cee Dee
(1 row)
my_database=> REFRESH MATERIALIZED VIEW emp_mview;
REFRESH MATERIALIZED VIEW
my_database=> SELECT employee_name FROM emp_mview where emp_id = 2;
employee_name
---------------
CEE DEE
• Database Architecture
• Users and Roles
• Database Objects
• Database Connections
• Database Operations and Transactions
• WAL
• Documentation
Agenda
Database Connections
● Information Required
● Connecting Django to Postgres
● Client Tools
What Information do I Need?
● host
● port
● database
● username
● password/certificate
What Information do I Need?
name of database to connect to
database host
database port (5432 by default)
database username
Format depends on client tool/driver
psql example: psql -h 127.0.0.1 -p 5432 -U myuser mydatabase
jdbc example: jdbc:postgresql://myuser@127.0.0.1:5432/mydatabase
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql',
'NAME': 'mydatabase',
'USER': 'myuser',
'HOST': '127.0.0.1',
'PORT': '5432',
'PASSWORD': 'password',
}
}
How do I Configure a Postgres Connection in Django?
https://docs.djangoproject.com/en/4.1/ref/databases/#postgresql-notes
or define a service in .pg_service.conf
settings.py conf file
or provide a password (.pgpass) file
What Client Tools are Available?
Many client tools allow connection to Postgres, including:
● psql
● DBeaver (multi-platform)
https://dbeaver.io/
● pgAdmin4
https://www.pgadmin.org/
• Database Architecture
• Users and Roles
• Database Objects
• Database Connections
• Database Operations and Transactions
• WAL
• Documentation
Agenda
Database Operations and Transactions
• SQL, DML, DDL
• Join Types
• Execution Plans
• Transactions
• Commit
• Rollback
What is a Transaction?
What is a Transaction?
● Ends with a COMMIT or a ROLLBACK:
• COMMIT: operations permanently applied to database
• ROLLBACK: operations cancelled
● Changes only visible to other transactions after COMMIT
● Certain locks retained until end of transaction
● ACID (Atomic, Consistent, Isolated, Durable)
A single unit of work that consists of one or more operations
How do I Begin/End a Transaction?
How do I Begin/End a Transaction?
● No need to explicitly BEGIN or COMMIT any transactions
● Each command is a distinct transaction with implicit COMMIT
● To manage a transaction manually:
• Issue BEGIN to start a transaction
• Execute the operations that comprise the transaction
• Issue COMMIT to make the changes permanent or/
• Issue ROLLBACK to undo the changes
Example for psql: default behaviour is autocommit
What is SQL/DML/DDL?
What is SQL/DML/DDL?
SQL Structured Query Language
https://www.postgresql.org/docs/current/sql.html
What is SQL/DML/DDL?
SQL Structured Query Language
DML Data Manipulation Language SELECT, INSERT, UPDATE, DELETE...
https://www.postgresql.org/docs/current/sql.html
What is SQL/DML/DDL?
SQL Structured Query Language
DML Data Manipulation Language SELECT, INSERT, UPDATE, DELETE...
DDL Data Definition Language CREATE, ALTER, DROP, RENAME ...
https://www.postgresql.org/docs/current/sql.html
What is a Join?
What is a Join?
A way to select from multiple tables in one statement
What are the Different Types of Join?
What are the Different Types of Join?
INNER JOIN
my_database=> SELECT emp.emp_id, emp.emp_name, dept.dept_name
my_database-> FROM emp INNER JOIN dept
my_database-> ON emp.dept_id = dept.dept_id;
What are the Different Types of Join?
INNER JOIN or JOIN (default join type)
my_database=> SELECT emp.emp_id, emp.emp_name, dept.dept_name
my_database-> FROM emp JOIN dept
my_database-> ON emp.dept_id = dept.dept_id;
What are the Different Types of Join?
INNER JOIN or JOIN default join type
my_database=> SELECT emp.emp_id, emp.emp_name, dept.dept_name
my_database-> FROM emp JOIN dept
my_database-> ON emp.dept_id = dept.dept_id;
What are the Different Types of Join?
INNER JOIN or JOIN default join type
my_database=> SELECT emp.emp_id, emp.emp_name, dept.dept_name
my_database-> FROM emp JOIN dept
my_database-> ON emp.dept_id = dept.dept_id;
emp_id | emp_name | dept_name
--------+----------+------------
1 | Ay Bee | two
2 | Cee Dee | two
3 | E.F. Gee | one
5 | Jay Kay | four
(4 rows)
What are the Different Types of Join?
LEFT OUTER JOIN
my_database=> SELECT emp.emp_id, emp.emp_name, dept.dept_name
my_database-> FROM emp LEFT OUTER JOIN dept
my_database-> ON emp.dept_id = dept.dept_id;
What are the Different Types of Join?
LEFT OUTER JOIN or LEFT JOIN
my_database=> SELECT emp.emp_id, emp.emp_name, dept.dept_name
my_database-> FROM emp LEFT JOIN dept
my_database-> ON emp.dept_id = dept.dept_id;
What are the Different Types of Join?
LEFT OUTER JOIN or LEFT JOIN
my_database=> SELECT emp.emp_id, emp.emp_name, dept.dept_name
my_database-> FROM emp LEFT JOIN dept
my_database-> ON emp.dept_id = dept.dept_id;
What are the Different Types of Join?
LEFT OUTER JOIN or LEFT JOIN
my_database=> SELECT emp.emp_id, emp.emp_name, dept.dept_name
my_database-> FROM emp LEFT JOIN dept
my_database-> ON emp.dept_id = dept.dept_id;
emp_id | emp_name | dept_name
--------+-----------+------------
1 | Ay Bee | two
2 | Cee Dee | two
3 | E.F. Gee | one
4 | Aitch Eye |
5 | Jay Kay | four
(5 rows)
What are the Different Types of Join?
RIGHT OUTER JOIN
my_database=> SELECT emp.emp_id, emp.emp_name, dept.dept_name
my_database-> FROM emp RIGHT OUTER JOIN dept
my_database-> ON emp.dept_id = dept.dept_id;
What are the Different Types of Join?
RIGHT OUTER JOIN or RIGHT JOIN
my_database=> SELECT emp.emp_id, emp.emp_name, dept.dept_name
my_database-> FROM emp RIGHT JOIN dept
my_database-> ON emp.dept_id = dept.dept_id;
What are the Different Types of Join?
RIGHT OUTER JOIN or RIGHT JOIN
my_database=> SELECT emp.emp_id, emp.emp_name, dept.dept_name
my_database-> FROM emp RIGHT JOIN dept
my_database-> ON emp.dept_id = dept.dept_id;
What are the Different Types of Join?
RIGHT OUTER JOIN or RIGHT JOIN
my_database=> SELECT emp.emp_id, emp.emp_name, dept.dept_name
my_database-> FROM emp RIGHT JOIN dept
my_database-> ON emp.dept_id = dept.dept_id;
emp_id | emp_name | dept_name
--------+----------+------------
1 | Ay Bee | two
2 | Cee Dee | two
3 | E.F. Gee | one
5 | Jay Kay | four
| | three
(5 rows)
What are the Different Types of Join?
FULL OUTER JOIN
my_database=> SELECT emp.emp_id, emp.emp_name, dept.dept_name
my_database-> FROM emp FULL OUTER JOIN dept
my_database-> ON emp.dept_id = dept.dept_id;
What are the Different Types of Join?
FULL OUTER JOIN
my_database=> SELECT emp.emp_id, emp.emp_name, dept.dept_name
my_database-> FROM emp FULL OUTER JOIN dept
my_database-> ON emp.dept_id = dept.dept_id;
What are the Different Types of Join?
FULL OUTER JOIN
my_database=> SELECT emp.emp_id, emp.emp_name, dept.dept_name
my_database-> FROM emp FULL OUTER JOIN dept
my_database-> ON emp.dept_id = dept.dept_id;
emp_id | emp_name | dept_name
--------+-----------+------------
1 | Ay Bee | two
2 | Cee Dee | two
3 | E.F. Gee | one
4 | Aitch Eye |
5 | Jay Kay | four
| | three
(6 rows)
What is an Execution Plan?
What is an Execution Plan?
● Access path: full table scan, index lookup
● Join algorithm: nested loop, hash, merge
● Estimated # rows
● Estimated cost
The steps that Postgres takes to execute a SQL statement
How do I Generate an Execution Plan?
How do I Generate an Execution Plan?
https://www.postgresql.org/docs/current/using-explain.html
my_database=> EXPLAIN SELECT * FROM emp_name_view;
How do I Generate an Execution Plan?
https://www.postgresql.org/docs/current/using-explain.html
my_database=> EXPLAIN SELECT * FROM emp_name_view;
QUERY PLAN
----------------------------------------------------------
Sort (cost=1.11..1.12 rows=5 width=32)
Sort Key: emp.emp_name
-> Seq Scan on emp (cost=0.00..1.05 rows=5 width=32)
(3 rows)
How do I Generate an Execution Plan?
https://www.postgresql.org/docs/current/using-explain.html
my_database=> EXPLAIN ANALYZE SELECT * FROM emp_name_view;
How do I Generate an Execution Plan?
https://www.postgresql.org/docs/current/using-explain.html
my_database=> EXPLAIN ANALYZE SELECT * FROM emp_name_view;
QUERY PLAN
--------------------------------------------------------------------------------------------------------
Sort (cost=83.37..86.37 rows=1200 width=32) (actual time=0.225..0.226 rows=5 loops=1)
Sort Key: emp.emp_name
Sort Method: quicksort Memory: 25kB
-> Seq Scan on emp (cost=0.00..22.00 rows=1200 width=32) (actual time=0.009..0.010 rows=5 loops=1)
Planning Time: 0.310 ms
Execution Time: 0.434 ms
(6 rows)
• Database Architecture
• Users and Roles
• Database Objects
• Database Connections
• Database Operations and Transactions
• WAL
• Documentation
Agenda
What is WAL?
What is WAL?
● Write-ahead log
● Details of changes executed against the Postgres cluster
● Allows crash-recovery, online backups and restores
● WAL files can be archived to allow point in time recovery (PITR)
● DON’T DELETE THEM!
https://www.postgresql.org/docs/current/wal-intro.html
Where are my WAL Files?
postgres@my_vm$ ls -ltr $PGDATA/pg_wal
Where are my WAL Files?
postgres@my_vm$ ls -ltr $PGDATA/pg_wal
total 32768
drwx------. 2 postgres postgres 6 Nov 9 11:00 archive_status
-rw-------. 1 postgres postgres 16777216 Nov 14 08:41 000000010000000000000001
-rw-------. 1 postgres postgres 16777216 Nov 14 08:41 000000010000000000000002
-rw-------. 1 postgres postgres 16777216 Nov 14 08:46 000000010000000000000003
• Database Architecture
• Users and Roles
• Database Objects
• Database Connections
• Database Operations and Transactions
• Documentation
Agenda
Where is the PostgreSQL Documentation?
PostgreSQL Documentation
Current online version: https://www.postgresql.org/docs/current/index.html
https://www.postgresql.org/docs
Where else can I get information?
Slack Channel
● Over 100 channels
● 20k members
https://postgres-slack.herokuapp.com/
Mailing Lists
● Linked to PostgreSQL Community account
● Many different lists
https://lists.postgresql.org/
Congratulations!
Thank You!
Karen Jex | @karenhjex | karen.jex@crunchydata.com
Slides: https://karenjex.blogspot.com
Keith Fiske | @keithf4 | keith.fiske@crunchydata.com

More Related Content

PDF
Trivadis TechEvent 2017 PostgreSQL für die (Orakel) DBA by Ludovico Caldara
PDF
Exploring Postgres with Bruce Momjian
 
PDF
Whats wrong with postgres | PGConf EU 2019 | Craig Kerstiens
PDF
0292-introduction-postgresql.pdf
ODP
Postgre sql unleashed
ODP
Introduction to PostgreSQL
PDF
Introduction to PostgreSQL for System Administrators
PDF
Postgres 12 Cluster Database operations.
Trivadis TechEvent 2017 PostgreSQL für die (Orakel) DBA by Ludovico Caldara
Exploring Postgres with Bruce Momjian
 
Whats wrong with postgres | PGConf EU 2019 | Craig Kerstiens
0292-introduction-postgresql.pdf
Postgre sql unleashed
Introduction to PostgreSQL
Introduction to PostgreSQL for System Administrators
Postgres 12 Cluster Database operations.

Similar to Everything You Wanted to Know About Databases (Keith).pdf (20)

PPTX
Postgre sql best_practices
PPTX
Postgresql Database Administration Basic - Day2
PPTX
TechEvent PostgreSQL Best Practices
PPTX
Postgre sql best_practices
PPTX
Chjkkkkkkkkkkkkkkkkkjjjjjjjjjjjjjjjjjjjjjjjjjj01_The Basics.pptx
PDF
Hello World with EDB Postgres
 
PDF
Postgresql Up And Running Regina Obe Leo Hsu
PDF
9.6_Course Material-Postgresql_002.pdf
PPTX
Getting started with postgresql
PDF
Postgresql quick guide
ODP
PostgreSQL Administration for System Administrators
PPTX
PostgreSQL as a Strategic Tool
 
PDF
PostgreSQL 10; Long Awaited Enterprise Solutions
ODP
Pro PostgreSQL, OSCon 2008
PDF
PostgreSQL : Introduction
PDF
Mastering PostgreSQL Administration
 
PDF
Mastering PostgreSQL Administration
PPTX
Easy Installation and Setup of PostgreSQL on Linux, OSX, & Windows
PDF
PostgreSQL Prologue
PPTX
My Favorite PostgreSQL Books
 
Postgre sql best_practices
Postgresql Database Administration Basic - Day2
TechEvent PostgreSQL Best Practices
Postgre sql best_practices
Chjkkkkkkkkkkkkkkkkkjjjjjjjjjjjjjjjjjjjjjjjjjj01_The Basics.pptx
Hello World with EDB Postgres
 
Postgresql Up And Running Regina Obe Leo Hsu
9.6_Course Material-Postgresql_002.pdf
Getting started with postgresql
Postgresql quick guide
PostgreSQL Administration for System Administrators
PostgreSQL as a Strategic Tool
 
PostgreSQL 10; Long Awaited Enterprise Solutions
Pro PostgreSQL, OSCon 2008
PostgreSQL : Introduction
Mastering PostgreSQL Administration
 
Mastering PostgreSQL Administration
Easy Installation and Setup of PostgreSQL on Linux, OSX, & Windows
PostgreSQL Prologue
My Favorite PostgreSQL Books
 
Ad

More from All Things Open (20)

PDF
Agentic AI for Developers and Data Scientists Build an AI Agent in 10 Lines o...
PPTX
Big Data on a Small Budget: Scalable Data Visualization for the Rest of Us - ...
PDF
AI 3-in-1: Agents, RAG, and Local Models - Brent Laster
PDF
Let's Create a GitHub Copilot Extension! - Nick Taylor, Pomerium
PDF
Leveraging Pre-Trained Transformer Models for Protein Function Prediction - T...
PDF
Gen AI: AI Agents - Making LLMs work together in an organized way - Brent Las...
PDF
You Don't Need an AI Strategy, But You Do Need to Be Strategic About AI - Jes...
PPTX
DON’T PANIC: AI IS COMING – The Hitchhiker’s Guide to AI - Mark Hinkle, Perip...
PDF
Fine-Tuning Large Language Models with Declarative ML Orchestration - Shivay ...
PDF
Leveraging Knowledge Graphs for RAG: A Smarter Approach to Contextual AI Appl...
PPTX
Artificial Intelligence Needs Community Intelligence - Sriram Raghavan, IBM R...
PDF
Don't just talk to AI, do more with AI: how to improve productivity with AI a...
PPTX
Open-Source GenAI vs. Enterprise GenAI: Navigating the Future of AI Innovatio...
PDF
The Death of the Browser - Rachel-Lee Nabors, AgentQL
PDF
Making Operating System updates fast, easy, and safe
PDF
Reshaping the landscape of belonging to transform community
PDF
The Unseen, Underappreciated Security Work Your Maintainers May (or may not) ...
PDF
Integrating Diversity, Equity, and Inclusion into Product Design
PDF
The Open Source Ecosystem for eBPF in Kubernetes
PDF
Open Source Privacy-Preserving Metrics - Sarah Gran & Brandon Pitman
Agentic AI for Developers and Data Scientists Build an AI Agent in 10 Lines o...
Big Data on a Small Budget: Scalable Data Visualization for the Rest of Us - ...
AI 3-in-1: Agents, RAG, and Local Models - Brent Laster
Let's Create a GitHub Copilot Extension! - Nick Taylor, Pomerium
Leveraging Pre-Trained Transformer Models for Protein Function Prediction - T...
Gen AI: AI Agents - Making LLMs work together in an organized way - Brent Las...
You Don't Need an AI Strategy, But You Do Need to Be Strategic About AI - Jes...
DON’T PANIC: AI IS COMING – The Hitchhiker’s Guide to AI - Mark Hinkle, Perip...
Fine-Tuning Large Language Models with Declarative ML Orchestration - Shivay ...
Leveraging Knowledge Graphs for RAG: A Smarter Approach to Contextual AI Appl...
Artificial Intelligence Needs Community Intelligence - Sriram Raghavan, IBM R...
Don't just talk to AI, do more with AI: how to improve productivity with AI a...
Open-Source GenAI vs. Enterprise GenAI: Navigating the Future of AI Innovatio...
The Death of the Browser - Rachel-Lee Nabors, AgentQL
Making Operating System updates fast, easy, and safe
Reshaping the landscape of belonging to transform community
The Unseen, Underappreciated Security Work Your Maintainers May (or may not) ...
Integrating Diversity, Equity, and Inclusion into Product Design
The Open Source Ecosystem for eBPF in Kubernetes
Open Source Privacy-Preserving Metrics - Sarah Gran & Brandon Pitman
Ad

Recently uploaded (20)

PDF
Unlocking AI with Model Context Protocol (MCP)
PDF
Reach Out and Touch Someone: Haptics and Empathic Computing
PDF
NewMind AI Monthly Chronicles - July 2025
PPTX
VMware vSphere Foundation How to Sell Presentation-Ver1.4-2-14-2024.pptx
PDF
cuic standard and advanced reporting.pdf
PPTX
MYSQL Presentation for SQL database connectivity
PDF
Dropbox Q2 2025 Financial Results & Investor Presentation
PDF
Machine learning based COVID-19 study performance prediction
PDF
Architecting across the Boundaries of two Complex Domains - Healthcare & Tech...
PDF
Review of recent advances in non-invasive hemoglobin estimation
PPTX
Big Data Technologies - Introduction.pptx
PDF
TokAI - TikTok AI Agent : The First AI Application That Analyzes 10,000+ Vira...
DOCX
The AUB Centre for AI in Media Proposal.docx
PDF
Advanced methodologies resolving dimensionality complications for autism neur...
PDF
Approach and Philosophy of On baking technology
PDF
Mobile App Security Testing_ A Comprehensive Guide.pdf
PDF
Empathic Computing: Creating Shared Understanding
PPTX
Detection-First SIEM: Rule Types, Dashboards, and Threat-Informed Strategy
PPT
“AI and Expert System Decision Support & Business Intelligence Systems”
PDF
Electronic commerce courselecture one. Pdf
Unlocking AI with Model Context Protocol (MCP)
Reach Out and Touch Someone: Haptics and Empathic Computing
NewMind AI Monthly Chronicles - July 2025
VMware vSphere Foundation How to Sell Presentation-Ver1.4-2-14-2024.pptx
cuic standard and advanced reporting.pdf
MYSQL Presentation for SQL database connectivity
Dropbox Q2 2025 Financial Results & Investor Presentation
Machine learning based COVID-19 study performance prediction
Architecting across the Boundaries of two Complex Domains - Healthcare & Tech...
Review of recent advances in non-invasive hemoglobin estimation
Big Data Technologies - Introduction.pptx
TokAI - TikTok AI Agent : The First AI Application That Analyzes 10,000+ Vira...
The AUB Centre for AI in Media Proposal.docx
Advanced methodologies resolving dimensionality complications for autism neur...
Approach and Philosophy of On baking technology
Mobile App Security Testing_ A Comprehensive Guide.pdf
Empathic Computing: Creating Shared Understanding
Detection-First SIEM: Rule Types, Dashboards, and Threat-Informed Strategy
“AI and Expert System Decision Support & Business Intelligence Systems”
Electronic commerce courselecture one. Pdf

Everything You Wanted to Know About Databases (Keith).pdf

  • 1. Everything You Wanted to Know about Databases as a Developer but Were Too Afraid to Ask Your DBA Keith Fiske | Senior Database Engineer @ Crunchy Data All Things Open | October 2024
  • 2. whoami? Keith Fiske | @keithf4 ● Senior Database Engineer ● Principle Engineer for ○ pg_partman ○ pgMonitor ○ Other third-party PG Projects at Crunchy ● Ansible Automation w/ PostgreSQL and Patroni ● Bird Enthusiast
  • 3. Introduction ● Databases are essential to most applications ● Most developers aren’t trained in database administration ● DBAs are grumpy busy people
  • 4. • Database Architecture • Users and Roles • Database Objects • Database Connections • Database Operations and Transactions • WAL • Documentation Agenda
  • 5. • Database Architecture • Users and Roles • Database Objects • Database Connections • Database Operations and Transactions • WAL • Documentation Agenda
  • 7. How do I Install PostgreSQL?
  • 8. How do I Install Postgres? • Install from source code https://www.postgresql.org/docs/current/install-procedure.html • Install package for given platform https://www.postgresql.org/download/ • Choose a managed service https://crunchybridge.com/register • Try out the Postgres Playground (Postgres in your browser using WASM): https://www.crunchydata.com/developers/tutorials
  • 9. How do I Install Postgres? # 1. Configure the PostgreSQL repository user@my_vm$ sudo sh -c 'echo "deb http://apt.postgresql.org/pub/repos/apt $(lsb_release -cs)-pgdg main" > /etc/apt/sources.list.d/pgdg.list’ # 2. Import the repository signing key user@my_vm$ wget --quiet -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc | sudo apt-key add – # 3. Update the package lists user@my_vm$ sudo apt-get update # 3. Install PostgreSQL user@my_vm$ sudo apt-get -y install postgresql Ubuntu 20.04 (Focal)
  • 10. How do I Install Postgres? # 1. Enable the pgdg repository user@my_vm$ sudo dnf install -y https://download.postgresql.org/pub/repos/yum/reporpms/EL-9-x86_64/pgdg-redhat-repo-latest.noarch.rpm # 2. Disable the built-in PostgreSQL module user@my_vm$ sudo dnf -qy module disable postgresql # 3. Install PostgreSQL user@my_vm$ sudo dnf install -y postgresql14-server Alma9 (RHEL)
  • 11. What is a PostgreSQL Cluster?
  • 12. What is a PostgreSQL Cluster? https://www.postgresql.org/docs/current/creating-cluster.html data directory Collection of DBs managed by single instance
  • 13. How do I Create a PostgreSQL Cluster?
  • 14. How do I create a PostgreSQL Cluster? postgres@my_vm$
  • 15. How do I create a PostgreSQL Cluster? postgres@my_vm$ export PATH=$PATH:/usr/pgsql-14/bin postgres@my_vm$
  • 16. How do I create a PostgreSQL Cluster? postgres@my_vm$ export PATH=$PATH:/usr/pgsql-14/bin postgres@my_vm$ export PGDATA=/var/lib/pgsql/14/data postgres@my_vm$
  • 17. How do I create a PostgreSQL Cluster? https://www.postgresql.org/docs/current/app-initdb.html postgres@my_vm$ export PATH=$PATH:/usr/pgsql-14/bin postgres@my_vm$ export PGDATA=/var/lib/pgsql/14/data postgres@my_vm$ initdb
  • 18. How do I create a PostgreSQL Cluster? https://www.postgresql.org/docs/current/app-initdb.html postgres@my_vm$ export PATH=$PATH:/usr/pgsql-14/bin postgres@my_vm$ export PGDATA=/var/lib/pgsql/14/data postgres@my_vm$ initdb ... Success. You can now start the database server using: /usr/pgsql-14/bin/pg_ctl -D /var/lib/pgsql/14/data -l logfile start
  • 19. How do I Start (or Stop) PostgreSQL?
  • 20. How do I Start Postgres? postgres@my_vm$
  • 21. How do I Start Postgres? https://www.postgresql.org/docs/current/app-pg-ctl.html postgres@my_vm$ pg_ctl -l logfile start
  • 22. How do I Start Postgres? https://www.postgresql.org/docs/current/app-pg-ctl.html postgres@my_vm$ pg_ctl -l logfile start waiting for server to start.... done server started
  • 23. How do I Start Postgres? postgres@my_vm$ ps -ef|grep postgres postgres 4236 1 0 11:59 ? 00:00:00 /usr/pgsql-14/bin/postgres postgres 4237 4236 0 11:59 ? 00:00:00 postgres: logger postgres 4239 4236 0 11:59 ? 00:00:00 postgres: checkpointer postgres 4240 4236 0 11:59 ? 00:00:00 postgres: background writer postgres 4241 4236 0 11:59 ? 00:00:00 postgres: walwriter postgres 4242 4236 0 11:59 ? 00:00:00 postgres: autovacuum launcher postgres 4243 4236 0 11:59 ? 00:00:00 postgres: stats collector postgres 4244 4236 0 11:59 ? 00:00:00 postgres: logical replication launcher List the Postgres processes
  • 24. How do I Stop Postgres? https://www.postgresql.org/docs/current/app-pg-ctl.html postgres@my_vm$ pg_ctl stop
  • 25. How do I Stop Postgres? https://www.postgresql.org/docs/current/app-pg-ctl.html postgres@my_vm$ pg_ctl stop waiting for server to shut down.... done server stopped
  • 26. How do I Control Postgres using systemd? root@my_vm$ systemctl start|stop|restart postgresql-14
  • 27. How do I Control Postgres using systemd? root@my_vm$ systemctl start|stop|restart postgresql-14 root@my_vm$ systemctl status postgresql-14
  • 28. How do I Control Postgres using systemd? root@my_vm$ systemctl start|stop|restart postgresql-14 root@my_vm$ systemctl status postgresql-14 ● postgresql-14.service - PostgreSQL 14 database server Loaded: loaded (/usr/lib/systemd/system/postgresql-14.service; disabled; vendor preset: disabled) Active: active (running) since ...
  • 29. What is a Database?
  • 30. What is a Database? https://www.postgresql.org/docs/current/managing-databases.html ● Topmost hierarchical level for organizing database objects ● Databases are isolated from users’ point of view ● Databases are closely bound from an administrative perspective ● Pre-defined databases: postgres, template0, template1 ● User-defined databases (optional)
  • 31. What is a Database? https://www.postgresql.org/docs/current/managing-databases.html
  • 32. What is a Database? https://www.postgresql.org/docs/current/managing-databases.html
  • 33. What is a Database? https://www.postgresql.org/docs/current/managing-databases.html
  • 34. What is a Database? https://www.postgresql.org/docs/current/managing-databases.html postgres template0 template1 Pre-defined databases
  • 35. Aside: What is psql?
  • 36. Aside: What is psql? https://www.postgresql.org/docs/current/app-psql.html ● Command line tool ● Execute commands and/or scripts against the database • SQL • psql commands: e.g. c to connect to a database d to view table details ● Try it out in your browser at the Crunchy Data Postgres Playground: https://www.crunchydata.com/developers/tutorials
  • 37. How do I Create a Database?
  • 38. How do I Create a Database? https://www.postgresql.org/docs/current/sql-createdatabase.html postgres@my_vm$
  • 39. How do I Create a Database? https://www.postgresql.org/docs/current/sql-createdatabase.html postgres@my_vm$ psql
  • 40. How do I Create a Database? https://www.postgresql.org/docs/current/sql-createdatabase.html postgres@my_vm$ psql psql (14.0) Type "help" for help. postgres=#
  • 41. How do I Create a Database? https://www.postgresql.org/docs/current/sql-createdatabase.html postgres@my_vm$ psql psql (14.0) Type "help" for help. postgres=# CREATE DATABASE my_database;
  • 42. How do I Create a Database? https://www.postgresql.org/docs/current/sql-createdatabase.html postgres@my_vm$ psql psql (14.0) Type "help" for help. postgres=# CREATE DATABASE my_database; CREATE DATABASE
  • 43. How do I List my Databases?
  • 44. How do I List my Databases? postgres=# l
  • 45. How do I List my Databases? postgres=# l List of databases Name | Owner | Encoding | Collate | Ctype | Access privileges -------------+----------+----------+-------------+-------------+----------------------- my_database | postgres | UTF8 | en_US.UTF-8 | en_US.UTF-8 | postgres | postgres | UTF8 | en_US.UTF-8 | en_US.UTF-8 | template0 | postgres | UTF8 | en_US.UTF-8 | en_US.UTF-8 | =c/postgres + | | | | | postgres=CTc/postgres template1 | postgres | UTF8 | en_US.UTF-8 | en_US.UTF-8 | =c/postgres + | | | | | postgres=CTc/postgres (4 rows)
  • 46. How do I List my Databases? postgres=# SELECT datname FROM pg_database; https://www.postgresql.org/docs/current/catalogs.html
  • 47. How do I List my Databases? postgres=# SELECT datname FROM pg_database; datname ----------- my_database postgres template0 template1 (4 rows) https://www.postgresql.org/docs/current/catalogs.html
  • 48. What is a Tablespace?
  • 49. What is a Tablespace? And why should I create one? ● Physical location of the database objects ● Default tablespace pg_default ● Allows control of the disk layout of a PostgreSQL installation ● Accessible to the entire cluster
  • 50. How do I Create a Tablespace?
  • 51. How do I Create a Tablespace? https://www.postgresql.org/docs/current/sql-createtablespace.html postgres@my_vm$
  • 52. How do I Create a Tablespace? https://www.postgresql.org/docs/current/sql-createtablespace.html postgres@my_vm$ mkdir -p /my_tablespaces/tbsp_1 postgres@my_vm$
  • 53. How do I Create a Tablespace? https://www.postgresql.org/docs/current/sql-createtablespace.html postgres@my_vm$ mkdir -p /my_tablespaces/tbsp_1 postgres@my_vm$ psql -c "CREATE TABLESPACE tablespace_1 location '/my_tablespaces/tbsp_1'"
  • 54. How do I Create a Tablespace? https://www.postgresql.org/docs/current/sql-createtablespace.html postgres@my_vm$ mkdir -p /my_tablespaces/tbsp_1 postgres@my_vm$ psql -c "CREATE TABLESPACE tablespace_1 location '/my_tablespaces/tbsp_1'" CREATE TABLESPACE
  • 55. How do I List my Tablespaces?
  • 56. How do I List my Tablespaces? postgres=# db
  • 57. How do I List my Tablespaces? postgres=# db List of tablespaces Name | Owner | Location --------------+----------+------------------------ pg_default | postgres | pg_global | postgres | tablespace_1 | postgres | /my_tablespaces/tbsp_1 (3 rows)
  • 58. How do I List my Tablespaces? postgres=# db+
  • 59. How do I List my Tablespaces? postgres=# db+ List of tablespaces Name | Owner | Location | Access privileges | Options | Size | Description -----------------+----------+------------------------+-------------------+---------+---------+-------------- pg_default | postgres | | | | 33 MB | pg_global | postgres | | | | 560 kB | tablespace_1 | postgres | /my_tablespaces/tbsp_1 | | | 0 bytes | (3 rows)
  • 60. How do I List my Tablespaces? postgres@my_vm$ psql -c "select spcname from pg_tablespace" https://www.postgresql.org/docs/current/catalogs.html
  • 61. How do I List my Tablespaces? postgres@my_vm$ psql -c "select spcname from pg_tablespace" spcname ----------------- pg_default pg_global tablespace_1 (3 rows) https://www.postgresql.org/docs/current/catalogs.html
  • 62. What is a Schema?
  • 63. What is a Schema? Collection of objects within the database ● Logical grouping - no impact on physical location of objects ● Schema and owner of objects need not be the same ● Specific to the database in which it is created ● Namespace
  • 64. How do I Create a Schema?
  • 65. How do I Create a Schema? https://www.postgresql.org/docs/current/sql-createschema.html postgres@my_vm$
  • 66. How do I Create a Schema? https://www.postgresql.org/docs/current/sql-createschema.html postgres@my_vm$ psql my_database -c "CREATE SCHEMA my_schema"
  • 67. How do I Create a Schema? https://www.postgresql.org/docs/current/sql-createschema.html postgres@my_vm$ psql my_database -c "CREATE SCHEMA my_schema" CREATE SCHEMA
  • 68. How do I List my Schemas? my_database=# dn+ List of schemas Name | Owner | Access privileges | Description -----------+----------+-----------------------+------------------------ my_schema | postgres | | public | postgres | postgres=UC/postgres+ | standard public schema | | =UC/postgres | (2 rows)
  • 69. • Database Architecture • Users and Roles • Database Objects • Database Connections • Database Operations and Transactions • WAL • Documentation Agenda
  • 70. What’s the Difference Between a User and a Role?
  • 71. What’s the Difference Between a User and a Role? ● CREATE USER and CREATE ROLE are synonyms except: • CREATE USER: LOGIN by default • CREATE ROLE: NOLOGIN by default ● A role can be considered a user or a group (or both) ● A role is available to the entire cluster ● A role can be granted to another role ● A role can own database objects
  • 72. How do I Create a Role?
  • 73. How do I Create a Role? https://www.postgresql.org/docs/current/sql-createrole.html postgres@my_vm$
  • 74. How do I Create a Role? https://www.postgresql.org/docs/current/sql-createrole.html postgres@my_vm$ psql -c "CREATE ROLE my_role"
  • 75. How do I Create a Role? https://www.postgresql.org/docs/current/sql-createrole.html postgres@my_vm$ psql -c "CREATE ROLE my_role" CREATE ROLE postgres@my_vm$
  • 76. How do I Create a Role? https://www.postgresql.org/docs/current/sql-createrole.html postgres@my_vm$ psql -c "CREATE ROLE my_role" CREATE ROLE postgres@my_vm$ psql -d my_database -U my_role
  • 77. How do I Create a Role? https://www.postgresql.org/docs/current/sql-createrole.html postgres@my_vm$ psql -c "CREATE ROLE my_role" CREATE ROLE postgres@my_vm$ psql -d my_database -U my_role psql: error: connection to server on socket "/var/run/postgresql/.s.PGSQL.5432" failed: FATAL: role "my_role" is not permitted to log in
  • 78. How do I Create a User?
  • 79. How do I Create a User? https://www.postgresql.org/docs/current/sql-createuser.html postgres@my_vm$
  • 80. How do I Create a User? https://www.postgresql.org/docs/current/sql-createuser.html postgres@my_vm$ psql -c "CREATE USER my_user"
  • 81. How do I Create a User Role? https://www.postgresql.org/docs/current/sql-createuser.html postgres@my_vm$ psql -c "CREATE USER my_user" CREATE ROLE
  • 82. How do I Create a User Role? https://www.postgresql.org/docs/current/sql-createuser.html postgres@my_vm$ psql -c "CREATE USER my_user" CREATE ROLE postgres@my_vm$ psql -d my_database -U my_user
  • 83. How do I Create a User Role? https://www.postgresql.org/docs/current/sql-createuser.html postgres@my_vm$ psql -c "CREATE USER my_user" CREATE ROLE postgres@my_vm$ psql -d my_database -U my_user psql (14.0) Type "help" for help.
  • 84. What is a Privilege?
  • 85. What is a Privilege? Permission to perform certain action(s) on given object(s)
  • 86. What is a Privilege? Permission to perform certain action(s) on given object(s) ● Granted by the owner of the object or by a superuser ● Objects : DATABASE, FUNCTION, SCHEMA, TABLE etc ● Permissions: SELECT, INSERT, UPDATE, DELETE, TRUNCATE, CREATE etc ● ALTER DEFAULT PRIVILEGES
  • 87. Granting Privileges https://www.postgresql.org/docs/current/ddl-priv.html postgres@my_vm$ psql -d my_database my_database=# GRANT CREATE ON DATABASE my_database TO my_user; GRANT my_database=# GRANT CREATE ON SCHEMA my_schema TO my_user; GRANT my_database=# GRANT CREATE ON TABLESPACE tablespace_1 TO my_user; GRANT
  • 88. • Database Architecture • Users and Roles • Database Objects • Database Connections • Database Operations and Transactions • WAL • Documentation Agenda
  • 89. Database Objects • View • Materialized View • Sequence • Table • Index • Constraint
  • 90. What is a Table?
  • 91. What is a Table? • A “relation” • Data arranged in columns and rows • Rows are not ordered
  • 92. How do I Create a Table?
  • 93. How do I Create a Table? https://www.postgresql.org/docs/current/sql-createtable.html postgres@my_vm$ psql -d my_database -U my_user my_database=>
  • 94. How do I Create a Table? https://www.postgresql.org/docs/current/sql-createtable.html postgres@my_vm$ psql -d my_database -U my_user my_database=> CREATE TABLE my_schema.dept ( my_database(>
  • 95. How do I Create a Table? https://www.postgresql.org/docs/current/sql-createtable.html postgres@my_vm$ psql -d my_database -U my_user my_database=> CREATE TABLE my_schema.dept ( my_database(> dept_id integer, my_database(> dept_name varchar); my_database->
  • 96. How do I Create a Table? https://www.postgresql.org/docs/current/sql-createtable.html postgres@my_vm$ psql -d my_database -U my_user my_database=> CREATE TABLE my_schema.dept ( my_database(> dept_id integer, my_database(> dept_name varchar); my_database-> TABLESPACE tablespace_1;
  • 97. How do I Create a Table? https://www.postgresql.org/docs/current/sql-createtable.html postgres@my_vm$ psql -d my_database -U my_user my_database=> CREATE TABLE my_schema.dept ( my_database(> dept_id integer, my_database(> dept_name varchar); my_database-> TABLESPACE tablespace_1; CREATE TABLE
  • 98. What is a Sequence?
  • 99. What is a Sequence? Object that generates a sequence of integers ● Specific to a schema ● Used to generate unique numeric identifiers ● Implemented as a single row table
  • 100. How do I Create a Sequence?
  • 101. How do I Create a Sequence? https://www.postgresql.org/docs/current/sql-createsequence.html my_database=>
  • 102. How do I Create a Sequence? https://www.postgresql.org/docs/current/sql-createsequence.html my_database=> CREATE SEQUENCE my_schema.s_dept my_database->
  • 103. How do I Create a Sequence? https://www.postgresql.org/docs/current/sql-createsequence.html my_database=> CREATE SEQUENCE my_schema.s_dept my_database-> INCREMENT BY 1 MINVALUE 1 NO MAXVALUE;
  • 104. How do I Create a Sequence? https://www.postgresql.org/docs/current/sql-createsequence.html my_database=> CREATE SEQUENCE my_schema.s_dept my_database-> INCREMENT BY 1 MINVALUE 1 NO MAXVALUE; CREATE SEQUENCE
  • 105. How do I Create a Sequence? https://www.postgresql.org/docs/current/sql-createsequence.html my_database=> CREATE SEQUENCE my_schema.s_dept my_database-> INCREMENT BY 1 MINVALUE 1 NO MAXVALUE; CREATE SEQUENCE my_database=> ALTER TABLE my_schema.dept my_database->
  • 106. How do I Create a Sequence? https://www.postgresql.org/docs/current/sql-createsequence.html my_database=> CREATE SEQUENCE my_schema.s_dept my_database-> INCREMENT BY 1 MINVALUE 1 NO MAXVALUE; CREATE SEQUENCE my_database=> ALTER TABLE my_schema.dept my_database-> ALTER COLUMN dept_id SET DEFAULT nextval('my_schema.s_dept');
  • 107. How do I Create a Sequence? https://www.postgresql.org/docs/current/sql-createsequence.html my_database=> CREATE SEQUENCE my_schema.s_dept my_database-> INCREMENT BY 1 MINVALUE 1 NO MAXVALUE; CREATE SEQUENCE my_database=> ALTER TABLE my_schema.dept my_database-> ALTER COLUMN dept_id SET DEFAULT nextval('my_schema.s_dept'); ALTER TABLE
  • 108. How do I Auto Generate an ID Column?
  • 109. How do I Auto Generate an ID Column? https://www.postgresql.org/docs/current/sql-createtable.html my_database=> CREATE TABLE my_schema.emp ( my_database(> emp_id integer, my_database(> emp_name varchar, my_database(> dept_id integer) my_database-> TABLESPACE tablespace_1;
  • 110. How do I Auto Generate an ID Column? https://www.postgresql.org/docs/current/sql-createtable.html my_database=> CREATE TABLE my_schema.emp ( my_database(> emp_id integer generated always as identity, my_database(> emp_name varchar, my_database(> dept_id integer) my_database-> TABLESPACE tablespace_1;
  • 111. How do I Auto Generate an ID Column? https://www.postgresql.org/docs/current/sql-createtable.html my_database=> CREATE TABLE my_schema.emp ( my_database(> emp_id integer generated always as identity, my_database(> emp_name varchar, my_database(> dept_id integer) my_database-> TABLESPACE tablespace_1; CREATE TABLE
  • 112. How do I View the Table Definitions?
  • 113. How do I View the Table Definitions? my_database=> d my_schema.dept
  • 114. How do I View the Table Definitions? my_database=> d my_schema.dept Table "my_schema.dept" Column | Type | Collation | Nullable | Default -----------+-------------------+-----------+----------+------------------------------ dept_id | integer | | | nextval('s_dept'::regclass) dept_name | character varying | | | Tablespace: "tablespace_1"
  • 115. How do I View the Table Definitions? my_database=> d my_schema.emp Table "my_schema.emp" Column | Type | Collation | Nullable | Default ----------+-------------------+-----------+----------+------------------------------ emp_id | integer | | not null | generated always as identity emp_name | character varying | | | dept_id | integer | | | Tablespace: "tablespace_1"
  • 116. What is an Index?
  • 117. What is an Index? Ordered list of entries containing a value and a pointer to the table row ● Can create on one or more columns or expressions ● Can speed up searches based on values of column(s) in the index ● Default type is btree ● Various types available in Postgres https://www.postgresql.org/docs/current/indexes-types.html
  • 118. How do I Create an Index?
  • 119. How do I Create an Index? https://www.postgresql.org/docs/current/sql-createindex.html my_database=>
  • 120. How do I Create an Index? https://www.postgresql.org/docs/current/sql-createindex.html my_database=> CREATE INDEX emp_dept_id my_database->
  • 121. How do I Create an Index? https://www.postgresql.org/docs/current/sql-createindex.html my_database=> CREATE INDEX emp_dept_id my_database-> ON my_schema.emp (dept_id) my_database->
  • 122. How do I Create an Index? https://www.postgresql.org/docs/current/sql-createindex.html my_database=> CREATE INDEX emp_dept_id my_database-> ON my_schema.emp (dept_id) my_database-> TABLESPACE tablespace_1;
  • 123. How do I Create an Index? https://www.postgresql.org/docs/current/sql-createindex.html my_database=> CREATE INDEX emp_dept_id my_database-> ON my_schema.emp (dept_id) my_database-> TABLESPACE tablespace_1; CREATE INDEX
  • 124. What is a Constraint?
  • 125. What is a Constraint? https://www.postgresql.org/docs/current/ddl-constraints.html ● NOT NULL constraint ● CHECK constraint ● PRIMARY KEY (PK) constraint ● FOREIGN KEY (FK) constraint
  • 126. What is a NOT NULL Constraint? ● Column must contain a value
  • 127. What is a NOT NULL Constraint? ● Column must contain a value What is a CHECK Constraint? ● Value must conform to certain rules ● for example: dept_name must contain only letters and spaces
  • 128. What is a Primary Key?
  • 129. What is a Primary Key? One or more columns that allow a row in a table to be identified uniquely
  • 130. What is a Primary Key? One or more columns that allow a row in a table to be identified uniquely ● Enforced through PK constraint + unique index ● A table may have only one PK constraint ● The columns in the PK must be NOT NULL
  • 131. How do I Create a Primary Key Constraint?
  • 132. How do I Create a PK Constraint? https://www.postgresql.org/docs/current/sql-altertable.html my_database=>
  • 133. How do I Create a PK Constraint? https://www.postgresql.org/docs/current/sql-altertable.html my_database=> ALTER TABLE my_schema.emp my_database-> ADD CONSTRAINT emp_pk PRIMARY KEY (emp_id) my_database-> USING INDEX TABLESPACE tablespace_1;
  • 134. How do I Create a PK Constraint? https://www.postgresql.org/docs/current/sql-altertable.html my_database=> ALTER TABLE my_schema.emp my_database-> ADD CONSTRAINT emp_pk PRIMARY KEY (emp_id) my_database-> USING INDEX TABLESPACE tablespace_1; ALTER TABLE
  • 135. How do I Create a PK Constraint? https://www.postgresql.org/docs/current/sql-altertable.html my_database=> ALTER TABLE my_schema.emp my_database-> ADD CONSTRAINT emp_pk PRIMARY KEY (emp_id) my_database-> USING INDEX TABLESPACE tablespace_1; ALTER TABLE my_database=> d my_schema.emp
  • 136. How do I Create a PK Constraint? https://www.postgresql.org/docs/current/sql-altertable.html my_database=> ALTER TABLE my_schema.emp my_database-> ADD CONSTRAINT emp_pk PRIMARY KEY (emp_id) my_database-> USING INDEX TABLESPACE tablespace_1; ALTER TABLE my_database=> d my_schema.emp Table "my_schema.emp" Column | Type | Collation | Nullable | Default ----------+-------------------+-----------+----------+------------------------------ emp_id | integer | | not null | generated always as identity emp_name | character varying | | | dept_id | integer | | | Indexes: "emp_pk" PRIMARY KEY, btree (emp_id), tablespace "tablespace_1" "emp_dept_id" btree (dept_id), tablespace "tablespace_1" Tablespace: "tablespace_1"
  • 137. How do I Create a PK Constraint? https://www.postgresql.org/docs/current/sql-altertable.html my_database=> ALTER TABLE my_schema.dept my_database-> ADD CONSTRAINT dept_pk PRIMARY KEY (dept_id) my_database-> USING INDEX TABLESPACE tablespace_1; ALTER TABLE my_database=> d my_schema.dept Table "my_schema.dept" Column | Type | Collation | Nullable | Default -----------+-------------------+-----------+----------+------------------------------ dept_id | integer | | not null | generated always as identity dept_name | character varying | | | Indexes: "dept_pk" PRIMARY KEY, btree (dept_id), tablespace "tablespace_1" Tablespace: "tablespace_1"
  • 138. What is a Foreign Key?
  • 139. What is a Foreign Key? A relationship between two tables
  • 140. What is a Foreign Key? A relationship between a “parent” and a “child” table ● A way to enforce “referential integrity” ● References the primary key or another unique key in the parent table
  • 141. What is a Foreign Key? A relationship between a “parent” and a “child” table ● A way to enforce “referential integrity” ● References the primary key or another unique key in the parent table
  • 142. How do I Create a Foreign Key Constraint?
  • 143. How do I Create a FK Constraint? https://www.postgresql.org/docs/current/sql-altertable.html my_database=>
  • 144. How do I Create a FK Constraint? https://www.postgresql.org/docs/current/sql-altertable.html my_database=> ALTER TABLE my_schema.emp my_database-> ADD CONSTRAINT emp_dept_fk FOREIGN KEY (dept_id) my_database-> REFERENCES my_schema.dept(dept_id);
  • 145. How do I Create a FK Constraint? https://www.postgresql.org/docs/current/sql-altertable.html my_database=> ALTER TABLE my_schema.emp my_database-> ADD CONSTRAINT emp_dept_fk FOREIGN KEY (dept_id) my_database-> REFERENCES my_schema.dept(dept_id); ALTER TABLE
  • 146. How do I Create a FK Constraint? https://www.postgresql.org/docs/current/sql-altertable.html my_database=> ALTER TABLE my_schema.emp my_database-> ADD CONSTRAINT emp_dept_fk FOREIGN KEY (dept_id) my_database-> REFERENCES my_schema.dept(dept_id); ALTER TABLE my_database=> d my_schema.emp ... Foreign-key constraints: "emp_dept_fk" FOREIGN KEY (dept_id) REFERENCES my_schema.dept(dept_id) ...
  • 147. How do I Create a FK Constraint? https://www.postgresql.org/docs/current/sql-altertable.html my_database=> ALTER TABLE my_schema.emp my_database-> ADD CONSTRAINT emp_dept_fk FOREIGN KEY (dept_id) my_database-> REFERENCES my_schema.dept(dept_id); ALTER TABLE my_database=> d my_schema.emp ... Foreign-key constraints: "emp_dept_fk" FOREIGN KEY (dept_id) REFERENCES my_schema.dept(dept_id) ... my_database=> d my_schema.dept ... Referenced by: TABLE "emp" CONSTRAINT "emp_dept_fk" FOREIGN KEY (dept_id) REFERENCES dept(dept_id) ...
  • 148. Aside: What is a Search Path?
  • 149. Aside: What is a Search Path? Schema(s) that will be searched if I don’t refer to an object using a fully-qualified object name my_database=>
  • 150. Aside: What is a Search Path? Schema(s) that will be searched if I don’t refer to an object using a fully-qualified object name my_database=> SHOW search_path;
  • 151. Aside: What is a Search Path? Schema(s) that will be searched if I don’t refer to an object using a fully-qualified object name my_database=> SHOW search_path; search_path ----------------- "$user", public my_database=>
  • 152. Aside: What is a Search Path? Schema(s) that will be searched if I don’t refer to an object using a fully-qualified object name my_database=> SHOW search_path; search_path ----------------- "$user", public my_database=> select * from emp;
  • 153. Aside: What is a Search Path? Schema(s) that will be searched if I don’t refer to an object using a fully-qualified object name my_database=> SHOW search_path; search_path ----------------- "$user", public my_database=> select * from emp; ERROR: relation "emp" does not exist LINE 1: select * from emp ^
  • 154. Aside: What is a Search Path? Schema(s) that will be searched if I don’t refer to an object using a fully-qualified object name my_database=> SET search_path = my_schema;
  • 155. Aside: What is a Search Path? Schema(s) that will be searched if I don’t refer to an object using a fully-qualified object name my_database=> SET search_path = my_schema; SET my_database=>
  • 156. Aside: What is a Search Path? Schema(s) that will be searched if I don’t refer to an object using a fully-qualified object name my_database=> SET search_path = my_schema; SET my_database=> SHOW search_path;
  • 157. Aside: What is a Search Path? Schema(s) that will be searched if I don’t refer to an object using a fully-qualified object name my_database=> SET search_path = my_schema; SET my_database=> SHOW search_path; search_path ------------- my_schema (1 row) my_database=>
  • 158. Aside: What is a Search Path? Schema(s) that will be searched if I don’t refer to an object using a fully-qualified object name my_database=> SET search_path = my_schema; SET my_database=> SHOW search_path; search_path ------------- my_schema (1 row) my_database=> select * from emp;
  • 159. Aside: What is a Search Path? Schema(s) that will be searched if I don’t refer to an object using a fully-qualified object name my_database=> SET search_path = my_schema; SET my_database=> SHOW search_path; search_path ------------- my_schema (1 row) my_database=> select * from emp; emp_id | emp_name | dept_id --------+----------+--------- (0 rows)
  • 160. Aside: What is a Search Path? I can set the search path (and other options) automatically postgres@my_vm$ echo 'set search_path to my_schema' >> ~/.psqlrc postgres@my_vm$
  • 161. Aside: What is a Search Path? I can set the search path (and other options) automatically postgres@my_vm$ echo 'set search_path to my_schema' >> ~/.psqlrc postgres@my_vm$ psql -d my_database -U my_user
  • 162. Aside: What is a Search Path? I can set the search path (and other options) automatically postgres@my_vm$ echo 'set search_path to my_schema' >> ~/.psqlrc postgres@my_vm$ psql -d my_database -U my_user SET psql (14.0) Type "help" for help. my_database=>
  • 163. Aside: What is a Search Path? I can set the search path (and other options) automatically postgres@my_vm$ echo 'set search_path to my_schema' >> ~/.psqlrc postgres@my_vm$ psql -d my_database -U my_user SET psql (14.0) Type "help" for help. my_database=> SHOW search_path;
  • 164. Aside: What is a Search Path? I can set the search path (and other options) automatically postgres@my_vm$ echo 'set search_path to my_schema' >> ~/.psqlrc postgres@my_vm$ psql -d my_database -U my_user SET psql (14.0) Type "help" for help. my_database=> SHOW search_path; search_path ------------- my_schema (1 row)
  • 165. How do I Populate my Tables?
  • 166. How do I Populate my Tables? https://www.postgresql.org/docs/current/sql-insert.html my_database=> INSERT INTO dept (dept_name) my_database-> VALUES ('Sales'),('Consulting'),('Product'),('HR'); INSERT 0 4
  • 167. How do I Populate my Tables? https://www.postgresql.org/docs/current/sql-insert.html my_database=> INSERT INTO dept (dept_name) my_database-> VALUES ('Sales'),('Consulting'),('Product'),('HR'); INSERT 0 4 my_database=> SELECT * FROM dept ORDER BY dept_id; dept_id | dept_name ---------+------------ 1 | Sales 2 | Consulting 3 | Product 4 | HR (4 rows)
  • 168. How do I Populate my Tables? https://www.postgresql.org/docs/current/sql-insert.html my_database=> INSERT INTO emp (emp_name, dept_id) my_database->VALUES ('Ay Bee',2),('Cee Dee',2),('E.F. Gee',1),('Aitch Eye',null),('Jay Kay',4); INSERT 0 4
  • 169. How do I Populate my Tables? https://www.postgresql.org/docs/current/sql-insert.html my_database=> INSERT INTO emp (emp_name, dept_id) my_database->VALUES ('Ay Bee',2),('Cee Dee',2),('E.F. Gee',1),('Aitch Eye',null),('Jay Kay',4); INSERT 0 4 my_database=> SELECT * FROM emp ORDER BY emp_id; emp_id | emp_name | dept_id --------+-----------+--------- 1 | Ay Bee | 2 2 | Cee Dee | 2 3 | E.F. Gee | 1 4 | Aitch Eye | 5 | Jay Kay | 4 (5 rows)
  • 170. What if I try to Insert an Invalid dept_id? https://www.postgresql.org/docs/current/sql-insert.html my_database=> INSERT INTO emp (emp_name, dept_id) my_database-> VALUES ('No Good',7);
  • 171. What if I try to Insert an Invalid dept_id? https://www.postgresql.org/docs/current/sql-insert.html my_database=> INSERT INTO emp (emp_name, dept_id) my_database-> VALUES ('No Good',7); ERROR: insert or update on table "emp" violates foreign key constraint "emp_dept_fk" DETAIL: Key (dept_id)=(7) is not present in table "dept".
  • 172. What is a View?
  • 173. What is a View? A virtual table, based on a query ● Does not take up any space ● Executed in real-time ● Shorthand for a long query ● Present just certain data to certain users
  • 174. How do I Create a View?
  • 175. How do I Create a View? https://www.postgresql.org/docs/current/sql-createview.html my_database=> CREATE VIEW emp_name_view AS my_database-> SELECT emp_name AS "employee_name" FROM emp ORDER BY emp_name;
  • 176. How do I Create a View? https://www.postgresql.org/docs/current/sql-createview.html my_database=> CREATE VIEW emp_name_view AS my_database-> SELECT emp_name AS "employee_name" FROM emp ORDER BY emp_name; CREATE VIEW
  • 177. How do I Create a View? https://www.postgresql.org/docs/current/sql-createview.html my_database=> CREATE VIEW emp_name_view AS my_database-> SELECT emp_name AS "employee_name" FROM emp ORDER BY emp_name; CREATE VIEW my_database=> SELECT * FROM emp_name_view; employee_name --------------- Aitch Eye Ay Bee Cee Dee E.F. Gee Jay Kay (5 rows)
  • 178. What is a Materialized View?
  • 179. What is a Materialized View? ● Query is not executed in real-time ● Can be “refreshed” (query re-executed to gather latest results) ● Useful for aggregating data ● Avoids re-executing long-running/frequently executed queries A table that contains the results of a query
  • 180. How do I Create a Materialized View?
  • 181. How do I Create a Materialized View? https://www.postgresql.org/docs/current/sql-creatematerializedview.html my_database=> CREATE MATERIALIZED VIEW emp_mview my_database-> TABLESPACE tablespace_1 AS my_database-> SELECT emp_id, emp_name AS "employee_name" FROM emp WHERE dept_id in (1,2,3);
  • 182. How do I Create a Materialized View? https://www.postgresql.org/docs/current/sql-creatematerializedview.html my_database=> CREATE MATERIALIZED VIEW emp_mview my_database-> TABLESPACE tablespace_1 AS my_database-> SELECT emp_id, emp_name AS "employee_name" FROM emp WHERE dept_id in (1,2,3); SELECT 3
  • 183. How do I Create a Materialized View? https://www.postgresql.org/docs/current/sql-creatematerializedview.html my_database=> CREATE MATERIALIZED VIEW emp_mview my_database-> TABLESPACE tablespace_1 AS my_database-> SELECT emp_id, emp_name AS "employee_name" FROM emp WHERE dept_id in (1,2,3); SELECT 3 my_database=> SELECT * FROM emp_mview; emp_id | employee_name --------+--------------- 1 | Ay Bee 2 | Cee Dee 3 | E.F. Gee (3 rows)
  • 184. How do I Refresh a Materialized View? First, update some data… my_database=> UPDATE emp SET emp_name = 'CEE DEE' WHERE emp_id = 2; UPDATE 1
  • 185. How do I Refresh a Materialized View? First, update some data… my_database=> UPDATE emp SET emp_name = 'CEE DEE' WHERE emp_id = 2; UPDATE 1 my_database=> SELECT emp_name FROM emp WHERE emp_id = 2; emp_name ---------- CEE DEE
  • 186. How do I Refresh a Materialized View? First, update some data… my_database=> UPDATE emp SET emp_name = 'CEE DEE' WHERE emp_id = 2; UPDATE 1 my_database=> SELECT emp_name FROM emp WHERE emp_id = 2; emp_name ---------- CEE DEE my_database=> SELECT employee_name FROM emp_name_view WHERE employee_name like 'C%'; employee_name --------------- CEE DEE (1 row)
  • 187. How do I Refresh a Materialized View? First, update some data… my_database=>SELECT employee_name FROM emp_mview where emp_id = 2; employee_name --------------- Cee Dee (1 row)
  • 188. How do I Refresh a Materialized View? https://www.postgresql.org/docs/current/sql-refreshmaterializedview.html my_database=>SELECT employee_name FROM emp_mview where emp_id = 2; employee_name --------------- Cee Dee (1 row) my_database=> REFRESH MATERIALIZED VIEW emp_mview; REFRESH MATERIALIZED VIEW
  • 189. How do I Refresh a Materialized View? https://www.postgresql.org/docs/current/sql-refreshmaterializedview.html my_database=>SELECT employee_name FROM emp_mview where emp_id = 2; employee_name --------------- Cee Dee (1 row) my_database=> REFRESH MATERIALIZED VIEW emp_mview; REFRESH MATERIALIZED VIEW my_database=> SELECT employee_name FROM emp_mview where emp_id = 2; employee_name --------------- CEE DEE
  • 190. • Database Architecture • Users and Roles • Database Objects • Database Connections • Database Operations and Transactions • WAL • Documentation Agenda
  • 191. Database Connections ● Information Required ● Connecting Django to Postgres ● Client Tools
  • 192. What Information do I Need? ● host ● port ● database ● username ● password/certificate
  • 193. What Information do I Need? name of database to connect to database host database port (5432 by default) database username Format depends on client tool/driver psql example: psql -h 127.0.0.1 -p 5432 -U myuser mydatabase jdbc example: jdbc:postgresql://myuser@127.0.0.1:5432/mydatabase
  • 194. DATABASES = { 'default': { 'ENGINE': 'django.db.backends.postgresql', 'NAME': 'mydatabase', 'USER': 'myuser', 'HOST': '127.0.0.1', 'PORT': '5432', 'PASSWORD': 'password', } } How do I Configure a Postgres Connection in Django? https://docs.djangoproject.com/en/4.1/ref/databases/#postgresql-notes or define a service in .pg_service.conf settings.py conf file or provide a password (.pgpass) file
  • 195. What Client Tools are Available? Many client tools allow connection to Postgres, including: ● psql ● DBeaver (multi-platform) https://dbeaver.io/ ● pgAdmin4 https://www.pgadmin.org/
  • 196. • Database Architecture • Users and Roles • Database Objects • Database Connections • Database Operations and Transactions • WAL • Documentation Agenda
  • 197. Database Operations and Transactions • SQL, DML, DDL • Join Types • Execution Plans • Transactions • Commit • Rollback
  • 198. What is a Transaction?
  • 199. What is a Transaction? ● Ends with a COMMIT or a ROLLBACK: • COMMIT: operations permanently applied to database • ROLLBACK: operations cancelled ● Changes only visible to other transactions after COMMIT ● Certain locks retained until end of transaction ● ACID (Atomic, Consistent, Isolated, Durable) A single unit of work that consists of one or more operations
  • 200. How do I Begin/End a Transaction?
  • 201. How do I Begin/End a Transaction? ● No need to explicitly BEGIN or COMMIT any transactions ● Each command is a distinct transaction with implicit COMMIT ● To manage a transaction manually: • Issue BEGIN to start a transaction • Execute the operations that comprise the transaction • Issue COMMIT to make the changes permanent or/ • Issue ROLLBACK to undo the changes Example for psql: default behaviour is autocommit
  • 203. What is SQL/DML/DDL? SQL Structured Query Language https://www.postgresql.org/docs/current/sql.html
  • 204. What is SQL/DML/DDL? SQL Structured Query Language DML Data Manipulation Language SELECT, INSERT, UPDATE, DELETE... https://www.postgresql.org/docs/current/sql.html
  • 205. What is SQL/DML/DDL? SQL Structured Query Language DML Data Manipulation Language SELECT, INSERT, UPDATE, DELETE... DDL Data Definition Language CREATE, ALTER, DROP, RENAME ... https://www.postgresql.org/docs/current/sql.html
  • 206. What is a Join?
  • 207. What is a Join? A way to select from multiple tables in one statement
  • 208. What are the Different Types of Join?
  • 209. What are the Different Types of Join? INNER JOIN my_database=> SELECT emp.emp_id, emp.emp_name, dept.dept_name my_database-> FROM emp INNER JOIN dept my_database-> ON emp.dept_id = dept.dept_id;
  • 210. What are the Different Types of Join? INNER JOIN or JOIN (default join type) my_database=> SELECT emp.emp_id, emp.emp_name, dept.dept_name my_database-> FROM emp JOIN dept my_database-> ON emp.dept_id = dept.dept_id;
  • 211. What are the Different Types of Join? INNER JOIN or JOIN default join type my_database=> SELECT emp.emp_id, emp.emp_name, dept.dept_name my_database-> FROM emp JOIN dept my_database-> ON emp.dept_id = dept.dept_id;
  • 212. What are the Different Types of Join? INNER JOIN or JOIN default join type my_database=> SELECT emp.emp_id, emp.emp_name, dept.dept_name my_database-> FROM emp JOIN dept my_database-> ON emp.dept_id = dept.dept_id; emp_id | emp_name | dept_name --------+----------+------------ 1 | Ay Bee | two 2 | Cee Dee | two 3 | E.F. Gee | one 5 | Jay Kay | four (4 rows)
  • 213. What are the Different Types of Join? LEFT OUTER JOIN my_database=> SELECT emp.emp_id, emp.emp_name, dept.dept_name my_database-> FROM emp LEFT OUTER JOIN dept my_database-> ON emp.dept_id = dept.dept_id;
  • 214. What are the Different Types of Join? LEFT OUTER JOIN or LEFT JOIN my_database=> SELECT emp.emp_id, emp.emp_name, dept.dept_name my_database-> FROM emp LEFT JOIN dept my_database-> ON emp.dept_id = dept.dept_id;
  • 215. What are the Different Types of Join? LEFT OUTER JOIN or LEFT JOIN my_database=> SELECT emp.emp_id, emp.emp_name, dept.dept_name my_database-> FROM emp LEFT JOIN dept my_database-> ON emp.dept_id = dept.dept_id;
  • 216. What are the Different Types of Join? LEFT OUTER JOIN or LEFT JOIN my_database=> SELECT emp.emp_id, emp.emp_name, dept.dept_name my_database-> FROM emp LEFT JOIN dept my_database-> ON emp.dept_id = dept.dept_id; emp_id | emp_name | dept_name --------+-----------+------------ 1 | Ay Bee | two 2 | Cee Dee | two 3 | E.F. Gee | one 4 | Aitch Eye | 5 | Jay Kay | four (5 rows)
  • 217. What are the Different Types of Join? RIGHT OUTER JOIN my_database=> SELECT emp.emp_id, emp.emp_name, dept.dept_name my_database-> FROM emp RIGHT OUTER JOIN dept my_database-> ON emp.dept_id = dept.dept_id;
  • 218. What are the Different Types of Join? RIGHT OUTER JOIN or RIGHT JOIN my_database=> SELECT emp.emp_id, emp.emp_name, dept.dept_name my_database-> FROM emp RIGHT JOIN dept my_database-> ON emp.dept_id = dept.dept_id;
  • 219. What are the Different Types of Join? RIGHT OUTER JOIN or RIGHT JOIN my_database=> SELECT emp.emp_id, emp.emp_name, dept.dept_name my_database-> FROM emp RIGHT JOIN dept my_database-> ON emp.dept_id = dept.dept_id;
  • 220. What are the Different Types of Join? RIGHT OUTER JOIN or RIGHT JOIN my_database=> SELECT emp.emp_id, emp.emp_name, dept.dept_name my_database-> FROM emp RIGHT JOIN dept my_database-> ON emp.dept_id = dept.dept_id; emp_id | emp_name | dept_name --------+----------+------------ 1 | Ay Bee | two 2 | Cee Dee | two 3 | E.F. Gee | one 5 | Jay Kay | four | | three (5 rows)
  • 221. What are the Different Types of Join? FULL OUTER JOIN my_database=> SELECT emp.emp_id, emp.emp_name, dept.dept_name my_database-> FROM emp FULL OUTER JOIN dept my_database-> ON emp.dept_id = dept.dept_id;
  • 222. What are the Different Types of Join? FULL OUTER JOIN my_database=> SELECT emp.emp_id, emp.emp_name, dept.dept_name my_database-> FROM emp FULL OUTER JOIN dept my_database-> ON emp.dept_id = dept.dept_id;
  • 223. What are the Different Types of Join? FULL OUTER JOIN my_database=> SELECT emp.emp_id, emp.emp_name, dept.dept_name my_database-> FROM emp FULL OUTER JOIN dept my_database-> ON emp.dept_id = dept.dept_id; emp_id | emp_name | dept_name --------+-----------+------------ 1 | Ay Bee | two 2 | Cee Dee | two 3 | E.F. Gee | one 4 | Aitch Eye | 5 | Jay Kay | four | | three (6 rows)
  • 224. What is an Execution Plan?
  • 225. What is an Execution Plan? ● Access path: full table scan, index lookup ● Join algorithm: nested loop, hash, merge ● Estimated # rows ● Estimated cost The steps that Postgres takes to execute a SQL statement
  • 226. How do I Generate an Execution Plan?
  • 227. How do I Generate an Execution Plan? https://www.postgresql.org/docs/current/using-explain.html my_database=> EXPLAIN SELECT * FROM emp_name_view;
  • 228. How do I Generate an Execution Plan? https://www.postgresql.org/docs/current/using-explain.html my_database=> EXPLAIN SELECT * FROM emp_name_view; QUERY PLAN ---------------------------------------------------------- Sort (cost=1.11..1.12 rows=5 width=32) Sort Key: emp.emp_name -> Seq Scan on emp (cost=0.00..1.05 rows=5 width=32) (3 rows)
  • 229. How do I Generate an Execution Plan? https://www.postgresql.org/docs/current/using-explain.html my_database=> EXPLAIN ANALYZE SELECT * FROM emp_name_view;
  • 230. How do I Generate an Execution Plan? https://www.postgresql.org/docs/current/using-explain.html my_database=> EXPLAIN ANALYZE SELECT * FROM emp_name_view; QUERY PLAN -------------------------------------------------------------------------------------------------------- Sort (cost=83.37..86.37 rows=1200 width=32) (actual time=0.225..0.226 rows=5 loops=1) Sort Key: emp.emp_name Sort Method: quicksort Memory: 25kB -> Seq Scan on emp (cost=0.00..22.00 rows=1200 width=32) (actual time=0.009..0.010 rows=5 loops=1) Planning Time: 0.310 ms Execution Time: 0.434 ms (6 rows)
  • 231. • Database Architecture • Users and Roles • Database Objects • Database Connections • Database Operations and Transactions • WAL • Documentation Agenda
  • 233. What is WAL? ● Write-ahead log ● Details of changes executed against the Postgres cluster ● Allows crash-recovery, online backups and restores ● WAL files can be archived to allow point in time recovery (PITR) ● DON’T DELETE THEM! https://www.postgresql.org/docs/current/wal-intro.html
  • 234. Where are my WAL Files? postgres@my_vm$ ls -ltr $PGDATA/pg_wal
  • 235. Where are my WAL Files? postgres@my_vm$ ls -ltr $PGDATA/pg_wal total 32768 drwx------. 2 postgres postgres 6 Nov 9 11:00 archive_status -rw-------. 1 postgres postgres 16777216 Nov 14 08:41 000000010000000000000001 -rw-------. 1 postgres postgres 16777216 Nov 14 08:41 000000010000000000000002 -rw-------. 1 postgres postgres 16777216 Nov 14 08:46 000000010000000000000003
  • 236. • Database Architecture • Users and Roles • Database Objects • Database Connections • Database Operations and Transactions • Documentation Agenda
  • 237. Where is the PostgreSQL Documentation?
  • 238. PostgreSQL Documentation Current online version: https://www.postgresql.org/docs/current/index.html https://www.postgresql.org/docs
  • 239. Where else can I get information?
  • 240. Slack Channel ● Over 100 channels ● 20k members https://postgres-slack.herokuapp.com/
  • 241. Mailing Lists ● Linked to PostgreSQL Community account ● Many different lists https://lists.postgresql.org/
  • 243. Thank You! Karen Jex | @karenhjex | karen.jex@crunchydata.com Slides: https://karenjex.blogspot.com Keith Fiske | @keithf4 | keith.fiske@crunchydata.com