SlideShare a Scribd company logo

Functional Database Strategies

Jason Swartz, profile picture
Jason Swartz

This document discusses functional database strategies, specifically focusing on immutable rows, window functions, event-sourcing, and database interactions from a functional perspective. The key points covered include: 1. Making database rows immutable by storing them as a log of changes rather than allowing in-place updates. This allows reconstructing the full history. 2. Using window functions to partition and aggregate over row groups to enable querying the latest or historical state. 3. Storing a log of events rather than the current application state, in line with event-sourcing principles. 4. Techniques for interacting with databases in a functional way, including restricting mutations and optimizing data structures.

Engineering
1 of 101
Download to read offline
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
Functional Database Strategies Scalæ by the Bay 2016
YesQL
by Jason Swartz @swartzrock
Functional Database Strategies
Functional Database Strategies
Step One Buy A Functional Database
Silly! Databases Are mutable, not functional!
Well? Aren’tthey?
1. Immutable Rows 2. Window Functions 3. Events, Not State 4. DB Interactions Agenda
1. Immutable Rows 2. Window Functions 3. Events, Not State 4. DB Interactions Agenda
Let’s Talk About Immutable Tables
Functional Database Strategies
Functional Database Strategies
Create-Read- Update-Delete
GET /issues GET /issues/{id} POST /issues PUT /issues/{id} Issue Endpoints
How Do These Events Affect The Database?
1. POST /issues title=’Config ELB’ +------+-------------+------------+----------+-------+ | id | updated | title | assignee | done | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:13 | Config ELB | NULL | false | +------+-------------+------------+----------+-------+
1. POST /issues title=’Config ELB’ 2. PUT /issues/1 assignee=10 +------+-------------+------------+----------+-------+ | id | updated | title | assignee | done | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:16 | Config ELB | 10 | false | +------+-------------+------------+----------+-------+
1. POST /issues title=’Config ELB’ 2. PUT /issues/1 assignee=10 3. PUT /issues/1 done=true +------+-------------+------------+----------+-------+ | id | updated | title | assignee | done | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:24 | Config ELB | NULL | true | +------+-------------+------------+----------+-------+
1. POST /issues title=’Config ELB’ 2. PUT /issues/1 assignee=10 3. PUT /issues/1 done=true +------+-------------+------------+----------+-------+ | id | updated | title | assignee | done | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:24 | Config ELB | NULL | true | +------+-------------+------------+----------+-------+ Not Bad.
1. POST /issues title=’Config ELB’ 2. PUT /issues/1 assignee=10 3. PUT /issues/1 done=true +------+-------------+------------+----------+-------+ | id | updated | title | assignee | done | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:24 | Config ELB | NULL | true | +------+-------------+------------+----------+-------+ Do You Know How We Got Here?
+------+-------------+------------+----------+-------+ | id | updated | title | assignee | done | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:24 | Config ELB | NULL | true | +------+-------------+------------+----------+-------+ Do You Know How We Got Here?
1. POST /issues title=’Config ELB’ 2. PUT /issues/1 assignee=10 3. PUT /issues/1 done=true +------+-------------+------------+----------+-------+ | id | updated | title | assignee | done | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:24 | Config ELB | NULL | true | +------+-------------+------------+----------+-------+ Do You Know How We Got Here?
1. POST /issues title=’Config ELB’ 2. PUT /issues/1 assignee=10 3. PUT /issues/1 done=true +------+-------------+------------+----------+-------+ | id | updated | title | assignee | done | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:24 | Config ELB | NULL | true | +------+-------------+------------+----------+-------+ Why is ‘assignee’ NULL?
Mutable Table Rows Lose History
Immutable Table Rows KeepTheir History
Let’s Try To Lock Down Our State
1. POST /issues title=’Config ELB’ +------+-------------+------------+----------+-------+ | id | updated | title | assignee | done | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:13 | Config ELB | NULL | false | +------+-------------+------------+----------+-------+
1. POST /issues title=’Config ELB’ 2. PUT /issues/1 assignee=10 +------+-------------+------------+----------+-------+ | id | updated | title | assignee | done | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:13 | Config ELB | NULL | false | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:16 | Config ELB | 10 | false | +------+-------------+------------+----------+-------+
1. POST /issues title=’Config ELB’ 2. PUT /issues/1 assignee=10 3. PUT /issues/1 done=true +------+-------------+------------+----------+-------+ | id | updated | title | assignee | done | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:13 | Config ELB | NULL | false | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:16 | Config ELB | 10 | false | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:19 | Config ELB | NULL | false | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:24 | Config ELB | NULL | true | +------+-------------+------------+----------+-------+
1. POST /issues title=’Config ELB’ 2. PUT /issues/1 assignee=10 3. PUT /issues/1 done=true +------+-------------+------------+----------+-------+ | id | updated | title | assignee | done | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:13 | Config ELB | NULL | false | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:16 | Config ELB | 10 | false | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:19 | Config ELB | NULL | false | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:24 | Config ELB | NULL | true | +------+-------------+------------+----------+-------+
1. GET /issues/1 +------+-------------+------------+----------+-------+ | id | updated | title | assignee | done | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:13 | Config ELB | NULL | false | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:16 | Config ELB | 10 | false | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:19 | Config ELB | NULL | false | +------+-------------+------------+----------+-------+ +------+-------------+------------+----------+-------+ | 1 | 09-18 18:24 | Config ELB | NULL | true | +------+-------------+------------+----------+-------+
Tables Are Mutable, But Table Rows Should Be Immutable
In Other Words, Tables Should Be Append-Only
How Do You Make An Append-Only Table?
One: Don’t Let Your DB User Make Changes
Grant select, insert on issues to my-db-user; -- tested on Postgresql
Thank You! Goodbye!
Two: Pick The Right Columns
1. GET /issues/1 +------+-------------+------------+----------+-------+ | id | updated | title | assignee | done | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:13 | Config ELB | NULL | false | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:16 | Config ELB | 10 | false | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:19 | Config ELB | NULL | false | +------+-------------+------------+----------+-------+ +------+-------------+------------+----------+-------+ | 1 | 09-18 18:24 | Config ELB | NULL | true | +------+-------------+------------+----------+-------+
create table issues ( id serial, created timestamp default now(), issue_id int default nextval(‘iseq’), title text, assignee int, done boolean default false )
1. GET /issues/1 +------+-------------+------------+------------+----------+-------+ | id | created | issue_id | title | assignee | done | +------+-------------+------------+------------+----------+-------+ | 1 | 09-18 18:13 | 1 | Config ELB | NULL | false | +------+-------------+------------+------------+----------+-------+ | 2 | 09-18 18:16 | 1 | Config ELB | 10 | false | +------+-------------+------------+------------+----------+-------+ | 3 | 09-18 18:19 | 1 | Config ELB | NULL | false | +------+-------------+------------+------------+----------+-------+ +------+-------------+------------+------------+----------+-------+ | 4 | 09-18 18:24 | 1 | Config ELB | NULL | true | +------+-------------+------------+------------+----------+-------+
select * from issues where issue_id = :issue_id order by id desc limit 1;
That’s The Basics Of Immutability In Table Rows
1. Immutable Rows 2. Window Functions 3. Events, Not State 4. DB Interactions Agenda
1. Immutable Rows 2. Window Functions 3. Events, Not State 4. DB Interactions Agenda
SQL:2003 expands Groups into Windows
Works Great In Postgresql
● Aggregation functions, eg sum(), rank(), avg() ● Window definitions with over() ● Grouping with partition by, order Window Functions
+------+------------+------------+----------+-------+ | id | issue_id | title | assignee | done | +------+------------+------------+----------+-------+ | 1 | 1 | Config ELB | NULL | false | +------+------------+------------+----------+-------+ | 2 | 1 | Config ELB | 10 | false | +------+------------+------------+----------+-------+ | 3 | 2 | Bug to fix | 11 | false | +------+------------+------------+----------+-------+ | 4 | 2 | Bug to fix | 11 | true | +------+------------+------------+----------+-------+
+------+------------+------------+----------+-------+ | id | issue_id | title | assignee | done | +------+------------+------------+----------+-------+ | 1 | 1 | Config ELB | NULL | false | +------+------------+------------+----------+-------+ | 2 | 1 | Config ELB | 10 | false | +------+------------+------------+----------+-------+ | 3 | 2 | Bug to fix | 11 | false | +------+------------+------------+----------+-------+ | 4 | 2 | Bug to fix | 11 | true | +------+------------+------------+----------+-------+
with latest_issues as ( select *, row_number() over ( partition by issue_id order by id desc ) from issues where created > now() - interval '7 day' ) select * from latest_issues where row_number = 1
with latest_issues as ( select *, row_number() over ( partition by issue_id order by id desc ) from issues where created > now() - interval '7 day' ) select * from latest_issues where row_number = 1
with latest_issues as ( select *, row_number() over ( partition by issue_id order by id desc ) from issues where created > now() - interval '7 day' ) select * from latest_issues where row_number = 1
with latest_issues as ( select *, row_number() over ( partition by issue_id order by id desc ) from issues where created > now() - interval '7 day' ) select * from latest_issues where row_number = 1
+------+------------+------------+----------+-------+------------+ | id | issue_id | title | assignee | done | row_number | +------+------------+------------+----------+-------+------------+ | 1 | 1 | Config ELB | NULL | false | 2 | +------+------------+------------+----------+-------+------------+ | 2 | 1 | Config ELB | 10 | false | 1 | +------+------------+------------+----------+-------+------------+ | 3 | 2 | Bug to fix | 11 | false | 2 | +------+------------+------------+----------+-------+------------+ | 4 | 2 | Bug to fix | 11 | true | 1 | +------+------------+------------+----------+-------+------------+
+------+------------+------------+----------+-------+------------+ | id | issue_id | title | assignee | done | row_number | +------+------------+------------+----------+-------+------------+ | 1 | 1 | Config ELB | NULL | false | 2 | +------+------------+------------+----------+-------+------------+ | 2 | 1 | Config ELB | 10 | false | 1 | +------+------------+------------+----------+-------+------------+ | 3 | 2 | Bug to fix | 11 | false | 2 | +------+------------+------------+----------+-------+------------+ | 4 | 2 | Bug to fix | 11 | true | 1 | +------+------------+------------+----------+-------+------------+
+------+------------+------------+----------+-------+------------+ | id | issue_id | title | assignee | done | row_number | +------+------------+------------+----------+-------+------------+ | 1 | 1 | Config ELB | NULL | false | 2 | +------+------------+------------+----------+-------+------------+ | 2 | 1 | Config ELB | 10 | false | 1 | +------+------------+------------+----------+-------+------------+ | 3 | 2 | Bug to fix | 11 | false | 2 | +------+------------+------------+----------+-------+------------+ | 4 | 2 | Bug to fix | 11 | true | 1 | +------+------------+------------+----------+-------+------------+
That Was Window Functions
1. Immutable Rows 2. Window Functions 3. Events, Not State 4. DB Interactions Agenda
1. Immutable Rows 2. Window Functions 3. Events, Not State 4. DB Interactions Agenda
You Know How To Maintain State
Do We Still Need State?
Let’s Talk About Event-Sourcing
1. POST /issues title=’Config ELB’ 2. PUT /issues/1 assignee=10 3. PUT /issues/1 done=true +------+-------------+------------+----------+-------+ | id | updated | title | assignee | done | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:13 | Config ELB | NULL | false | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:16 | Config ELB | 10 | false | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:24 | Config ELB | 10 | true | +------+-------------+------------+----------+-------+
1. POST /issues title=’Config ELB’ 2. PUT /issues/1 assignee=10 3. PUT /issues/1 done=true +------+-------------+------------+----------+-------+ | id | updated | title | assignee | done | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:13 | Config ELB | NULL | false | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:16 | Config ELB | 10 | false | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:24 | Config ELB | 10 | true | +------+-------------+------------+----------+-------+ Events States
Functional Database Strategies
Functional Database Strategies
Functional Database Strategies
Functional Database Strategies
Now We’re Storing Events, Not States
create table issue_events ( id serial, created timestamp default now(), issue_id int default nextval(‘iseq’), originator text, payload text )
1. POST /issue/1/event ‘Originator: 4a48239-8a..’ payload=’<Update val=”done=true”>’ +----+-------------+----------+------------+---------+ | id | created | issue_id | originator | payload | +----+-------------+----------+------------+---------+ | 14 | 09-18 18:50 | 1 | 4a482... | <...> | +----+-------------+----------+------------+---------+
Create Events And Simulate The State
1. Create-Issue Issue(“Config ELB”, null, false); Real Events Virtual States
1. Create-Issue 2. Assign-Issue Issue(“Config ELB”, 10, false); Real Events Virtual States
1. Create-Issue 2. Assign-Issue 3. Complete-Issue Issue(“Config ELB”, 10, true); Real Events Virtual States
So Why Use Event-Sourcing?
1. High Write Performance 2. Potential for Command/Query Separation 3. Auditable 4. Replayable 5. Undo-able 6. Monitorable Reasons For Event-Sourcing
It’s Like Having Control Over The Versions Of Your State Changes
It’s Like Having Control Over The Versions Of Your Data
It’s Like Git For Your Data
1. Frankly, It’s Weird 2. Requires Events. No Events, No Event-Sourcing. 3. As Of November 2016, It’s Still Non-Standard Reasons Against Event-Sourcing
Wait! We’re Scala developers!
Who Cares About Being Non-Standard?
That About Sums Up Event Sourcing
1. Immutable Rows 2. Window Functions 3. Events, Not State 4. DB Interactions Agenda
1. Immutable Rows 2. Window Functions 3. Events, Not State 4. DB Interactions Agenda
Remember That Your Database is mutable.
Avoid Sharing Your State
● Avoid shared mutable SESSIONS ● Avoid shared mutable CURSORS ● Mutating state? Cool! But MODEL IT FIRST ● Execute state changes at THE EDGE Safe Database Interactions
Doobie A Typelevel Project
Are these steps? Or a Monad?
Functional Database Strategies
That About Sums Up Database Interactions
Okay, Actually That’s The Entire Talk Unless There’s More Time
Functional Database Strategies Scalæ by the Bay 2016
by Jason Swartz @swartzrock
Thank You For Attending
Fin
THIS SPACE LEFT BLANK

More Related Content

PPT
GC free coding in @Java presented @Geecon
Peter Lawrey
 
PDF
Functional Database Strategies at Scala Bay
Jason Swartz
 
PDF
Big-Data-Analysen mit MariaDB ColumnStore
MariaDB plc
 
PDF
Java 8 - An Introduction by Jason Swartz
Jason Swartz
 
PDF
Continuous load testing
codepitbull
 
PDF
How MySQL can boost (or kill) your application
Federico Razzoli
 
PPTX
Unit 2 DBMS.pptx
ssuserc8e1481
 
PPTX
Data modeling tips from the trenches
Terry Bunio
 
GC free coding in @Java presented @Geecon
Peter Lawrey
 
Functional Database Strategies at Scala Bay
Jason Swartz
 
Big-Data-Analysen mit MariaDB ColumnStore
MariaDB plc
 
Java 8 - An Introduction by Jason Swartz
Jason Swartz
 
Continuous load testing
codepitbull
 
How MySQL can boost (or kill) your application
Federico Razzoli
 
Unit 2 DBMS.pptx
ssuserc8e1481
 
Data modeling tips from the trenches
Terry Bunio
 

Similar to Functional Database Strategies (20)

PDF
PHPDay 2019 - MySQL 8, not only good, great!
Gabriela Ferrara
 
PDF
Database Design most common pitfalls
Federico Razzoli
 
PDF
Constraints: A Developer's Secret Weapon | pgDay Paris 2018 | Will Leinweber
Citus Data
 
PDF
Data Modeling, Normalization, and Denormalisation | PostgreSQL Conference Eur...
Citus Data
 
PDF
MySQL 8.0: not only good, it’s GREAT! - PHP UK 2019
Gabriela Ferrara
 
PDF
Runaway complexity in Big Data... and a plan to stop it
nathanmarz
 
PDF
Constraints: A Developer's Secret Weapon | PostgreSQL Conference EU 2018 | Wi...
Citus Data
 
PDF
Data Modeling, Normalization, and De-Normalization | PostgresOpen 2019 | Dimi...
Citus Data
 
PPTX
SQL Database Design For Developers at PhpTek 2025.pptx
Scott Keck-Warren
 
PDF
How MySQL can boost (or kill) your application v2
Federico Razzoli
 
PPTX
CS 542 Database Index Structures
J Singh
 
PPTX
DBMS DAY-6666666666666666666666666666666666666666.pptx
imaieabhinaw
 
PDF
Microservices Tutorial Session at JavaOne 2016
Jason Swartz
 
PDF
Duplicaterows
oracle documents
 
PDF
Coding Potpourri: MySQL
Nicole C. Engard
 
PPTX
Avoiding common database pitfalls
Derek Binkley
 
PPTX
PostgreSQL Database Slides
metsarin
 
PDF
Ijetr012023
ER Publication.org
 
PPTX
Ch07_SQL- The PostgreSQL Wkkkkkkkkay.pptx
MhmdMk10
 
PPTX
Relational Database Management System part II
KavithaA19
 
PHPDay 2019 - MySQL 8, not only good, great!
Gabriela Ferrara
 
Database Design most common pitfalls
Federico Razzoli
 
Constraints: A Developer's Secret Weapon | pgDay Paris 2018 | Will Leinweber
Citus Data
 
Data Modeling, Normalization, and Denormalisation | PostgreSQL Conference Eur...
Citus Data
 
MySQL 8.0: not only good, it’s GREAT! - PHP UK 2019
Gabriela Ferrara
 
Runaway complexity in Big Data... and a plan to stop it
nathanmarz
 
Constraints: A Developer's Secret Weapon | PostgreSQL Conference EU 2018 | Wi...
Citus Data
 
Data Modeling, Normalization, and De-Normalization | PostgresOpen 2019 | Dimi...
Citus Data
 
SQL Database Design For Developers at PhpTek 2025.pptx
Scott Keck-Warren
 
How MySQL can boost (or kill) your application v2
Federico Razzoli
 
CS 542 Database Index Structures
J Singh
 
DBMS DAY-6666666666666666666666666666666666666666.pptx
imaieabhinaw
 
Microservices Tutorial Session at JavaOne 2016
Jason Swartz
 
Duplicaterows
oracle documents
 
Coding Potpourri: MySQL
Nicole C. Engard
 
Avoiding common database pitfalls
Derek Binkley
 
PostgreSQL Database Slides
metsarin
 
Ijetr012023
ER Publication.org
 
Ch07_SQL- The PostgreSQL Wkkkkkkkkay.pptx
MhmdMk10
 
Relational Database Management System part II
KavithaA19
 
Ad

More from Jason Swartz (7)

PPTX
High Performance Serverless Functions in Scala
Jason Swartz
 
PDF
Everyone's Guide to States, Events and Async-Messaging for Microservices
Jason Swartz
 
PDF
Everyone's guide to event sourcing and async-messaging
Jason Swartz
 
PDF
Enterprise APIs With Ease - Scala Developers of Barcelona
Jason Swartz
 
PDF
Build Enterprise APIs WIth Ease (And Scala)
Jason Swartz
 
PDF
OSCON - Get Started Developing With Scala
Jason Swartz
 
PDF
APICon SF - Enterprise APIs With Ease
Jason Swartz
 
High Performance Serverless Functions in Scala
Jason Swartz
 
Everyone's Guide to States, Events and Async-Messaging for Microservices
Jason Swartz
 
Everyone's guide to event sourcing and async-messaging
Jason Swartz
 
Enterprise APIs With Ease - Scala Developers of Barcelona
Jason Swartz
 
Build Enterprise APIs WIth Ease (And Scala)
Jason Swartz
 
OSCON - Get Started Developing With Scala
Jason Swartz
 
APICon SF - Enterprise APIs With Ease
Jason Swartz
 
Ad

Recently uploaded (20)

PDF
TFEC-4-2020-Design-Guide-for-Timber-Roof-Trusses.pdf
AinieButt1
 
PDF
Evaluating the Democratization of the Turkish Armed Forces from a Normative P...
jpsjournal1
 
PPTX
Internet of Things (IOT) - A guide to understanding
Davies Chacko
 
PPTX
CH1 Production IntroductoryConcepts.pptx
RacemMellouli1
 
PDF
SM_6th-Sem__Cse_Internet-of-Things.pdf IOT
smceramu
 
PDF
Unit I ESSENTIAL OF DIGITAL MARKETING.pdf
Nitin Shelake
 
PDF
July 2025 - Top 10 Read Articles in International Journal of Software Enginee...
sebastianku31
 
PDF
Embodied AI: Ushering in the Next Era of Intelligent Systems
Yu Huang
 
PPTX
Sustainable Sites - Green Building Construction
mhdumerali
 
PDF
composite construction of structures.pdf
AinieButt1
 
DOCX
573137875-Attendance-Management-System-original
AYUSHMANAV
 
PPTX
web development for engineering and engineering
keshriji700
 
PPTX
Artificial Intelligence
dikshendrasarpate2
 
PDF
Mitigating Risks through Effective Management for Enhancing Organizational Pe...
IJAEMSJORNAL
 
PPTX
additive manufacturing of ss316l using mig welding
premcdr7799
 
PPT
Mechanical Engineering MATERIALS Selection
arsalanahmad705384
 
PPTX
Geodesy 1.pptx...............................................
abhi1361yadav
 
PPTX
CARTOGRAPHY AND GEOINFORMATION VISUALIZATION chapter1 NPTE (2).pptx
abhi1361yadav
 
PPTX
Lecture Notes Electrical Wiring System Components
eedgecbhojpur
 
PDF
BMEC211 - INTRODUCTION TO MECHATRONICS-1.pdf
ryankakungu
 
TFEC-4-2020-Design-Guide-for-Timber-Roof-Trusses.pdf
AinieButt1
 
Evaluating the Democratization of the Turkish Armed Forces from a Normative P...
jpsjournal1
 
Internet of Things (IOT) - A guide to understanding
Davies Chacko
 
CH1 Production IntroductoryConcepts.pptx
RacemMellouli1
 
SM_6th-Sem__Cse_Internet-of-Things.pdf IOT
smceramu
 
Unit I ESSENTIAL OF DIGITAL MARKETING.pdf
Nitin Shelake
 
July 2025 - Top 10 Read Articles in International Journal of Software Enginee...
sebastianku31
 
Embodied AI: Ushering in the Next Era of Intelligent Systems
Yu Huang
 
Sustainable Sites - Green Building Construction
mhdumerali
 
composite construction of structures.pdf
AinieButt1
 
573137875-Attendance-Management-System-original
AYUSHMANAV
 
web development for engineering and engineering
keshriji700
 
Artificial Intelligence
dikshendrasarpate2
 
Mitigating Risks through Effective Management for Enhancing Organizational Pe...
IJAEMSJORNAL
 
additive manufacturing of ss316l using mig welding
premcdr7799
 
Mechanical Engineering MATERIALS Selection
arsalanahmad705384
 
Geodesy 1.pptx...............................................
abhi1361yadav
 
CARTOGRAPHY AND GEOINFORMATION VISUALIZATION chapter1 NPTE (2).pptx
abhi1361yadav
 
Lecture Notes Electrical Wiring System Components
eedgecbhojpur
 
BMEC211 - INTRODUCTION TO MECHATRONICS-1.pdf
ryankakungu
 

Functional Database Strategies

  • 6. Step One Buy A Functional Database
  • 9. 1. Immutable Rows 2. Window Functions 3. Events, Not State 4. DB Interactions Agenda
  • 10. 1. Immutable Rows 2. Window Functions 3. Events, Not State 4. DB Interactions Agenda
  • 15. GET /issues GET /issues/{id} POST /issues PUT /issues/{id} Issue Endpoints
  • 16. How Do These Events Affect The Database?
  • 17. 1. POST /issues title=’Config ELB’ +------+-------------+------------+----------+-------+ | id | updated | title | assignee | done | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:13 | Config ELB | NULL | false | +------+-------------+------------+----------+-------+
  • 18. 1. POST /issues title=’Config ELB’ 2. PUT /issues/1 assignee=10 +------+-------------+------------+----------+-------+ | id | updated | title | assignee | done | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:16 | Config ELB | 10 | false | +------+-------------+------------+----------+-------+
  • 19. 1. POST /issues title=’Config ELB’ 2. PUT /issues/1 assignee=10 3. PUT /issues/1 done=true +------+-------------+------------+----------+-------+ | id | updated | title | assignee | done | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:24 | Config ELB | NULL | true | +------+-------------+------------+----------+-------+
  • 20. 1. POST /issues title=’Config ELB’ 2. PUT /issues/1 assignee=10 3. PUT /issues/1 done=true +------+-------------+------------+----------+-------+ | id | updated | title | assignee | done | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:24 | Config ELB | NULL | true | +------+-------------+------------+----------+-------+ Not Bad.
  • 21. 1. POST /issues title=’Config ELB’ 2. PUT /issues/1 assignee=10 3. PUT /issues/1 done=true +------+-------------+------------+----------+-------+ | id | updated | title | assignee | done | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:24 | Config ELB | NULL | true | +------+-------------+------------+----------+-------+ Do You Know How We Got Here?
  • 22. +------+-------------+------------+----------+-------+ | id | updated | title | assignee | done | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:24 | Config ELB | NULL | true | +------+-------------+------------+----------+-------+ Do You Know How We Got Here?
  • 23. 1. POST /issues title=’Config ELB’ 2. PUT /issues/1 assignee=10 3. PUT /issues/1 done=true +------+-------------+------------+----------+-------+ | id | updated | title | assignee | done | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:24 | Config ELB | NULL | true | +------+-------------+------------+----------+-------+ Do You Know How We Got Here?
  • 24. 1. POST /issues title=’Config ELB’ 2. PUT /issues/1 assignee=10 3. PUT /issues/1 done=true +------+-------------+------------+----------+-------+ | id | updated | title | assignee | done | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:24 | Config ELB | NULL | true | +------+-------------+------------+----------+-------+ Why is ‘assignee’ NULL?
  • 27. Let’s Try To Lock Down Our State
  • 28. 1. POST /issues title=’Config ELB’ +------+-------------+------------+----------+-------+ | id | updated | title | assignee | done | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:13 | Config ELB | NULL | false | +------+-------------+------------+----------+-------+
  • 29. 1. POST /issues title=’Config ELB’ 2. PUT /issues/1 assignee=10 +------+-------------+------------+----------+-------+ | id | updated | title | assignee | done | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:13 | Config ELB | NULL | false | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:16 | Config ELB | 10 | false | +------+-------------+------------+----------+-------+
  • 30. 1. POST /issues title=’Config ELB’ 2. PUT /issues/1 assignee=10 3. PUT /issues/1 done=true +------+-------------+------------+----------+-------+ | id | updated | title | assignee | done | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:13 | Config ELB | NULL | false | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:16 | Config ELB | 10 | false | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:19 | Config ELB | NULL | false | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:24 | Config ELB | NULL | true | +------+-------------+------------+----------+-------+
  • 31. 1. POST /issues title=’Config ELB’ 2. PUT /issues/1 assignee=10 3. PUT /issues/1 done=true +------+-------------+------------+----------+-------+ | id | updated | title | assignee | done | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:13 | Config ELB | NULL | false | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:16 | Config ELB | 10 | false | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:19 | Config ELB | NULL | false | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:24 | Config ELB | NULL | true | +------+-------------+------------+----------+-------+
  • 32. 1. GET /issues/1 +------+-------------+------------+----------+-------+ | id | updated | title | assignee | done | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:13 | Config ELB | NULL | false | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:16 | Config ELB | 10 | false | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:19 | Config ELB | NULL | false | +------+-------------+------------+----------+-------+ +------+-------------+------------+----------+-------+ | 1 | 09-18 18:24 | Config ELB | NULL | true | +------+-------------+------------+----------+-------+
  • 33. Tables Are Mutable, But Table Rows Should Be Immutable
  • 34. In Other Words, Tables Should Be Append-Only
  • 35. How Do You Make An Append-Only Table?
  • 36. One: Don’t Let Your DB User Make Changes
  • 37. Grant select, insert on issues to my-db-user; -- tested on Postgresql
  • 39. Two: Pick The Right Columns
  • 40. 1. GET /issues/1 +------+-------------+------------+----------+-------+ | id | updated | title | assignee | done | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:13 | Config ELB | NULL | false | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:16 | Config ELB | 10 | false | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:19 | Config ELB | NULL | false | +------+-------------+------------+----------+-------+ +------+-------------+------------+----------+-------+ | 1 | 09-18 18:24 | Config ELB | NULL | true | +------+-------------+------------+----------+-------+
  • 41. create table issues ( id serial, created timestamp default now(), issue_id int default nextval(‘iseq’), title text, assignee int, done boolean default false )
  • 42. 1. GET /issues/1 +------+-------------+------------+------------+----------+-------+ | id | created | issue_id | title | assignee | done | +------+-------------+------------+------------+----------+-------+ | 1 | 09-18 18:13 | 1 | Config ELB | NULL | false | +------+-------------+------------+------------+----------+-------+ | 2 | 09-18 18:16 | 1 | Config ELB | 10 | false | +------+-------------+------------+------------+----------+-------+ | 3 | 09-18 18:19 | 1 | Config ELB | NULL | false | +------+-------------+------------+------------+----------+-------+ +------+-------------+------------+------------+----------+-------+ | 4 | 09-18 18:24 | 1 | Config ELB | NULL | true | +------+-------------+------------+------------+----------+-------+
  • 43. select * from issues where issue_id = :issue_id order by id desc limit 1;
  • 44. That’s The Basics Of Immutability In Table Rows
  • 45. 1. Immutable Rows 2. Window Functions 3. Events, Not State 4. DB Interactions Agenda
  • 46. 1. Immutable Rows 2. Window Functions 3. Events, Not State 4. DB Interactions Agenda
  • 49. ● Aggregation functions, eg sum(), rank(), avg() ● Window definitions with over() ● Grouping with partition by, order Window Functions
  • 50. +------+------------+------------+----------+-------+ | id | issue_id | title | assignee | done | +------+------------+------------+----------+-------+ | 1 | 1 | Config ELB | NULL | false | +------+------------+------------+----------+-------+ | 2 | 1 | Config ELB | 10 | false | +------+------------+------------+----------+-------+ | 3 | 2 | Bug to fix | 11 | false | +------+------------+------------+----------+-------+ | 4 | 2 | Bug to fix | 11 | true | +------+------------+------------+----------+-------+
  • 51. +------+------------+------------+----------+-------+ | id | issue_id | title | assignee | done | +------+------------+------------+----------+-------+ | 1 | 1 | Config ELB | NULL | false | +------+------------+------------+----------+-------+ | 2 | 1 | Config ELB | 10 | false | +------+------------+------------+----------+-------+ | 3 | 2 | Bug to fix | 11 | false | +------+------------+------------+----------+-------+ | 4 | 2 | Bug to fix | 11 | true | +------+------------+------------+----------+-------+
  • 52. with latest_issues as ( select *, row_number() over ( partition by issue_id order by id desc ) from issues where created > now() - interval '7 day' ) select * from latest_issues where row_number = 1
  • 53. with latest_issues as ( select *, row_number() over ( partition by issue_id order by id desc ) from issues where created > now() - interval '7 day' ) select * from latest_issues where row_number = 1
  • 54. with latest_issues as ( select *, row_number() over ( partition by issue_id order by id desc ) from issues where created > now() - interval '7 day' ) select * from latest_issues where row_number = 1
  • 55. with latest_issues as ( select *, row_number() over ( partition by issue_id order by id desc ) from issues where created > now() - interval '7 day' ) select * from latest_issues where row_number = 1
  • 56. +------+------------+------------+----------+-------+------------+ | id | issue_id | title | assignee | done | row_number | +------+------------+------------+----------+-------+------------+ | 1 | 1 | Config ELB | NULL | false | 2 | +------+------------+------------+----------+-------+------------+ | 2 | 1 | Config ELB | 10 | false | 1 | +------+------------+------------+----------+-------+------------+ | 3 | 2 | Bug to fix | 11 | false | 2 | +------+------------+------------+----------+-------+------------+ | 4 | 2 | Bug to fix | 11 | true | 1 | +------+------------+------------+----------+-------+------------+
  • 57. +------+------------+------------+----------+-------+------------+ | id | issue_id | title | assignee | done | row_number | +------+------------+------------+----------+-------+------------+ | 1 | 1 | Config ELB | NULL | false | 2 | +------+------------+------------+----------+-------+------------+ | 2 | 1 | Config ELB | 10 | false | 1 | +------+------------+------------+----------+-------+------------+ | 3 | 2 | Bug to fix | 11 | false | 2 | +------+------------+------------+----------+-------+------------+ | 4 | 2 | Bug to fix | 11 | true | 1 | +------+------------+------------+----------+-------+------------+
  • 58. +------+------------+------------+----------+-------+------------+ | id | issue_id | title | assignee | done | row_number | +------+------------+------------+----------+-------+------------+ | 1 | 1 | Config ELB | NULL | false | 2 | +------+------------+------------+----------+-------+------------+ | 2 | 1 | Config ELB | 10 | false | 1 | +------+------------+------------+----------+-------+------------+ | 3 | 2 | Bug to fix | 11 | false | 2 | +------+------------+------------+----------+-------+------------+ | 4 | 2 | Bug to fix | 11 | true | 1 | +------+------------+------------+----------+-------+------------+
  • 60. 1. Immutable Rows 2. Window Functions 3. Events, Not State 4. DB Interactions Agenda
  • 61. 1. Immutable Rows 2. Window Functions 3. Events, Not State 4. DB Interactions Agenda
  • 62. You Know How To Maintain State
  • 63. Do We Still Need State?
  • 65. 1. POST /issues title=’Config ELB’ 2. PUT /issues/1 assignee=10 3. PUT /issues/1 done=true +------+-------------+------------+----------+-------+ | id | updated | title | assignee | done | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:13 | Config ELB | NULL | false | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:16 | Config ELB | 10 | false | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:24 | Config ELB | 10 | true | +------+-------------+------------+----------+-------+
  • 66. 1. POST /issues title=’Config ELB’ 2. PUT /issues/1 assignee=10 3. PUT /issues/1 done=true +------+-------------+------------+----------+-------+ | id | updated | title | assignee | done | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:13 | Config ELB | NULL | false | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:16 | Config ELB | 10 | false | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:24 | Config ELB | 10 | true | +------+-------------+------------+----------+-------+ Events States
  • 72. create table issue_events ( id serial, created timestamp default now(), issue_id int default nextval(‘iseq’), originator text, payload text )
  • 73. 1. POST /issue/1/event ‘Originator: 4a48239-8a..’ payload=’<Update val=”done=true”>’ +----+-------------+----------+------------+---------+ | id | created | issue_id | originator | payload | +----+-------------+----------+------------+---------+ | 14 | 09-18 18:50 | 1 | 4a482... | <...> | +----+-------------+----------+------------+---------+
  • 75. 1. Create-Issue Issue(“Config ELB”, null, false); Real Events Virtual States
  • 76. 1. Create-Issue 2. Assign-Issue Issue(“Config ELB”, 10, false); Real Events Virtual States
  • 77. 1. Create-Issue 2. Assign-Issue 3. Complete-Issue Issue(“Config ELB”, 10, true); Real Events Virtual States
  • 79. 1. High Write Performance 2. Potential for Command/Query Separation 3. Auditable 4. Replayable 5. Undo-able 6. Monitorable Reasons For Event-Sourcing
  • 80. It’s Like Having Control Over The Versions Of Your State Changes
  • 81. It’s Like Having Control Over The Versions Of Your Data
  • 82. It’s Like Git For Your Data
  • 83. 1. Frankly, It’s Weird 2. Requires Events. No Events, No Event-Sourcing. 3. As Of November 2016, It’s Still Non-Standard Reasons Against Event-Sourcing
  • 85. Who Cares About Being Non-Standard?
  • 86. That About Sums Up Event Sourcing
  • 87. 1. Immutable Rows 2. Window Functions 3. Events, Not State 4. DB Interactions Agenda
  • 88. 1. Immutable Rows 2. Window Functions 3. Events, Not State 4. DB Interactions Agenda
  • 91. ● Avoid shared mutable SESSIONS ● Avoid shared mutable CURSORS ● Mutating state? Cool! But MODEL IT FIRST ● Execute state changes at THE EDGE Safe Database Interactions
  • 93. Are these steps? Or a Monad?
  • 95. That About Sums Up Database Interactions
  • 96. Okay, Actually That’s The Entire Talk Unless There’s More Time
  • 100. Fin