PostgreSQL Replication High Availability Methods
1 like1,177 views
The document discusses PostgreSQL replication and high availability (HA) methods, including types like synchronous and asynchronous replication, as well as physical and logical replication. It outlines the importance of replication in maintaining database availability and performance, detailing various architectures and tools such as Patroni and repmgr for effective management. Additionally, it covers essential terminologies and practices related to replication and HA frameworks.
PostgreSQL Replication High Availability Methods
- 1. PostgreSQL Replication/HA Methods Aakash M DBRE, Mydbops January 30, 2021 Mydbops Database Meetup -8
- 2. Database Reliability Engineer Interest in PostgreSQL and MySQL Active Blogger Aakash M About Me
- 3. Database Consulting Services Managed Database Services Focuses on Top Opensource database MySQL, MongoDB and PostgreSQL ON Premises and Cloud Mydbops Services
- 4. End of Life on February 11, 2021 Current Major Release is 13 AWS - March 16, 2021 PostgreSQL 9.5
- 5. Modes of Replication Replication Agenda Replication Types How to achieve HA
- 6. Replication
- 7. Continuous copying of data changes from one database server to another. What is Replication? Application Changes Replicate Database Servers
- 8. High Availabilty Why do we need it? Application S Q L Database Servers
- 9. Load Balancing Why do we need it? Application S Q L Database Servers S Q L 50% 50%
- 10. Terminologies
- 11. Terminologies Write Ahead Log (WAL) Every transactions -> Transaction Log Each statement creates a new WAL record Each WAL record contains a bunch of records 16MB by default pg_xlog/pg_wal WAL Records WAL File
- 12. Terminologies Write Ahead Log (WAL) Crash Recovery Point-In-Time-Recovery(PITR) Replication
- 13. Terminologies Primary Server / Master Direct Writes Application Primary Server Writes
- 14. Terminologies Standby / Secondary Server Following up the primary and applying the changes Application Primary Server Writes Secondary Server
- 15. Terminologies Warm / Hot Standby Server Standby Warm Standby Hot Standby
- 16. Failover Switching to the backup server(standby) when the primary server is not available Terminologies Application S Q L Primary Server Standby Server
- 19. Asynchronous Replication Transaction complete - considering only the primary server Never consider its replicas Can be out of sync
- 20. Asynchronous Replication Time Secondary - II Secondary - I Primary Replication Lag Replication Lag Ins/Upd/Del X Y Z
- 21. Asynchronous Replication No Additional Overhead Chances for dataloss
- 22. Synchronous Replication Transaction Complete - when changes have been replicated to the replicas. Replicas should be available all the time Replicas will be in sync
- 23. Synchronous Replication Time Secondary - II Secondary - I Primary Ins/Upd/Del X
- 24. Synchronous Replication No data loss Additional Overhead
- 26. Types of Replication Replication Physical Logical Log Shipping Streaming Replication Publisher Subscriber
- 27. Physical Vs Logical Replication Physical Replication Primary - at offset 11 of disk page 5 of relation 14312, add a row with hex value 0x234... Secondaries - at offset 11 of disk page 5 of relation 14312, add a row with hex value 0x234... Disk Block Level changes made by PostgreSQL An exact copy of the Primary Example:
- 28. Physical Vs Logical Replication Logical Replication Primary - INSERT INTO test1 VALUES (now()); Secondaries - INSERT INTO test1 VALUES (2021-01-31); Data getting in and is replicated Example:
- 29. Physical Replication Log Shipping Streaming Replication
- 30. Physical Replication - Log Shipping PostgreSQL 8.2 Replication based on the archival archive_command on the primary restore_command on the standby File Based
- 31. Physical Replication - Log Shipping pgxlog/ pg_wal 2920514960194996 NFS / Archive Server Primary Server Standby Server Archive Command Restore Command WAL File
- 32. Physical Replication - Log Shipping No direct connection No replication user Replica always be in lag No support for synchronous mode
- 33. Physical Replication - Streaming Replication PostgreSQL 9.0 Stream WAL files to the standbys primary_conninfo in recovery.conf Block Based
- 34. Physical Replication - Streaming Replication WAL Receiver - on Standbys Startup - on Standbys WAL Sender - on Primary
- 35. Physical Replication - Streaming Replication Start Connection primary_conninfo Authentication Password Response Authentication Done Identify System and start to stream WAL System Identified and send WAL files Standby WAL Receiver Primary WAL Sender
- 36. Physical Replication - Streaming Replication Primary Server Secondary Server WAL Records WAL Records WAL Records WAL Receiver WAL Sender
- 37. Physical Replication - Streaming Replication Replication Lag can be reduced Supports Synchronous Replication Used for Failover Requires direct connection Requires DB user
- 38. Physical Replication - Streaming Replication / Log Shipping Same PostgreSQL Version Same Architecture Entire Dataset No writes are possible
- 39. Logical Replication - Publisher/Subscriber Model table - t1 and t2 on database test Connection Details PostgreSQL 10 Publication - Source Server Publisher Subscription - Destination Server Subsciber
- 40. Logical Replication - Publisher/Subscriber Model WAL Sender - Publisher Logical Replication Launcher - Subscriber
- 41. Logical Replication - Publisher/Subscriber Model Source/Publisher Destination/Subscriber WAL Records WAL Sender Logical Replication Launcher Publication Decoding Logical Messages Subscriber
- 42. Logical Replication - Publisher/Subscriber Model Granularity in replicating data can write on the destination Replicate across different versions DDL is not supported Not for failover
- 44. How to achieve HA
- 45. How to achieve HA Application Primary Server Writes Secondary Servers Reads
- 46. How to achieve HA Application Primary Server Writes Secondary Servers Reads
- 49. High Availability Frameworks - Patroni ETCD Server Secondary Servers Primary Server Patroni Patroni Patroni
- 50. High Availability Frameworks - Patroni Continuous Monitoring Automatic Failover Easy to add or remove the node Join back the node Support manual switchover too Scheduling maintenance
- 51. Points to Note Test the tool for auto-failover Load Balancing Frequent DR activity Maintain Backups Proper Monitoring
- 52. Reach Us : Info@mydbops.com Thank You