PostgreSQL Enterprise Class Features and Capabilities
2 likes1,020 views
The document discusses the challenges and strategies for migrating from Oracle to PostgreSQL, focusing on design, architecture, data migration, and application impacts. It outlines differences in database objects, data types, and the need for application code changes, as well as ways to optimize migration efforts. Additionally, it highlights PostgreSQL capabilities, including replication features and performance improvements.
PostgreSQL Enterprise Class Features and Capabilities
- 1. Copyright 2016 FUJITSU LIMITED I am.. Venkata B Nagothi Call me.. VENKAT
- 2. Copyright 2016 FUJITSU LIMITED Agenda Migration Challenges (Oracle to PostgreSQL) PostgreSQL capabilities
- 3. Copyright 2016 FUJITSU LIMITED Migration Challenges Design and Architecture challenges Development effort required for successful database migration to PostgreSQL Minimize development effort when migrating Applications
- 4. Copyright 2016 FUJITSU LIMITED Migration Assessment Infrastructure Database Objects Data Migration PL/SQL Objects Database Environment Application
- 5. Copyright 2016 FUJITSU LIMITED Infrastructure Migration
- 6. Copyright 2016 FUJITSU LIMITED High Availability Oracle Dataguard Broker PostgreSQL Streaming Replication Migration Impact • More simpler to implement • Supports all the protection modes in Oracle • Supports cascading replication with some limitations
- 7. Copyright 2016 FUJITSU LIMITED High Availability Oracle Dataguard Observer PostgreSQL pgPool-II Master Standby Switches over when the master crashes pgPool-II Automatically promotes the standby to be the new master pgPool-II Automatic failover disable Application
- 8. Copyright 2016 FUJITSU LIMITED High Availability Role reversal – Limitations in PostgreSQL Master Standby (new master) Streaming Replication Standby can be promoted to Standalone new master manually Old master cannot be made standby. Needs to rebuilt from new master’s backup
- 9. Copyright 2013 FUJITSU LIMITED High Availability PostgreSQL Streaming Replication – Limitations Filesystem structure has to be identical v Master Streaming Replication /data/tbs01 /data/tbs02 /data/tbs03 Standby /data/tbs01 /data/tbs02 /data/tbs03
- 10. Copyright 2016 FUJITSU LIMITED Tablespaces Oracle Tablespaces PostgreSQL Tablespaces Migration Impact • More OS dependent • Re-Architect database disk architecture • Performance testing and benchmarking • Re-do Capacity Planning
- 11. Copyright 2016 FUJITSU LIMITED Tablespaces /disk1/datafile-1 /disk2/datafile-2 /disk3/datafile-3 Logical Structure Logical Structure Physical Structure Physical Structure /disk1/tbs01 Oracle Tablespace Structure PostgreSQL Tablespace Structure Table 1 Table 2 Index 1 Table 1 Table 2 Index 1 Datafile 1 Datafile 2 Datafile 3
- 12. Copyright 2016 FUJITSU LIMITED Tablespaces PostgreSQL Disk Architecture /disk1/tbs01 /disk2/tbs02 /disk3/tbs03 /disk4/tbs04 /disk5/tbs05 Each partition has to be an extendable partition for better capacity management A single partition over multiple disk spindles (RAID) /data/pgdata
- 13. Copyright 2013 FUJITSU LIMITED Tablespaces Limitations - PostgreSQL Tablespaces alone cannot be backed up Transportable tablespaces not possible Tablespace restore/recovery is not quite possible
- 14. Copyright 2013 FUJITSU LIMITED Databases DB01 Instance 1 DB02 Instance 2 DB03 Instance 3 DB01 Instance 1 DB02 DB03 Oracle PostgreSQL • Database Size • Database performance • Application Architecture/behaviour • Database operations • Security • Multitenancy • Other business and operational constraints
- 15. Copyright 2016 FUJITSU LIMITED Database Objects Migration
- 16. Copyright 2016 FUJITSU LIMITED Schemas Oracle Users PostgreSQL Users & Schemas Migration Impact • Users and Schemas must be separately created in PostgreSQL • A user can own multiple schemas • Flexibility around changing ownerships
- 17. Copyright 2016 FUJITSU LIMITED Database Objects Oracle Objects PostgreSQL Objects Materialized Views Materialized Views (limitations) Index-Organized-Tables Cluster a Table (partial support) Public Synonyms search_path Global Temporary Tables Unlogged Tables
- 18. Copyright 2016 FUJITSU LIMITED Data Types Oracle Data type PostgreSQL Data Type VARCHAR2 VARCHAR NUMBER INTEGER DATE TIMESTAMP CLOB TEXT / JSON / JSONB BLOB BYTEA / JSONB LO
- 19. Copyright 2016 FUJITSU LIMITED Data Migration
- 20. Copyright 2016 FUJITSU LIMITED Data Migration BLOB BYTEA (1 GB LIMIT) LO BLOB JSONB
- 21. Copyright 2016 FUJITSU LIMITED Data Migration Oracle BLOB PostgreSQL BYTEA (max limit 1 GB) Ora2pg (automatic) Migration Impact • Oracle BLOBs are automatically migrated to PostgreSQL bytea by ora2pg • Unstable Application performance can be expected which would hit the benchmarking metrics • Can accommodate more than 1 GB due to PostgreSQL compression algorithm • Triggers an Application code change
- 22. Copyright 2016 FUJITSU LIMITED Application Impact (Example : JAVA) getBytes() BYTEA Migration Impact • Only getBytes() function can be used against BYTEA data type in PostgreSQL • getBytes() attempts to read the whole BYTEA into the memory at a time which will impose performance problems resulting in excessive usage of Memory and Network bandwidth • Cannot stream BYTEA in small chunks BYTEA Column Size Text File 10K Text File 10K Image 20M Image 300M Text file 100K Data Migration getBLOB() BLOB
- 23. Copyright 2016 FUJITSU LIMITED Data Migration Oracle BLOB PostgreSQL LOManual Migration Impact • Custom ETLs must be built • Migration effort and time can be a challenge • Application code change is expected • Application functionality and Performance testing
- 24. Copyright 2016 FUJITSU LIMITED Application Impact (Example : JAVA) getBlob() LO Migration Impact • getBlob() function is used against LO similar to Oracle • getBlob() is the recommended approach for accessing LO (Large Objects) datatypes from PostgreSQL • Large Objects can be streamed • BLOBs can be streamed in multiple chunks resulting in effective performance Data Migration BLOB
- 25. Copyright 2016 FUJITSU LIMITED Data Migration Oracle BLOB with JSON data PostgreSQL JSONBManual Migration Impact • Can result in an heavy Application design and code change • Migration effort and time can be a challenge • Heavy development can be on the cards SELECT UTL_RAW.CAST_TO_VARCHAR2 select col->>‘value' FROM table;
- 26. Copyright 2016 FUJITSU LIMITED Data Migration Oracle CLOB with JSON data PostgreSQL TEXTAUTOMATIC (Ora2pg) Migration Impact • Can result in an heavy Application design and code change • Migration effort and time can be a challenge PostgreSQL JSONB Manual
- 27. Copyright 2016 FUJITSU LIMITED PL/SQL Migration
- 28. Copyright 2016 FUJITSU LIMITED PL/SQL Migration PL/SQL Object PostgreSQL Alternative Procedures Functions with RETURNS VOID Functions Functions Triggers Triggers Packages Schemas + Temp tables + Functions
- 29. Copyright 2013 FUJITSU LIMITED System Packages SYSTEM PACKAGES PostgreSQL Alternative DBMS* PACKAGES • ORAFCE external contrib module • PostgreSQL alternatives • Custom functions
- 30. Copyright 2016 FUJITSU LIMITED System Packages Some of the system packages supported by PostgreSQL PostgreSQL - ORAFCE • DBMS_OUTPUT • DBMS_ALERT • DBMS_PIPE • DBMS_UTILITY • DBMS_ASSERT • DBMS_RANDOM
- 31. Copyright 2016 FUJITSU LIMITED System Packages Challenges in migrating DBMS_JOBS system package Oracle PostgreSQL • DBMS_JOBS • pg_agent
- 32. Copyright 2016 FUJITSU LIMITED Packages Packages PACKAGES PostgreSQL Alternative PACKAGE DEFINITION SCHEMA GLOBAL VARIABLES TEMP TABLES PACKAGE BODY FUNCTION(S)
- 33. Copyright 2016 FUJITSU LIMITED Migrating Oracle Queries
- 34. Copyright 2016 FUJITSU LIMITED Oracle Hints Oracle Hints PostgreSQL NO HINTS Migration Impact • HINTS are not supported in PostgreSQL - NOT REQUIRED • Code changes required • Functionality testing • Performance testing and Benchmarking required
- 35. Copyright 2016 FUJITSU LIMITED Oracle Outer joins Oracle LEFT OUTER JOIN PostgreSQL LEFT OUTER JOIN SELECT e.ename, d.dname 2 FROM emp e, dept d 3 WHERE e.deptno = d.deptno(+); SELECT e.ename, d.dname FROM emp e LEFT OUTER JOIN dept d ON (e.deptno = d.deptno); Migration Impact • Application code change • Functionality testing
- 36. Copyright 2016 FUJITSU LIMITED Oracle PostgreSQL SELECT sequencename.nextval from dual SELECT nextval(‘sequencename'); SELECT sequencename.currval from dual SELECT currval(‘sequencename’); Sequences
- 37. Copyright 2013 FUJITSU LIMITED Subqueries Every subquery has to have an alias Oracle PostgreSQL select * from (select * from emp) select * from (select * from emp) as foo
- 38. Copyright 2016 FUJITSU LIMITED Implicit Type Casting Implicit type casting can be highly beneficial and can help reduced the need to change the application code to a greater extent Increases the possibility usage of Indexes Avoids the need to create function-based Indexes Table PostgreSQL select * from table where col1::int > 2; create cast(varchar as integer) with inout as implicit; select * from table where col1 > 2; Oracle select * from table where col1 > 2; varchar varchar 2 Sydney 3 London 4 Singapore 3 London 4 Singapore 3 London 4 Singapore
- 39. Copyright 2016 FUJITSU LIMITED Hierarchical Queries Oracle Hierarchical Queries PostgreSQL Hierarchical Queries (There is no straight forward way) Oracle PostgreSQL • CONNECT BY, START WITH, SYS_CONNECT_BY_PATH and CONNECT_BY_ROOT • WITH RECURSIVE • connectby() function part of tablefunc contrib module • UNION and UNION ALL • pl-pgsql functions
- 40. Copyright 2016 FUJITSU LIMITED Application Migration Challenges
- 41. Copyright 2016 FUJITSU LIMITED JDBC Driver Oracle JDBC Driver PostgreSQL JDBC Driver Migration Impact AUTOCOMMIT OFF • Application behaviour is different in autocommit off mode • Multiple transactions will be automatically part of a transaction block, which means COMMIT ALL or NONE • Heavy application code change may be required • Functionality testing • Performance testing • Changing legacy code might impose further more challenges
- 42. Copyright 2013 FUJITSU LIMITED Monitoring Automation Integration Database Environment
- 43. Copyright 2016 FUJITSU LIMITED PostgreSQL Capabilities
- 44. Copyright 2016 FUJITSU LIMITED Replication Slots wal_keep_segments Earlier to 9.4 upgrade Replication Slots 9.4
- 45. WAL Optimization PostgreSQL – 9.4, the amount of WAL data to be written has been reduced in-case of updates. This results in IO performance gain and reduction in the size of WAL files. Example : When one column part of a 4 column row gets updated, only the updated part of the row is written to WAL files Table WAL File (9.3) Row 1 Row 2 Col 1 Col 2 Col 3 Col 4 Row 1 Row 2 WAL File (9.4 ) Row 1 Row 2 Copyright 2016 FUJITSU LIMITED
- 46. Copyright 2016 FUJITSU LIMITED WAL Optimization Table Size – 19GB 4 Columns Updated 1 column 0 5 10 15 20 25 WALs (GB) in 9.3 WALs (GB) in 9.4 Series1
- 47. Copyright 2013 FUJITSU LIMITED Questions ?