Beyond EBR, Quantum Columns, Spook Joins and Virtual Integrity for ZDT Data Versioning
Download as PPTX, PDF0 likes48 views
The document discusses a solution for managing complex applications with thousands of tables and views while ensuring zero downtime (ZDT) during releases using edition-based redefinition (EBR). It introduces concepts such as 'quantum columns' for managing seed data across different application editions, as well as techniques for maintaining referential integrity and minimizing performance impacts. The proposal emphasizes creating systems to facilitate the management of seed data versioning in tandem with application upgrades or patches.
Beyond EBR, Quantum Columns, Spook Joins and Virtual Integrity for ZDT Data Versioning
- 1. Beyond EBR (Draft) Quantum Columns, Spooky Joins and Virtual Integrity for ZDT Data Versioning Jeff Jacobs, Consulting Data Architect Jeffrey Jacobs & Associates
- 2. About Me
- 3. The Problem • Complex Application • 1,000s of Tables and Views • Referential integrity constraints require enforcement • Application code in PL/SQL, Java, etc. • New releases/upgrades/patches require significant downtime
- 4. Partial Solution - Edition Based Redefinition (EBR) • Migrate to Edition Based Redefinition EBR for Zero Down Time (ZDT) release • Stage and test new release in same infrastructure independent of current release
- 5. EBR Features • EBR manages editions of an application for: • PL/SQL changes • DDL changes • EBR’s edition triggers provide the ability to synch data between editions for changes to DDL • Edition to be used by code governed by SYS_CONTEXT variable • Very transparent to existing code • Each edition has an EDITION_NAME
- 6. What is Seed Data? • Seed Data tables are (usually) reference only tables • May be parent to multiple Non-Seed Data tables • “Seed Data” tables contain data that is added/updated for a new release • Data needs to be “versioned” for each release • Data is not synched across releases • Data changes not due to changes in DDL • New, coordinated version of data for multiple tables for new edition • Examples • Field labels for application screen • Text • Font • Color • Size • Demo/sample data • Etc.
- 7. Solution Goals • EBR does not provide support for managing Seed Data • But we want independent Seed Data for each edition • Minimal or no changes to existing DML • Minimal or no changes to existing code • Minimal or no affect on performance • Avoid creation of new DB objects, i.e. VIEWS, synonyms, etc. • Minimize rework • Preserve referential integrity enforcement
- 9. SYS_CONTEXT Variables • What are they? • How to create? • <code sample> • How to use? • <code sample>
- 10. Quantum Column Physics POP Quiz! (Schrodinger’s Cat + Quantum Superposition) • Can a given row’s virtual column have multiple values at the same instant in time with no in-flight transactions? • Yes • Call this a “Quantum Column” (QC) • How? • Virtual column is based on a SYS_CONTEXT variable • Value of QC is determined by value of SYS_CONTEXT variables • Can this be used to solve our Seed Data problem? • TBD!
- 11. Never Include a Quantum Column in an Index • QCs are inherently non-deterministic • Value is based on session’s value of SYS_CONTEXT variable at time of query • Virtual column value is determined at runtime • Indexes are deterministic: • A virtual column’s value is persisted in the index with the value at the time of INSERT/UPDATE
- 12. “Bollocks” – Bryn Lewellyn • Starting simple, focus on normal runtime behavior • Seed data patching issues later • How it’s done! • Add a Quantum Column to child table based on EBR SYS_CONTEXT variable • We’ll name the Quantum Column “EDITION_NAME_QC” • Drop the FK constraint(s) • Add a non-virtual version column to parent/Seed table • Holds the value of EBR editions • We’ll name this “EDITION_NAME • Drop PK constraint • Recreate PK constraint with prior columns plus new version column • Add Virtual Private Database expression to parent/Seed table to use appropriate EBR edition value • Normal runtime code uses above EBR SYS_CONTEXT variable value • Recreate FK • NO CHANGES TO EXISTING DML! Demos • Query to show original query works without new columns in join • Add new release to Seed Data with updated/new info • Show same query works in both editions returning appropriate data • Show referential integrity works
- 13. Part 1 – QCs and Foreign Key Constraints • FK columns may be based on a virtual column • Virtual column may be based on SYS_CONTEXT variable value • Such as EBR’s <need name> • QC is a virtual column • But what about the parent table’s primary key?
- 14. Part 2 – Modify Seed Table Primary Key • Add new column, “EDITION NAME” to PK • DEFAULT value of EBR_EDITION • PK now consists of original PK, i.e. “FIELD_NAME” • Patching code explicitly manages value of EDITION_NAME • Application code INSERT addressed later
- 15. New Data Model Diagram
- 16. VPD Example (Spooky Join) • VPD provides the additional join column between the Seed Data table and the child table • “Spooky” join
- 17. Spooky Join Demo • Create and populate tables with 1 release of data • Run Query • Enable EBR • Drop constraints • Add columns, PK and VPD
- 18. Virtual Integrity Constraint Demo • Add FK constraint • Insert into child table with valid original PK column values • succeeds • Demo Spooky Join • Attempt to INSERT with invalid original PK column values • fails
- 19. Important Stuff! • QC column can be added to any non-Seed table • FK constraint need not be changed if no Seed table reference • DO NOT ADD QC VIRTUAL COLUMN TO INDEX! • Inherently non-deterministic value! • Adding to index persists value • No need to use new columns in DML • DO NOT use EDITION_NAME (QC and parent PK columns) in DML • VPD handles appropriate join • For normal application runtime • Create virtual column with SYS_CONTEXT • Create new PK column with DEFAULT VALUE. (Double check this) • Seed data patching process needs explicit control
- 20. Recap • Demo’ed and explained basic technique
- 21. Life Is Never That Simple • Distinguish between: • Normal application code • Code to manage Seed Data – requires explicit control of values • Distinguish between: • Release/upgrade – a new version of the application • Typically requires new Seed Data • Patch/fix • No, or very few, changes to Seed Data • EBR may have many editions of either type • Need indirection for SYS_CONTEXT variables and EBR Edition values • What if end user can update/add to Seed Data table?
- 22. More Things to Consider • The technique described assumes a complete set of Seed Data for ZDT EBR Rollover to a given release • For new release, typically bulk copy/MERGE from prior release either: • At start of release work • Prior to rollover • For a fix/patch requiring a only a small amount of Seed Data changes and small downtime is acceptable • Backward MERGE from pending release
- 23. Beyond Simple EBR – Context Variable Management • There may be many EBR editions • DDL and PL/SQL changes form a single line inheritance structure • Each EBR edition holds only “true” changes • Not every new edition requires Seed Data changes • Solution • Create new set of SYS_CONTEXT variable, i.e. • RUNTIME_EDITION holds EBR edition name for current application code release • PATCHING_EDITION holds EBR edition name for the next release • Create a mapping table between DEFAULT_EBR and SYS_CONTEXT variable values • RUNTIME_EDITION <> EBR Default • Consider not using edition names • Morph edition name into “release” values
- 25. Application Code INSERT/UPDATE • Can application code INSERT/UPDATE into Seed Data Table • If Yes: • INSERT/UPDATE should be controlled in application, not ad-hoc INSERT/UPDATE • Prior to rollover, MERGE from runtime edition into patch edition • May need to handle sequence/identity column issues • Potential conflict between new release values and application values • Out of Scope • Assumes a brief freeze of such INSERT/UPDATE before rollover • Worst case requires AFTER INSERT/AFTER UPDATE triggers
- 26. Beyond EBR • This technique is NOT dependent on EBR!!! • QC, Spook Joins, Virtual Integrity Constraints • A new technique for versioning multi-row, multi-table datasets