2015 09-02 - transaction log prototype
0 likes137 views
The document discusses a transaction log prototype created by the GDS Publishing platform team to address bugs and issues in their system. It logs every change or transaction as a master copy for auditing purposes. Derived representations then allow future requirements to be easily supported by adding new representations based on the log. Examples shown include a bank account transaction log and how publishing documents could utilize derived representations like current/historical documents and editorial comments. The prototype implemented an event log, commands, queries to derived data, and was connected to their policy publisher. It used PostgreSQL for its transactions and JSON support. Remaining questions cover editing workflows, validation, performance, and snapshotting/replay capabilities.
2015 09-02 - transaction log prototype
- 1. David Heath Publishing platform team Government Digital Service GDSPublishing platform team
- 2. Transaction log prototype 2GDSPublishing platform team
- 5. many bugs... 5GDSPublishing platform team * many bugs and issues which we’ve experienced have been related to that
- 6. It's a master copy of every change or transaction 6GDSPublishing platform team
- 7. It’s an audit log 7GDSPublishing platform team a feature we’d really like to have
- 8. Future-proof 8GDSPublishing platform team if we have a log of every operation, it’s easy to adapt to possible future requirements, because we can easily add new “derived representations”
- 9. What is a derived representation? 9GDSPublishing platform team
- 10. Bank account 10GDSPublishing platform team Date Description Amount 1/9/2015 Initial deposit £50 Balance: £50
- 11. Bank account 11GDSPublishing platform team Date Description Amount 1/9/2015 Initial deposit £50 2/9/2015 Pod Ltd -£4.99 Balance: £45.01
- 12. Bank account 12GDSPublishing platform team Date Description Amount 1/9/2015 Initial deposit £50 2/9/2015 Pod Ltd -£4.99 3/9/2015 Cheque 003 £20 Balance: £65.01
- 13. Bank account 13GDSPublishing platform team Date Description Amount 1/9/2015 Initial deposit £50 2/9/2015 Pod Ltd -£4.99 3/9/2015 Cheque 003 £20 Balance: £65.01 Derived representation
- 14. In publishing… GDSPublishing platform team Event Log
- 15. In publishing… GDSPublishing platform team Event Log Current documents
- 16. In publishing… GDSPublishing platform team Event Log Current documents History of published documents
- 17. In publishing… GDSPublishing platform team Event Log Current documents History of published documents Editorial history and comments
- 18. GDSPublishing platform team Event Log Current documents History of published documents Editorial history and comments Admin elasticsearch index Message on message queue (asynchronous update) (separate service/data store) (Synchronously updated)Publishing API * Derived representations may be in separate systems * Some things need to be updated synchronously, those would need to be in the same data store and updated using a database transaction to guarantee consistency.
- 19. What we did in the prototype 19GDSPublishing platform team
- 20. What we did in the prototype • event log 20GDSPublishing platform team
- 21. What we did in the prototype • event log • commands (for modification) 21GDSPublishing platform team
- 22. What we did in the prototype • event log • commands (for modification) • queries (read from derived representation) 22GDSPublishing platform team
- 23. What we did in the prototype • event log • commands (for modification) • queries (read from derived representation) • wired up to policy publisher 23GDSPublishing platform team
- 24. 24GDSPublishing platform team we used postgresql because it’s awesome - real transactions - JSON data type
- 25. Show me some code! 25GDSPublishing platform team
- 26. Remaining questions 26GDSPublishing platform team • conceptual model of editing/ workflow
- 27. Remaining questions 27GDSPublishing platform team • conceptual model of editing/ workflow • how much validation? • use schemas?
- 28. Remaining questions 28GDSPublishing platform team • conceptual model of editing/ workflow • how much validation? • use schemas? • write performance
- 29. Remaining questions 29GDSPublishing platform team • conceptual model of editing/ workflow • how much validation? • use schemas? • write performance • replay, snapshotting etc
- 30. David Heath Publishing platform team Government Digital Service GDSPublishing platform team