Continuous Delivery for databases - microservices, team structures, and Conway's Law - develop:BBC 2014
10 likes10,111 views
The document discusses continuous delivery for databases within the context of microservices and team structures, emphasizing the impact of Conway's Law on software architecture and organizational design. It highlights the need for asynchronous data reconciliation and eventual consistency in database deployments, advocating for new team topologies to support more granular and frequent deployments. The content further explores the challenges and necessary shifts in responsibilities for developers and database administrators to adapt to these advancements.
Continuous Delivery for databases - microservices, team structures, and Conway's Law - develop:BBC 2014
- 1. Continuous Delivery for databases: microservices, team structures, and Conway's Law Matthew Skelton, Skelton Thatcher Consulting BBC:developconference, Thursday 13thNovember 2014, London, UK #bbcdevelop
- 2. Continuous Delivery Databases Microservices Team structures Conway’s Law
- 3. Developers & Database Administrators (DBAs)
- 7. Shared context and language for Developers and DBAs to facilitate a move to microservicesand Continuous Delivery
- 8. BUILDING BLOCKS Scale (magnification) Distributed transactions Event streams
- 9. OUTCOMES (New software architectures) New team topologies New responsibilities (Continuous Delivery)
- 10. Matthew Skelton •Building & operating commercial software systems since 1998 •Cybernetics + Neuroscience + Music •control engineering •psychology •‘network’ and group interactions @matthewpskelton
- 11. Skelton Thatcher Introduction and Services Overview
- 12. FORCES / CONTEXT
- 13. ‘Cloud’ changes the way we mustdesign, deliver, and operateour software systems
- 15. FORCES Deployability (Elasticity) (Conway’s Law)
- 16. DEPLOYABILITY Single application Single user journey Single feature
- 17. DATABASE DEPLOYABILITY Minimize changes in Production Reduce accidental complexity Archive, distinguish, and split data Name things transparently Source Business Intelligence from a data warehouse Value more highly the need for change Avoid Production-only tooling and config http://bit.ly/DatabaseDeployability
- 18. MICROSERVICES Single application Single user journey Single feature
- 19. DATABASE DEPLOYABILITY End the siege mentality
- 20. BUILDING BLOCKS
- 21. BUILDING BLOCKS Distributed transactions Scale (magnification) Event streams
- 27. BETWEEN ORGANISATIONS Asynchronous data reconciliation Eventual consistency
- 29. BETWEEN MICROSERVICES Asynchronous data reconciliation Eventual consistency
- 30. Familiar patterns, different ‘magnification’
- 31. EVENT STREAMS Event sourcing NoSQL, CQRS, Reactive Pub-Sub messaging
- 32. EVENT STREAMS RDBMS database transaction log
- 33. Familiar patterns, different ‘magnification’
- 35. FAMILIAR PATTERNS Distributed transactionsEvent streamsAsynchronous data reconciliation Eventual consistency
- 36. CHANGES
- 37. CHANGES Team topology Responsibilities
- 38. Conway’s Law
- 39. Mel Conway, 1968 “organizations which design systems ... are constrained to produce designs which are copies of the communication structures of these organizations” http://www.melconway.com/Home/Conways_Law.html
- 40. Ruth Malan, 2008 “if the architecture of the system and the architecture of the organization are at odds, the architecture of the organization wins” http://traceinthesand.com/blog/2008/02/13/conways-law/
- 48. There is no single ‘right’ team topology, but there are several ‘bad’ topologies for any one organisation
- 49. Make the team topology & responsibilities match the required software architecture
- 50. RECAPITULATION
- 51. Both Devsand DBAs know that: Distributed transactions do not scale wellEvent streams are the foundation of reliable systems
- 52. Shared concepts: Distributed transactionsEvent streamsAsynchronous data reconciliationEventual consistency
- 53. Familiar concepts, different scale & context
- 54. New team topologies needed
- 56. We did not really mention: Microservicestechnologies Team topologies in depth …
- 57. Continuous Delivery for databases We need finer-grained, more frequent and isolated deployments Microservicesallow these more fine-grained deployments We need to support the microservicedatabase model: a larger number of smaller databases with eventual consistency and asynchronous (back office) data reconciliation This is what we've been doing at an organisational level anyhow (Master Data, inter- bank reconciliation) so we apply the same principles at a Business Service level This needs team restructuring to be effective (Conway's Law) What team topologies might work well?
- 58. Continuous Delivery for databasesFurther details (article, slides): http://bit.ly/CDforDB
- 60. Further reading Build Quality In buildqualityin.com Continuous Delivery & DevOps experience reports Forewords by Dave Farley and Patrick Debois 70% of royalties go to Code Club (@CodeClub) Contributors include several presenters at BBC:develop2014: -Amy Phillips (Songkick) -Phil Wills (Guardian) Discount for #bbcdevelop! https://leanpub.com/buildqualityin/c/BBCdevelop2014
- 61. Further reading Ruth Malan (@ruthmalan) –Conway’s Law round-up: http://www.ruthmalan.com/Journal/2014/2014JournalMay.htm#Conways_Law Allan Kelly on Conway’s Law: https://vimeo.com/channels/londoncd/85378217 Common database deployment blockers and Continuous Delivery headaches (Simple Talk) http://bit.ly/DatabaseDeployability Team Topologies: http://bit.ly/DevOpsTopologies Matt Hilbert on ‘Next Generation DBA’: http://bit.ly/NextGenDBA Chris Richardson ‘Microservices: Decomposing Applications for Deployabilityand Scalability’ http://www.slideshare.net/chris.e.richardson/microservices-decomposing-applications-for- deployability-and-scalability-jax
- 62. Thanks BBC for inviting the talk Audience for listening, asking questions People at Red Gate Software for joining up some of the dots
- 63. Thank you! http://skeltonthatcher.com/ enquiries@skeltonthatcher.com @SkeltonThatcher +44 (0)20 8242 4103