JAX DevOps 2018 "Continuous Delivery Patterns for Modern Architectures"
Download as PPTX, PDF3 likes400 views
The document discusses the evolution of continuous delivery for Java architectures, emphasizing the transition from complicated to complex systems and the importance of technical leadership. Key challenges include managing multiple services, ensuring stable deployments, and optimizing for both functional and non-functional requirements within delivery pipelines. It advocates for a focus on design principles like cohesion and coupling to improve software quality and business agility.
JAX DevOps 2018 "Continuous Delivery Patterns for Modern Architectures"
- 1. Continuous Delivery Patterns for Modern Java Architectures Daniel Bryant @danielbryantuk
- 2. Architecture: Expectations versus reality 11/04/2018 @danielbryantuk “DevOps”
- 3. tl;dr • We are moving from complicated systems to complex systems • Architecture is becoming more about technical leadership • We must encode all requirements into a continuous delivery pipeline 11/04/2018 @danielbryantuk
- 4. @danielbryantuk • Independent Technical Consultant, Product Architect at Datawire • Architecture, DevOps, Java, microservices, cloud, containers • Continuous Delivery (CI/CD) advocate • Leading change through technology and teams 11/04/2018 @danielbryantuk bit.ly/2jWDSF7
- 5. Setting the Scene 11/04/2018 @danielbryantuk
- 6. Continuous Delivery • Produce valuable and robust software in short cycles • Optimising for feedback and learning • Not (necessarily) Continuous Deployment 11/04/2018 @danielbryantuk
- 7. Velocity (with stability) is key to business success “Continuous delivery is achieved when stability and speed can satisfy business demand. Discontinuous delivery occurs when stability and speed are insufficient.” - Steve Smith (@SteveSmithCD) 11/04/2018 @danielbryantuk
- 8. 11/04/2018 @danielbryantuk Feedback: - Was our hypothesis proven? - How can we improve biz, architecture and ops?
- 9. Let’s bring in some containers (or additional packaging) 11/04/2018 @danielbryantuk
- 11. What are our core challenges with modern Java app architectures? 11/04/2018 @danielbryantuk
- 14. 11/04/2018 @danielbryantuk Gated application deployment 1. 2. 3.
- 15. 11/04/2018 @danielbryantuk Independent service deployment
- 17. From monolith to… 11/04/2018 @danielbryantuk UI / Biz / Repo Monolith Domains Modules, components, frameworks, libraries http://scs-architecture.org/
- 18. 11/04/2018 @danielbryantuk Self Contained Systems (SCS) Microservices Function-as-a-Service “Serverless”
- 19. 11/04/2018 @danielbryantuk Monoliths SOA Microservices / SCS FaaS / Serverless Scope Project Enterprise Product Feature (or glue?) Focus Swiss Army Knife Reuse, governance, control Domain modelling, SRP, evolution Function (in/out), rapid evolution Organisation Implemented and maintained (typically) by single team Implemented by different org units. Maintenance done elsewhere Services implemented and owned by product teams Implemented by pioneers (hipsters?) Deployment Monolithic deployment Monolithic orchestration of multiple services Services deployed individually Functions deployed individually Management None Centralised Distributed Chaotic / Orchestrated Inter-process communication None RPC or messaging, typically via middleware (ESB/MQ) RPC via dumb pipes and smart endpoints, messaging/events Events Pioneers / stewards Organisations, community or individuals Enterprises and Vendors Community and high perf organisations Vendors/community Core Architectural Constraints Language and framework Canonical domain model, standards Interoperability Cost (Gbs/ms)
- 20. 11/04/2018 @danielbryantuk Monoliths SOA Microservices / SCS FaaS / Serverless Scope Project Enterprise Product Feature (or glue?) Focus Swiss Army Knife Reuse, governance, control Domain modelling, SRP, evolution Function (in/out), rapid evolution Organisation Implemented and maintained (typically) by single team Implemented by different org units. Maintenance done elsewhere Services implemented and owned by product teams Implemented by pioneers (hipsters?) Deployment Monolithic deployment Monolithic orchestration of multiple services Services deployed individually Functions deployed individually Management None Centralised Distributed Chaotic / Orchestrated Inter-process communication None RPC or messaging, typically via middleware (ESB/MQ) RPC via dumb pipes and smart endpoints, messaging/events Events Pioneers / stewards Organisations, community or individuals Enterprises and Vendors Community and high perf organisations Vendors/community Core Architectural Constraints Language and framework Canonical domain model, standards Interoperability Cost (Gbs/ms)
- 21. Architecture fundamentals • Coupling • ”Components have little or no knowledge of other components” • Interfaces • Schema • Cohesion • “Degree to which the elements within a component belong together” • Single reason to change • Separation of concerns 11/04/2018 @danielbryantuk
- 22. Coupling, Cohesion and Continuous Delivery • Design • Cohesion makes reasoning easy • Loose coupling reduces impact • Build, unit and integration • Cohesion limits dependencies • Loose coupling allows simulation • End-to-end tests • Cohesion/coupling orchestration • Deployment • Cohesion minimises number of components in play • Coupling leads to “monoliths” • Observability • Cohesive is easier to understand • High coupling typically leads to large blast radius and “murder mystery” style debugging 11/04/2018 @danielbryantuk
- 24. Testing: Core concepts 11/04/2018 @danielbryantuk /lisacrispin.com/2011/11/08/using-the-agile-testing-quadrants/martinfowler.com/bliki/TestPyramid.html
- 25. Testing: Core concepts 11/04/2018 @danielbryantuk martinfowler.com/articles/microservice-testing/#testing-progress-3 martinfowler.com/articles/practical-test-pyramid.html
- 26. Testing: Core (microservice) concepts 11/04/2018 @danielbryantuk https://medium.com/@copyconstruct/testing-microservices-the-sane-way-9bb31d158c16
- 27. Testing microservice integration 11/04/2018 @danielbryantuk
- 28. Functional testing: End-to-end 11/04/2018 @danielbryantuk
- 29. Functional Testing: Outside-in 11/04/2018 @danielbryantuk https://specto.io/blog/2016/8/16/recipe-for-designing-building-testing-microservices/ http://www.thucydides.info/docs/serenity/
- 31. Talking of contracts… 11/04/2018 @danielbryantuk
- 32. Talking of (service) contracts… • APIs are service contracts • Consumer-driven Contracts • martinfowler.com/articles/consumerDrivenContracts.html 11/04/2018 @danielbryantuk
- 33. Talking of (service) contracts… 11/04/2018 @danielbryantuk docs.pact.io cloud.spring.io/spring-cloud-contract github.com/spring-cloud-samples/spring-cloud-contract-samples
- 34. Talking of (messaging) contracts… • What about messaging? • Message schema are an API • www.infoq.com/presentations/contracts-streaming-microservices 11/04/2018 @danielbryantuk
- 35. Talking of (messaging) contracts… 11/04/2018 @danielbryantuk www.infoq.com/presentations/contracts-streaming-microservices docs.confluent.io/current/schema-registry/docs/maven-plugin.html
- 36. Contract verification • Is it worth the cost? • Tradeoff: Trust/comms vs time • Startups / SMEs • Enterprises… • My (anecdotal) experience • Choreography vs orchestration 11/04/2018 @danielbryantuk
- 38. Contract verification • Is it worth the cost? • Tradeoff: Trust and comms • Startups / SMEs • Enterprises… • My (anecdotal) experience • Choreography vs orchestration • Choreography • Verifying behaviour (interactions) • Contracts are part of this! • London school TDD • Orchestration • Verify state (output) • Lint/validate orchestration DSL • Chicago school TDD 11/04/2018 @danielbryantuk
- 39. Measure what matters 11/04/2018 @danielbryantuk
- 40. 11/04/2018 @danielbryantuk Feedback: - Was our hypothesis proven? - How can we improve biz, architecture and ops?
- 41. Visibility for the business 11/04/2018 @danielbryantuk
- 42. Testing NFRs in the build pipeline • Architecture quality • SonarQube / Code Climate • Performance and Load testing • Gatling / Locust / Bees with m’guns • Security testing • OWASP Dependency check / bdd-security • Docker Bench for Security / CoreOS Clair 11/04/2018 @danielbryantuk
- 44. Unit Testing Architecture 11/04/2018 @danielbryantuk https://www.archunit.org/
- 45. Security Visibility: Basic Java Code Scanning 11/04/2018 @danielbryantuk
- 46. Security Visibility: Static Package Scanning 11/04/2018 @danielbryantuk https://github.com/arminc/clair-scanner
- 47. Serverless security “Since the OS is unreachable, attackers will shift their attention to the areas that remain exposed – and first amongst those would be the application itself.” Responsibility for addressing vulnerable app libraries falls to you – the function developer. 11/04/2018 @danielbryantuk https://www.infoq.com/articles/serverless-security
- 48. Security Visibility: Dependencies 11/04/2018 @danielbryantuk www.owasp.org/index.php/OWASP_Dependency_Check
- 49. Delaying NFRs to the ‘Last Responsible Moment’ Newsflash! Sometimes the last responsible moment is up-front Modern architectures don’t necessarily make this easier 11/04/2018 @danielbryantuk
- 51. 11/04/2018 @danielbryantuk Monoliths SOA Microservices / SCS FaaS / Serverless Scope Project Enterprise Product Feature (or glue?) Focus Swiss Army Knife Reuse, governance, control Domain modelling, SRP, evolution Function (in/out), rapid evolution Organisation Implemented and maintained (typically) by single team Implemented by different org units. Maintenance done elsewhere Services implemented and owned by product teams Implemented by pioneers (hipsters?) Deployment Monolithic deployment Monolithic orchestration of multiple services Services deployed individually Functions deployed individually Management None Centralised Distributed Chaotic / Orchestrated Inter-process communication None RPC or messaging, typically via middleware (ESB/MQ) RPC via dumb pipes/smart endpoints, messaging/events Events Pioneers / stewards Organisations, community or individuals Enterprises and Vendors Community and high perf organisations Vendors/community Core Architectural Constraints Language and framework Canonical domain model, standards Interoperability Cost
- 52. 11/04/2018 @danielbryantuk Monoliths SOA Microservices / SCS FaaS / Serverless Cohesion and coupling enforced at modules CD focuses on end-to- end functionality Provide goals and “best practice” examples Cohesion and coupling enforced at service level CD focuses on service integrity Provide objectives and standards Cohesion and coupling enforced at service API level CD focuses on service interaction Provide principles and guidelines Cohesion and coupling enforced at function API level CD focuses on service output/state Provide principles and guidelines
- 53. In conclusion… • We are moving from complicated systems to complex systems • Design and test with coupling and cohesion in mind • Architecture is becoming more about technical leadership • Recognise your situation and influence accordingly • We must encode all requirements into a continuous delivery pipeline • Both functional and nonfunctional requirements 11/04/2018 @danielbryantuk
- 55. Thanks for listening… Twitter: @danielbryantuk Email: daniel.bryant@tai-dev.co.uk Writing: https://www.infoq.com/profile/Daniel-Bryant Talks: https://www.youtube.com/playlist?list=PLoVYf_0qOYNeBmrpjuBOOAqJnQb3QAEtM 11/04/2018 @danielbryantuk bit.ly/2jWDSF7 Coming soon!