Blast Radius
0 likes121 views
The document outlines the development of a Gradle plugin called 'blast radius' that minimizes deployment risks in a continuous deployment pipeline for a medium-sized project with multiple components managed in one repository. It analyzes commit behavior, revealing that a small percentage of commits accounted for most deployments, indicating hidden coupling between modules. Key takeaways include the tool's effectiveness in maintaining fast and precise builds, revealing possible design flaws, and the value of simple tools in enhancing deployment processes.
Blast Radius
- 1. Blast Radius How my team tackled Continuous Deployment and a handy tool I developed to minimise how much we deployed for each commit FrankFarrell
- 2. Some context • 3 Engineers • One app with lots of Deployable Components • Medium Sized project
- 3. 1 or many repositories? vs • share libraries • do code reviews across modules • refactor across modules Easier to
- 4. 1 or many repositories? vs • do Continuous Deployment • work with tooling • enforce architectural decisions We decided on 1 big repo Easier to • share libraries • do code reviews across modules • refactor across modules Easier to
- 5. Continuous Deployment from 1 big repo • Why not deploy everything on every commit? Risk Time Visibility
- 6. We need something better Safety Speed Precision
- 7. Blast Radius Gradle Plugin
- 8. How it works git diff: • /src/main/Foo.java in Library B • README.md in Library A Foo.java is deployable code, so we deploy its only dependency A README.md isn’t code so we won’t deploy any of its dependencies /src/main/Foo.javaREADME.md SCENARIO 1
- 9. How it works git diff: • /src/main/Foo.java in Library B • README.md in Library A Foo.java is deployable code, so we deploy its only dependency A README.md isn’t code so we won’t deploy any of its dependencies /src/main/Foo.javaREADME.md SCENARIO 1
- 10. How it works git diff • /src/main/Bar.java in Library A • /src/test/FooTest.java in Library B Bar.java is deployable code, so we deploy its 2 dependencies A unit test isn’t deployable code so it doesn’t have an effect. /src/main/Bar.java /src/test/FooTest.java SCENARIO 2
- 11. Hypothesis: Pareto Distribution I analysed our CD pipeline for the last 8 months with ~200 builds to try and get some insights into our codebase. Roughly speaking 20% of commits would cause 80% of module deployments. # Modules Deployed Per Commit 0 #Commits 50 100 0 5 10 20 30 40
- 12. So much for the theory This graph is a count of commits that triggered X deployments • Does nothing where nothing to deploy • Small set where everything deployed NO DEPLOYMENTS (79) JUST ONE (77) A SMALL SUBSET (22) EFFECTIVELY EVERYTHING (17) #Commits # Modules Deployed Per Commit
- 13. So much for the theory • Single module changes surprisingly common • The small subset is less frequent than I expected, but that’s the real advantage of the tool NO DEPLOYMENTS (79) JUST ONE (77) A SMALL SUBSET (22) EFFECTIVELY EVERYTHING (17) #Commits # Modules Deployed Per Commit
- 14. How often are individual Modules changing? More of a Pareto distribution J But, the plateau in the middle* helped us identify hidden coupling between modules * folk statistics Module Name #Deployments TERRAFORM API
- 15. Coincidence of Deployment How likely are any 2 modules to be deployed together? Bottom group is clearly coupled API TERRAFORM EVENT PIPELINE
- 16. Coincidence of Deployment Not very coupled, which is surprising for infrastructure. Did Blast Radius influence commit behaviour? TERRAFORM
- 17. Coincidence of Deployment Our API on the other hand has a lot of hidden coupling to other modules. Could it indicate design flaws in our code? API
- 18. Key takeaways •Tool worked to keep our builds fast & precise •Hidden coupling probably means some design flaws •However, lots of positives for design and workflow
- 19. Other takeaways •Maintaining a complicated pipeline is in itself time consuming •Deploy concurrently from 1 big repo •Simple tools can have a big impact
- 20. Get it: github.com/frankfarrell/blast-radius It could easily be translated to your favourite language and build tool Thanks for listening!