Refactoring Into Microservices 2016-11-06
0 likes242 views
The document discusses the challenges of refactoring monolithic applications into microservices, highlighting symptoms of a monolith and the benefits of transitioning to microservices, such as increased speed to market and better management. It outlines various tactics for refactoring different types of monolithic applications, emphasizing the importance of iterative refactoring rather than complete rewrites. Key points include the need for proper automated testing, managing dependencies, and recognizing when refactoring is complete.
Refactoring Into Microservices 2016-11-06
- 1. Refactoring Into Microservices Breaking the Monolith Given by Derek C. Ashmore November 6, 2016 ©2016 Derek C. Ashmore, All Rights Reserved 1
- 2. Who am I? • Professional Geek since 1987 • Java/J2EE/Java EE since 1999 • Roles include: • Developer • Architect • Project Manager • Specialties • Refactoring • Performance Tuning ©2016 Derek C. Ashmore, All Rights Reserved 2
- 3. Discussion Resources • This slide deck – http://www.slideshare.net/derekashmore • This resource page – http://breakthemonolith.guru/resources.html • Slide deck has hyper-links! – Don’t bother writing down URLs ©2016 Derek C. Ashmore, All Rights Reserved 3
- 4. Agenda Microservices and Monoliths Tactics to Refactor Common Problems Summary / Q&A ©2016 Derek C. Ashmore, All Rights Reserved 4
- 5. What is a Monolith? • Monolith – Single deployable application that’s gotten too large and complex to maintain • Symptoms – Hard to change • QA test cycles are long • Change causes unintended consequences – Hard to onboard new developers – Married to your technical stack – Harder to diagnose bottlenecks and memory issues ©2016 Derek C. Ashmore, All Rights Reserved 5
- 6. What’s the cure? • Smaller services rather than one very large application • Refactoring easier said than done – Hard to do safely – It’s a hard design problem • Existing team members often too close to the problem – Developers can’t resist common business binaries • Reduces code duplication Increases coupling • Coupling is a greater evil than code duplication! • Why not a complete rewrite? – No benefit until the end – Nobody knows *all* the business requirements! – Risk of negative impact to the business during the transition is high • Microservices are the other side of the spectrum ©2016 Derek C. Ashmore, All Rights Reserved 6
- 7. Why Break a Monolith? • Increases Speed to Market – Monolith is slow to change • Long QA cycles – Quicker response to business competition • New features added quickly • Makes continuous delivery possible – Microservices + Continuous Delivery • Lowers Business Risk – Reduces the defect rate – Reduces Downtime – Fixes are faster too 7 • Amazon deploys once a second • Netflix deploys over 5,500 times a day Source: newrelic.com
- 8. What are Microservices? • No concrete definition • Common microservice traits – Single functional purpose • Most/all changes only impact one service • Not dependent on execution context – “loosely coupled” – Independent process/jvm – Stateless – Standard Interface (typically Web Service/REST) – Analogy: Stereo system, Linux utilities ©2016 Derek C. Ashmore, All Rights Reserved 8
- 9. Microservices Application Architecture • Separate Databases • Eventual Consistency • More network activity ©2016 Derek C. Ashmore, All Rights Reserved 9
- 10. Faster Delivery to Market • High Deployment Rates – Amazon = 23,000/day – Netflix = 5,500 / day • Business Benefits – New features delivered more quickly – Quicker reaction to competitors actions • Combined with – Continuous Delivery – DevOps ©2016 Derek C. Ashmore, All Rights Reserved 10
- 11. Microservice Reuse ©2016 Derek C. Ashmore, All Rights Reserved 11
- 12. No Lock-in • Platform agnostic • Fewer dependency conflicts • Still have cross-cutting concerns • “Toll” for first app • Still have support concerns • Others need to be able to support your work 12©2016 Derek C. Ashmore, All Rights Reserved
- 13. Better Availability and Resiliency • Fault tolerance built in – Reaction to failure part of the design – Failures designed to be localized • Requirements for Resiliency – Identify non-essential features • Gogo and Twitter – Context Independent Design ©2016 Derek C. Ashmore, All Rights Reserved 13
- 14. Easier Management / Higher Throughput • Easier to manage large numbers of developers – Concentrate on intelligently drawing service boundaries – Manage/enforce service contracts • Each service team works independently • Team independence leads to higher development throughput ©2016 Derek C. Ashmore, All Rights Reserved 14
- 15. Agenda Microservices and Monoliths Tactics to Refactor Common Problems Summary / Q&A ©2016 Derek C. Ashmore, All Rights Reserved 15
- 16. Microservices / Monolith Mixture • 80 / 20 Rule Applies • Smaller, less complex, services are easier to support • Few companies are 100% Microservices • Traditional architectures do have a place • They are easier to create and manage while they are small • Some never grow large enough to refactor ©2016 Derek C. Ashmore, All Rights Reserved 16
- 17. Do NOT attempt a complete rewrite • It’s too big and complex – Usually no documentation of all implemented business rules • Not written down • Nobody has it in their head – Will negatively impact your business • The new won’t ‘identically’ match behavior – Business won’t benefit until the very end • It’s a large project Compelte rewrite will take a long time to realize – Often has no safeguards beyond manual testing – It’s a *very* big problem to handle all at once. • Refactor iteratively – Yes; this will take a longer time – It’s safer – It gives staff a chance to learn how to manage microservice libraries before they have too many to manage manually – It gives staff a chance to address non-functionals while not also being burdened with a complete redesign/rewrite ©2016 Derek C. Ashmore, All Rights Reserved 17
- 18. Stick with the same technical stack for the refactoring • Corollary: Do not rewrite and refactor at the same time. –Rewrite after automated contract testing is in place • New technical stacks have learning curves –Refactoring is hard enough ©2016 Derek C. Ashmore, All Rights Reserved 18
- 19. Preperation Steps • Establish a base automated test harness for the application –At an integration level, not unit test level –We want a chance of catching defects refactoring introduces –This will *not* be 100% comprehensive • The monolith is too complex for that • 80 / 20 rule applies • Establish Continuous Integration if it doesn’t exist –Jenkins CI Builds and Tests –Run Frequently (at least nightly) –Fix it immediately when somebody breaks the build ©2016 Derek C. Ashmore, All Rights Reserved 19
- 20. Types of Monoliths • Monoliths are not created equal • Types of Monoliths – Feature-Bloated Web Application – Data Store Strangle – Over-engineered Tarball – Distributed Monolith • Tactics differ per Monolith type ©2016 Derek C. Ashmore, All Rights Reserved 20
- 21. Feature-Bloated App • Feature set to large to manage – Usually a web app, but could be “Thick” or “Batch” as well • Symptoms – Unintended consequence problem – Long regression test cycles • Causes – New Feature Additions • Add complexity • Often not envisioned in original design – Tacked on – Developer assumptions • Streamlines code initially • Are not documented • Create landmines for new feature additions • Removing unused features – Never happens – Benefit not clear to the business – No stats on used vs unused ©2016 Derek C. Ashmore, All Rights Reserved 21
- 22. Tactics: Feature-Bloated Web App • Chop off it’s head! – Separate the User Interface • Separation of concerns • Leaves consolidated UI • Forces Monolith to have REST interface – Easier to test more completely • Makes further refactoring safer • Forces you to identify REST resources within the monolith ©2016 Derek C. Ashmore, All Rights Reserved 22
- 23. Separated UI • Monolith now has a REST resource interface – Easier for automated testing • Safety in further refactoring – Functionality categorized into logical units • Easier to identify sections to separate out • Cosmetic changes occur on their own schedule ©2016 Derek C. Ashmore, All Rights Reserved 23
- 24. Factor out additional services • Factor out by related REST resources – Separate underlying database too • Prioritize resources by – Code Change Velocity – Code Complexity – Enhancement needs ©2016 Derek C. Ashmore, All Rights Reserved 24
- 25. Data Store Strangle • Data Storage is the Monolith’s Achilles Heel • Symptoms – No flexibility in data access • Larger memory footprint – Lots of replicated data – Complex data access code – Storage design hardest part of adding new features • Causes – Bad Data Storage Design • Usually process oriented – Replicated Data – Coupled with business process • Causes code complexity – Amplifies technical debt • Storage refactoring difficult – Never happens – Benefits not clear to the business – Carries more risk than refactoring code ©2016 Derek C. Ashmore, All Rights Reserved 25
- 26. Tactics: Data Store Strangle • Tourniquet Strategy – Separate storage reads/writes into separate service • Separate into service • All access uses central service • Spin off a read-only copy asynchronously in legacy format for reporting if needed – Wrap automated testing around the tourniquet service – Refactor the underlying database incrementally • Refactoring databases is a difficult topic – Book: Refactoring Databases: Evolutionary Database Design • Introduce new API to simplify access – Reduces caller complexity – More targeted and flexible access – Usually possible only after refactoring • Deprecate Tourniquet API ©2016 Derek C. Ashmore, All Rights Reserved 26
- 27. Over-engineered Tarball • Internal component obstacle – Component with unnecessary complexity • Symptoms – Nobody understands it – Everyone fears changing it – Common impediment for new feature enhancements • Causes – Bored, talented developers • Good intentioned • Manufacturing interesting puzzles • Real problem when the developer leaves – Nobody else can change it safely – Creates complexity in code surrounding it ©2016 Derek C. Ashmore, All Rights Reserved 27
- 28. Tactics: Over-engineered Tarball • Tourniquet Strategy – Separate tarball into separate service • Separate into service • All access uses central service • Also separate out data storage – Wrap automated testing around the tourniquet service – Refactor the tarball incrementally or completely replace • Automated tests are safety net ©2016 Derek C. Ashmore, All Rights Reserved 28
- 29. Distributed Monolith • Related services that are tightly coupled / bound together • Symptoms – Always deploy a group of services together • Independent deployments rarely happen – New features require changes to the same service group • Causes – High coupling between services • Short-term developer convenience • Inappropriate Delegation – Micro-management for indirect service calls – Pass-through arguments to one service to indirectly control behavior of a subsequent service • Shared Business Code Binaries – Changes in binaries force redeployments • Higher management overhead without benefits from smaller independent services ©2016 Derek C. Ashmore, All Rights Reserved 29
- 30. Tactics: Distributed Monolith • Solving shared business code problem – Factor into separate service(s) • Deployed once and commonly called • Don’t deploy as library – Reduces forced deployments – Performance mitigations • Expiring Cache • Make asynchronous – Coupling worse evil than Code Duplication ©2016 Derek C. Ashmore, All Rights Reserved 30
- 31. Tactics: Distributed Monolith (con’t) • Solving the Inappropriate Delegation problem – Eliminate the middle- man ©2016 Derek C. Ashmore, All Rights Reserved 31
- 32. Agenda Microservices and Monoliths Tactics to Refactor Common Problems Summary / Q&A ©2016 Derek C. Ashmore, All Rights Reserved 32
- 33. Common code between services? • Yes, but…. – Version it • Services make decision as to when to upgrade – Changes to common code can’t require the deployment of multiple services ©2016 Derek C. Ashmore, All Rights Reserved 33
- 34. Real-World Problems • Corporate-Level Reporting –Reporting for data across the enterprise –Many read underlying databases for applications that house the data –Refactoring will impact this type of reporting • Database Refactoring –Much more expensive –Can’t simply rollback –Book Recommendation • Database Factoring: Evolutionary Database Design ©2016 Derek C. Ashmore, All Rights Reserved 34
- 35. When is your refactoring complete? • You can stop at any point –If what remains of your monolith is supportable, you can stop –80 / 20 rule applies • That’s why services are prioritized by volatility • You can resume at any point –Guided by code volatility and defect rates –Change produces refactoring needs ©2016 Derek C. Ashmore, All Rights Reserved 35
- 36. Further Reading • This slide deck – http://www.slideshare.net/derekashmore • This resource page – http://breakthemonolith.guru/resources.html ©2016 Derek C. Ashmore, All Rights Reserved 36
- 37. Questions? • Derek Ashmore: – Blog: www.derekashmore.com – LinkedIn: www.linkedin.com/in/derekashmore • Connect Invites from attendees welcome – Twitter: https://twitter.com/Derek_Ashmore – GitHub: https://github.com/Derek-Ashmore – Book: http://dvtpress.com/ ©2016 Derek C. Ashmore, All Rights Reserved 37