J-Spring 2017 - Microservices in action at the Dutch National Police
1 like622 views
The document outlines the implementation of microservices at the Dutch National Police, highlighting their methodology, cloud platform, architecture, frontend and backend development, testing, and deployment strategies. It emphasizes the importance of continuous delivery, minimal dependencies, and the use of various tools like OpenStack and Jenkins for efficient operations. The document also discusses future plans, including automated security testing and the creation of cross-functional teams.
J-Spring 2017 - Microservices in action at the Dutch National Police
- 1. bertjan@jpoint.nl Microservices in action at the Dutch National Police Bert Jan Schrijver @bjschrijver
- 2. Bert Jan Schrijver L e t โ s m e e t @bjschrijver
- 3. Architecture and platform Frontend Methodology and culture Introduction Development and testing Build tools, deployments and running in production Challenges and looking ahead Outline W h a t โ s n e x t ? Backend
- 4. The police protects the democracy, maintains the law and is the authority on the streets. Around 65.000 people work at the Dutch police, of which over 1500 IT professionals. Dutch National Police
- 5. CLOUD PLATFORM ANALYSE PATRONEN BIG DATA SECURITY3 DevOps teams are building high tech big data web applications in a private cloud environment. These applications support police related themes. Product line Cloud | Big Data | Internet
- 7. โข 3 teams with separate backlogs โข Overall planning at start of sprint โข Minimal planning ritual โข Usability tests as part of sprint โข Phabricator as tool of choice Methodology
- 8. โข Continuous Delivery & DevOps โข Short feedback loops โข Embrace change โข Minimal dependencies outside team โข Invest in people, not in products โข Open, transparent, verifiable Culture Source: http://kids.nationalgeographic.com/explore/countries/netherlands/#netherlands-tulip-fields.jpg
- 10. โข End-to-end security and encryption โข Version control for everything โข Horizontally scalable, no single point of failure โข No dependencies on external sources โข Standardised naming โข Application config lives with code โข Services defined by business functionality Architecture
- 11. Architecture
- 20. Source: https://www.google.com/about/datacenters โข Openstack private cloud โข General cloud services for police โจ organisation โข Ceph distributed storage โข Puppet & Ansible for config management โข 3000 managed desktops โข Automation starts when hardware boots Platform
- 21. Frontend
- 22. โข Angular 2.x โข Typescript โข RxJS โข Bootstrap โข Responsive design โข Feature toggles โข Graceful degradation when backend fails Frontend
- 23. Backend
- 24. โข Small in size, single responsibility โข Runs in its own process โข Independently develop, deploy, upgrade, scale โข Has its own data store โข Distributed by default โข Potentially heterogeneous/polyglot โข Light-weight communication Anatomy of a microservice
- 25. โข Spring Boot, Java 8, Maven โข Stateless โข 1 service in 1 jar on 1 JVM on 1 host โข Now: high available via Openstack load balancers โข Future: move from LBโs to service discovery โข Minimal amount of shared code: โข Security โข Logging and metrics Backend
- 27. โข Local environment runs only the component(s) that are being developed โข For other components, local env connects to development env on Openstack โข Feature branch based development โข Master branch must always be releasable Development
- 28. โข Unit tests: JUnit, Karma โข Mutation tests with PIT โข Service/integration tests: Spring boot integration, embedded in-memory data stores, REST assured โข End-to-end test: Protractor โข Load tests: Gatling Testing
- 29. Build tools, deployments and running in production
- 30. โข Gitlab โข Jenkins with Docker swarm slave nodes โข Jenkins 2 pipelines โข Nexus โข Sonar Build tools
- 31. โข Every push to master is a release โข Config is embedded in executable jar โข Deployments via Rundeck and Puppet โข Development: deploy on commit โข Everything from dev -> acc during sprint โข Everything from acc -> prod after sprint โข Single service dev -> acc -> prod when needed Deployments
- 32. โข Logging and dashboards via Graylog โข Metrics: โข Spring Boot actuator โข Grafana โข Kafka stats via Burrow โข Monitoring via Sensu and Flapjack Running in production
- 33. Photo: Dave Lehl Challenges and looking ahead
- 34. Challenges 01 Share as little as possible; prefer duplication over coupling. Sharing code between services 04Authentication and authorisation happen at every request. Find the balance between performance and security. Running stateless has a cost When moving fast, donโt forget to finish up before starting something new. Switching focus has a cost 06 Throwing something away and starting over can work out better than refactoring. Donโt be afraid to rebuild03 Microservices are not just for the backend. Modularity is just as imported on the frontend. Monolithic frontend 02 Minimalize dependencies on other teams, or it will slow you down. Cross functional team composition is vital 05 and lessons learned
- 35. Looking ahead Upgrades and fixes without users even noticing. 0-downtime deployments Our plans for the (near) future. @bjschrijver Cross-functional teams with vertical (full stack) responsibilities. Product teams Split the frontend in products and re- usable components. Modular frontend There is no silver bullet here, but useful tools and practices do exist. Automated security testing Get the teams the information they need, but only when they need it. Better dashboards and alerting
- 37. Thanks for your time. Got feedback? Tweet it! All pictures belong to their respective authors @bjschrijver