GeekOut 2017 - Microservices in action at the Dutch National Police
3 likes1,684 views
The document discusses the implementation of microservices at the Dutch National Police, highlighting the structure of development teams and their methodologies. It describes the architecture and technologies used, including a private cloud environment and continuous delivery practices. Challenges like minimizing dependencies and achieving zero-downtime deployments are also addressed, alongside future plans for modular frontends and automated security testing.
GeekOut 2017 - Microservices in action at the Dutch National Police
- 1. bertjan@jpoint.nl Microservices in action at the Dutch National Police Bert Jan Schrijver @bjschrijver
- 2. Bert Jan Schrijver L e t ’ s m e e t @bjschrijver
- 3. Architecture and platform Frontend Methodology and culture Introduction Development and testing Build tools, deployments and running in production Challenges and looking ahead Outline W h a t ‘ s n e x t ? Backend
- 4. The police protects the democracy, maintains the law and is the authority on the streets. Around 65.000 people work at the Dutch police, of which over 1500 IT professionals. Dutch National Police
- 5. CLOUD PLATFORM ANALYSE PATRONEN BIG DATA SECURITY3 DevOps teams are building high tech big data web applications in a private cloud environment. These applications support police related themes. Product line Cloud | Big Data | Internet
- 7. • 3 teams, separate backlogs • Overall planning at start of sprint • Minimal planning ritual • Usability tests as part of sprint • Phabricator as tool of choice Methodology
- 8. • Continuous Delivery & DevOps • Short feedback loops • Embrace change • Minimal dependencies outside team • Invest in people, not in products • Open, transparent, verifiable Culture Source: http://kids.nationalgeographic.com/explore/countries/netherlands/#netherlands-tulip-fields.jpg
- 10. • End-to-end security and encryption • Version control for everything • Horizontally scalable, no single point of failure • No dependencies on external sources • Standardised naming • Application config lives with code • Services defined by business functionality Architecture
- 11. Architecture • This slide is intentionally left blank…
- 20. Source: https://www.google.com/about/datacenters • Openstack private cloud • General cloud services for police organisation • Ceph distributed storage • Puppet & Ansible for config management • 3000 managed desktops • Automation starts when hardware boots Platform
- 21. Frontend
- 22. • Angular 4.x • TypeScript • RxJS • Bootstrap • Responsive design • Feature toggles • Graceful degradation when backend fails Frontend
- 23. Backend
- 24. • Small in size, single responsibility • Runs in its own process • Independently develop, deploy, upgrade, scale • Has its own data store • Distributed by default • Potentially heterogeneous/polyglot • Light-weight communication Anatomy of a microservice
- 25. • Spring Boot, Java 8, Maven • Stateless • 1 service in 1 jar on 1 JVM on 1 host • Now: high available via OpenStack load balancers • Future: move from LB’s to service discovery • Minimal amount of shared code: • Security • Logging and metrics Backend
- 27. • Local environment runs only the component(s) you work on • For other components, local env connects to development env on OpenStack • Feature branch based development • Master branch must always be releasable Development
- 28. • Unit tests • Mutation tests • Service/integration tests: Spring boot integration, embedded in-memory data stores, REST assured • End-to-end test: Protractor • Load tests: Gatling Testing
- 29. Build tools, deployments and running in production
- 30. • Gitlab • Jenkins with Docker swarm slave nodes • Jenkins 2 pipelines • Nexus • Sonar Build tools
- 31. • Every push to master is a release • Config embedded in executable jar • Deployments via Rundeck and Puppet: • Development: deploy service on commit • Everything from dev -> acc during sprint • Everything from acc -> prod after sprint • Single service dev -> acc -> prod when needed Deployments
- 32. • Logging and dashboards via Graylog • Metrics: • Spring Boot actuator • Grafana • Kafka stats via Burrow • Monitoring via Sensu and Flapjack Running in production
- 33. Photo: Dave Lehl Challenges and looking ahead
- 34. Challenges 01 Share as little as possible; prefer duplication over coupling. Sharing code between services 04Authentication and authorisation happen at every request. Find the balance between performance and security. Running stateless has a cost When moving fast, don’t forget to finish up before starting something new. Switching focus has a cost 06 Throwing something away and starting over can work out better than refactoring. Don’t be afraid to rebuild03 Microservices are not just for the backend. Modularity is just as imported on the frontend. Monolithic frontend 02 Minimalize dependencies on other teams, or it will slow you down. Cross functional team composition is vital 05 and lessons learned
- 35. Looking ahead Upgrades and fixes without users even noticing. 0-downtime deployments Our plans for the (near) future. @bjschrijver Cross-functional teams with vertical (full stack) responsibilities. Product teams Split the frontend in products and re- usable components. Modular frontend There is no silver bullet here, but useful tools and practices do exist. Automated security testing Get the teams the information they need, but only when they need it. Better dashboards and alerting
- 37. Thanks for your time. Got feedback? Tweet it! All pictures belong to their respective authors @bjschrijver