Observable Microservices (O'Reilly SACon London 2018)
6 likes969 views
The document discusses the principles of microservices, emphasizing the importance of observability, which is defined as the ability to interrogate a system for better understanding. It outlines concepts such as logging, monitoring, tracing, and alerting, suggesting practices for effective implementation. Key takeaways include the need for structured data consumption, intentional logging, and the use of existing tools to enhance observability in microservices architectures.
Observable Microservices (O'Reilly SACon London 2018)
- 2. 2 Head of Technology Thoughtworks Spain ©ThoughtWorks, Inc. 2018
- 5. PRINCIPLES OF MICROSERVICES ©ThoughtWorks, Inc. 2018
- 6. PRINCIPLES OF MICROSERVICES • Modeled around business concepts • Culture of automation • Hide internal implementation details • Decentralize all the things • Deploy independently • Isolate Failure • Highly observable ©ThoughtWorks, Inc. 2018
- 8. WHAT IS OBSERVABILITY? “Observability is the ability to interrogate your system and get accurate answers that improve your understanding of it” ©ThoughtWorks, Inc. 2018
- 9. Logging Monitoring Tracing Visualization & Alerting ©ThoughtWorks, Inc. 2018
- 14. Some suggestions ©ThoughtWorks, Inc. 2018
- 20. LOG AGGREGATION ©ThoughtWorks, Inc. 2018
- 21. LOG STRUCTURED DATA { CreationTime: “2017-02-24T17:49:15”, Id: “9fe0650565-3dac-425a-c83e38499c”, Host: “WEBSRVR06”, ServiceId: “MyShoppingCartService”, ComponentId: “DynamoDBWriter”, Message: “Body size exceeded 400kb limit” …. …. } ©ThoughtWorks, Inc. 2018
- 22. TRACK BUSINESS AND SYSTEM EVENTS { CreationTime: “2017-02-24T17:49:15”, Id: “9fe0650565-3dac-425a-c83e38499c”, Host: “WEBSRVR06”, ServiceId: “MyShoppingCartService”, Type: “BusinessEvent”, Event: “ItemAddedToShoppingCart”, ItemDetail: {…} …. …. ©ThoughtWorks, Inc. 2018
- 23. STANDARDIZE •Standardize certain keys across services •Bake logging into your service template •Build a common log aggregation infrastructure ©ThoughtWorks, Inc. 2018
- 25. Have enough information to help you make decisions ©ThoughtWorks, Inc. 2018
- 26. Evolves with your system ©ThoughtWorks, Inc. 2018
- 28. EXAMPLE Online marketplace 3M+ articles 10M+ users per month ©ThoughtWorks, Inc. 2018
- 30. EXAMPLE Data Stream A Data Stream B Service ©ThoughtWorks, Inc. 2018
- 31. EXAMPLE Service /health { “app” : “ok” } ©ThoughtWorks, Inc. 2018
- 32. EXAMPLE Service /health { “app” : “ok”, “dataStreamA” : “ok”, “dataStreamB” : ”ok” } Data Stream A Data Stream B ©ThoughtWorks, Inc. 2018
- 33. EXAMPLE Data Stream A Data Stream B Service ©ThoughtWorks, Inc. 2018
- 34. EXAMPLE Data Stream A Data Stream B ServiceDB ©ThoughtWorks, Inc. 2018
- 35. EXAMPLE Service /health { “app” : “ok”, “database” : “ok”, “dataStreamA” : “ok”, “dataStreamB” : ”ok” } DB Data Stream A Data Stream B ©ThoughtWorks, Inc. 2018
- 38. It is accessible by everyone in the team ©ThoughtWorks, Inc. 2018
- 43. Push notifications: Alerts ©ThoughtWorks, Inc. 2018
- 44. ALERTS SHOULD BE ACTIONABLE Title: ConnectionTimeOutException Description: Connection to service A has timed out More info: <link to visualization tool> How to act: <link to run book> ©ThoughtWorks, Inc. 2018
- 46. SMART ALERTS Service /customers Send an email to owner for every 5xx error ©ThoughtWorks, Inc. 2018
- 47. SMART ALERTS Service /customers Raise a high priority alarm when >10 5xx errors in 10 minutes Send an email to owner for every 5xx error ©ThoughtWorks, Inc. 2018
- 48. SMART ALERTS ©ThoughtWorks, Inc. 2018
- 49. SMART ALERTS ©ThoughtWorks, Inc. 2018
- 50. SMART ALERTS ©ThoughtWorks, Inc. 2018
- 52. ABILITY TO FIND THE ORIGIN OF A REQUEST Client 1 …. Client N Third party 1 Third party N Mainframe 1 Mainframe N …. Platform Domain1 …. Domain1 Domain1 Domain1 Domain1 Domain2 Domain1 Domain1 DomainN …. ©ThoughtWorks, Inc. 2018
- 53. ABILITY TO FIND THE ORIGIN OF A REQUEST Client 1 …. Client N Third party 1 Third party N …. Mainframe 1 Mainframe N …. Platform Domain1 …. Domain1 Domain1 Domain1 Domain1 Domain2 Domain1 Domain1 DomainN ©ThoughtWorks, Inc. 2018
- 54. ABILITY TO FIND THE ORIGIN OF A REQUEST Client 1 …. Client N Third party 1 Third party N …. Mainframe 1 Mainframe N …. Platform Domain1 …. Domain1 Domain1 Domain1 Domain1 Domain2 Domain1 Domain1 DomainN TraceID TraceID TraceID TraceID TraceID ©ThoughtWorks, Inc. 2018
- 55. LOG AGGREGATOR ©ThoughtWorks, Inc. 2018
- 56. LOG AGGREGATOR ©ThoughtWorks, Inc. 2018
- 57. LOG AGGREGATOR ©ThoughtWorks, Inc. 2018
- 58. LOG AGGREGATOR ©ThoughtWorks, Inc. 2018
- 59. REQUEST GRAPH ©ThoughtWorks, Inc. 2018
- 63. SERVICE MESH ©ThoughtWorks, Inc. 2018 Service A Service B Service C Service D Service Mesh Service Mesh Service Mesh Service Mesh Tracing system
- 65. ©ThoughtWorks, Inc. 2018 “Observability is the ability to interrogate your system and get accurate answers that improve your understanding of it”
- 66. Logging Monitoring Tracing Visualization & Alerting ©ThoughtWorks, Inc. 2018
- 67. TAKE AWAYS • Structure your data so it can be consumed by other tools • Be intentional about things you log, monitor, and alert on • Proactively iterate and eliminate noise • Evaluate and use existing tools and libraries (OpenTracing, Honeycomb, ioPipe, Datadog, Opsgenie, etc) • Service templates to bootstrap microservices over shared libraries. Service mesh if required. ©ThoughtWorks, Inc. 2018