Java il spanning services 2019
0 likes32 views
This document provides a practical guide to distributed tracing. It discusses the limitations of traditional metrics and logging and introduces use cases for distributed tracing such as optimizing slow segments, tests, and network latency. It then covers standard instrumentation techniques for HTTP calls and clients. Additional techniques discussed include method tracing, database tracing, and including stack traces. The document concludes with tips for setting up a distributed tracing system, including using Zipkin and Jaeger, and best practices around planning, resilience, and downsampling.
Java il spanning services 2019
- 1. Spanning Services The Practical Guide to DistributedTracing Yair Galler, @ygallerJava IL, June 2019
- 2. The Starting Point We already have metrics and logging! but...
- 3. Metrics are aggregates A metric in Grafana goes crazy as the team stares. Root cause unknown. Michael Ancher, 1902
- 4. Logging lacks context A developer realizes he has been looking at the wrong set of logs for the last 4 hours. Auguste Couder, 1817
- 5. Use Cases
- 6. Use Cases - Optimization Slow Running Segments Waiting for tests to complete. A dream of deployment by Edmund Leighton, 1922
- 7. Use Cases - Optimization SlowTests
- 8. Use Cases - Optimization Parallelization
- 9. Use Cases - Optimization Network Latency
- 10. Use Cases - Optimization Call Redundancy Multiple calls can be batched into one
- 11. Use Cases - Architecture Service Boundary Design Flaws Stateless edge service Central backend service
- 12. Use Cases - Debugging Run Comparisons
- 13. TheTracing Cookbook Standard Instrumentation Incoming HTTP calls - Supported by DropWizard & Spring HTTP Clients - Register a tracer
- 14. TheTracing Cookbook Extras - MethodTracing Using annotations & interceptors Better tracing resolution When you cannot trace calls from third party libraries
- 15. TheTracing Cookbook Extras - JDBITracing Invaluable for catching slow queries! *We used opentracing-jdbi3
- 16. TheTracing Cookbook Extras - StackTraces HTTP calls can come from various paths
- 17. TheTracing Cookbook Setting UpThe System -WhatWorked for us Zipkin Client Jaeger Server - Multiple instances for high availability Terraform for auto-deploy to AWS Data stored on AWS Elasticsearch Service
- 18. Tips & Best Practices You are going to invest time - Plan ahead and consider: Lambdas Kubernetes / containers in general Get buy-in from all the teams or the relay breaks Ensure system runs well when tracing server is down Downsampling Don’t trace everything!