re:Invent CON320 Tracing and Debugging for Containerized Services

1. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS re:INVENT Tracing and Debugging for Containerized Services C a l v i n F r e n c h - O w e n C o - f o u n d e r , S e g m e n t

8. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Step 1: Build your mental model - How does the system fit together? - How is my service configured? - What should I even be looking at?

9. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Step 1: Build your mental model - How does the system fit together? specs - How is my service configured? - What should I even be looking at?

14. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Step 1: Build your mental model - How does the system fit together? specs - How is my service configured? specs & terraform - What should I even be looking at?

19. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Step 1: Build your mental model - How does the system fit together? specs - How is my service configured? specs & terraform - What should I even be looking at? cloudwatch & datadog

22. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Step 1: Build your mental model - How does the system fit together? specs - How is my service configured? specs & terraform - What should I even be looking at? cloudwatch & datadog

42. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Logging • github.com/segmentio/ecs-logs • github.com/segmentio/ecs-logs-go • github.com/segmentio/ecs-logs-js • github.com/segmentio/rate-limiting-log-proxy • github.com/segmentio/cwlogs

44. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. • BPF Compiler Collection • No kernel modules, no instrumentation • Lots of very useful tools • https://github.com/iovisor/bcc BCC

47. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. • pprof is automatically exposed by the go runtime • gives you profiling, heatmaps, memory dumps and more • nice visualizations, one URL click away • server exposed by consul • github.com/segmentio/pprof- server pprof-server

51. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. $ docker run -it --rm -p 6061:6061 segment/pprof-server -registry consul://172.17.0.1:8500 github.com/segmentio/pprof-server pprof-server

55. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Cattle, not pets - reproduceable machine images - built with packer - run via systems - out-of-the-box autoscaling - created with terraform - github.com/segmentio/roll-instances

59. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. - comes default with any service - tuned for systems like dynamo - no autoscaling == not ready for prod Autoscaling Everywhere

61. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Lessons Learned • Build tools to surface actionable information first • Auto-scaling a huge win • Give developers alerting + scaling policies out of the box • Passive tools are easy to build adoption around

re:Invent CON320 Tracing and Debugging for Containerized Services

More Related Content

More from Calvin French-Owen (6)

Recently uploaded (20)

re:Invent CON320 Tracing and Debugging for Containerized Services