Self-Aware Applications: Automatic Production Monitoring (TechDays NL 2017)
1 like419 views
The document discusses various strategies for performance monitoring in software, highlighting the importance of a hierarchical approach for continuous monitoring of CPU, garbage collection (GC), memory usage, and deadlock detection. It emphasizes the use of performance counters and tools like CLRMD for detailed diagnostics while acknowledging both the benefits and drawbacks of such monitoring systems. Numerous scenarios and demonstrations are provided to illustrate the effectiveness of different monitoring techniques.
Self-Aware Applications: Automatic Production Monitoring (TechDays NL 2017)
- 3. • Motivation • Hierarchy of self-monitoring • CPU profiling • GC monitoring • Heap analysis • Deadlock detection
- 6. • Why monitor? • Why our own?
- 7. • Why monitor? • Why our own?
- 8. • Why monitor? • Why our own?
- 9. • Why monitor? • Why our own?
- 10. • Use a hierarchical monitoring strategy
- 11. • Use a hierarchical monitoring strategy • Lightweight for continuous and frequent monitoring of all basics Performance counters
- 12. • Use a hierarchical monitoring strategy • Lightweight for continuous and frequent monitoring of all basics Performance counters • Medium for less frequency events ETW ClrMD
- 13. • Use a hierarchical monitoring strategy • Lightweight for continuous and frequent monitoring of all basics Performance counters • Medium for less frequency events ETW ClrMD • Invasive for deep-dive and concrete diagnostics CLR Profiling API CLR Debugging API
- 14. Let’s Get to Business
- 15. • Monitor CPU using performance counters • Are we above a certain threshold for a certain amount of time? LiveStacks
- 16. • AuthenticationController takes 95% CPU, maybe we're being DDoS'ed • Image processing component takes 100% CPU, need to auto-scale the app • Encoding this 30 second video takes 3 minutes at 100% CPU, tell the user she can send us a bug report
- 17. CPU Spikes Demo
- 18. • Monitor GC performance using performance counters • Register on ETW’s GC events such as GCAllocationTick • Attach ClrMD to get heap breakdown
- 20. • Monitor memory usage using performance counters • Has memory increased above a certain threshold for a certain time?
- 21. Find a Leak Demo
- 22. • Monitor for potential deadlock • Attach ClrMD to create wait chains and detect deadlocks • Report, try to break, pray for a miracle…
- 24. • Many more scenarios are possible
- 25. • Many more scenarios are possible • Monitor heap fragmentation and compact large objects if needed
- 26. • Many more scenarios are possible • Monitor heap fragmentation and compact large objects if needed • Native memory leak analysis using ETW
- 27. • The pros are obvious (visibility, easy scaling…)
- 28. • The pros are obvious (visibility, easy scaling…) • But there are some cons as well…
- 29. • The pros are obvious (visibility, easy scaling…) • But there are some cons as well…
- 30. • The pros are obvious (visibility, easy scaling…) • But there are some cons as well…
- 31. • The pros are obvious (visibility, easy scaling…) • But there are some cons as well…
- 32. • Self-monitoring is important for all kinds of software • Best to create a hierarchy of monitoring (and overhead and complexity) • Lots of scenarios: CPU, GC, memory, deadlocks • Demos: https://github.com/dinazil/self-aware- applications