Microservices meetup April 2017
0 likes712 views
SignalFx is an advanced monitoring and alerting system for cloud applications delivered as SaaS. It provides real-time metrics, analytics, and tagging to monitor microservices architectures. Traditional monitoring approaches are noisy and reactive, while SignalFx aims to provide guided triage and correlate events using time series analytics to identify patterns and anomalies.
![METRICS IN A TIME SERIES
{
"gauge":
[{"metric":”cpu.idle",
"dimensions":
{"host":”hostname123",
"datacenter":”snc"},
"value":249}]
}
{
"gauge":
[{"metric":”cpu.idle",
"dimensions":
{"host":”hostname123",
"datacenter":”snc"},
"value":230}]
}
{
"gauge":
[{"metric":”cpu.idle",
"dimensions":
{"host":”hostname123",
"datacenter":”snc"},
"value":202}]
}
{
"gauge":
[{"metric":”cpu.idle",
"dimensions":
{"host":”hostname123",
"datacenter":”snc"},
"value":284}]
}
10:15:02 10:15:03 10:15:04 10:15:05](https://image.slidesharecdn.com/microservicesmeetupapr17v1-170427204831/85/Microservices-meetup-April-2017-9-320.jpg)
Microservices meetup April 2017
- 1. MONITORING MICROSERVICES APRI L 2 6 , 2 0 1 7 Rich Schofield rschofield@signalfx.com Jamison Clouthier jamison@signalfx.com
- 2. Karthik Rau, CEO VMware, VP Products Loudcloud/Opsware, Products S E L E C T C U S T O M E R S I N V E S T O R S Phillip Liu, CTO Facebook, Lead Architect Loudcloud/Opsware, Chief Architect F O U N D E R S A LITTLE BACKGROUND SignalFx is an advanced monitoring & alerting system for cloud apps, delivered as a SaaS solution
- 3. SIGNALFX HIGH LEVEL ARCHITECTURE METADATA 1s / 5s 1m 1h ••• MESSAGEBUS TIME SERIES DATABASE ANALYTICS 25 INGEST NOTIFICATION 8d 384d 32d 1s 16m (IN MEMORY) DATAPOINTS MICROSERVICES
- 4. BENEFITS OF MICROSERVICES ARCHITECTURE Application modules released independently • Enables agile development per team • Canary deployments for testing • Containers to enable rapid automated deployment & rollback Scale in/out with service load • Respond quickly to increased data flow or usage • Optimize infrastructure costs as load falls Technology flexibility • Easier to upgrade components or change platforms
- 5. MONITORING A MICROSERVICE APPLICATION Real-time metrics and analytics • Detect issues and trends quickly Apply context: history, related events, service metadata • Metrics without context are just numbers! Tag-based monitoring for elastic/ephemeral services • Adapt automatically to monitor services as they scale in/out Shared, self-service access across all groups • App developers, tech leads, support, management • Avoid different tools for different teams
- 6. ISSUES WITH TRADITIONAL MONITORING Noisy, reactive monitoring C H A L L E N G E • Too many alerts fire at once for a cluster-wide problem • Is the machine down because we scaled down the cluster or because we had a real problem? • Do we even care if a single node is down? • Component-specific monitoring configurations that require constant maintenance in ephemeral/elastic environments What matters? Where to start? ?
- 7. THE MODERN MONITORING LANDSCAPE A P M M E T R I C S L O G S Performance Testing Pre-Flight Streaming Metrics Aggregated In-Flight Black Box Recorder Post-Flight Luxury of TimeReal Time MattersLuxury of Time
- 8. LET’S TALK METRICS Metric name Metric value Metric type Timestamp Dimensions cpu.idle 27 gauge 1234567 host = relic47df datacenter = sjc1 env = prod … Dimensions allow you to filter, aggregate, compare across sources
- 9. METRICS IN A TIME SERIES { "gauge": [{"metric":”cpu.idle", "dimensions": {"host":”hostname123", "datacenter":”snc"}, "value":249}] } { "gauge": [{"metric":”cpu.idle", "dimensions": {"host":”hostname123", "datacenter":”snc"}, "value":230}] } { "gauge": [{"metric":”cpu.idle", "dimensions": {"host":”hostname123", "datacenter":”snc"}, "value":202}] } { "gauge": [{"metric":”cpu.idle", "dimensions": {"host":”hostname123", "datacenter":”snc"}, "value":284}] } 10:15:02 10:15:03 10:15:04 10:15:05
- 10. USING TIME SERIES ANALYTICS TO CORRELATE AND IDENTIFY PATTERNS How well load balanced is this 8-node Kafka cluster? Compare the signal against historical patterns and alert on anomalous patterns Create a signal to represent the cluster’s load balancing effectiveness, computed within seconds
- 11. GUIDED TRIAGE 0 2 4 6 8 10 12 14 16 18 20 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 INFRASTRUCTURE SERVICE PRODUCT LAUNCH EVENT CORRELATE EVENTS TO MONITORING IS THIS A PROBLEM? ANALYSIS 1. PRODUCT LAUNCH (EVENT RECORDED) 2. TRAFFIC SPIKE 3. TRANSIENT? 4. IMPACT EXISTING CUSTOMERS? 5. IS IT LEVELING OUT? 6. RAM ISSUE? 7. STORAGE ISSUE? 8. BUS BACKED UP?
- 12. JOURNEY TO METRICS BASED MONITORING PHASE 0 PHASE 2PHASE 1 PHASE 3 Health checks and logs Small internal metrics system Build out scalable, highly-available metrics system Build out more sophisticated analytics From individual component checks to proactive management of service-wide performance
- 13. M M / D D / Y Y YOUR TITLE HERE P R E P A R E D F O R : P L A C E L O G O H E R EDEMO
- 14. M M / D D / Y Y YOUR TITLE HERE P R E P A R E D F O R : P L A C E L O G O H E R E T H A N K Y O U ! jamison@signalfx.com rschofield@signalfx.com S I G N U P F O R A T R I A L A T : signalfx.com