Monitoring in the cloud with Puppet
13 likes9,875 views
This document summarizes a presentation about monitoring in the cloud using Puppet. It discusses challenges with traditional monitoring like Nagios being difficult to scale in cloud environments. It promotes using Puppet to configure monitoring tools like Collectd, Logstash, Graphite and Icinga in a dynamic and reproducible way. Specific examples are given around using Puppet to export monitoring configurations and collect metrics that are then graphed in Graphite.
Monitoring in the cloud with Puppet
- 1. Monitoring in the Cloud with Puppet PuppetCamp Ghent 2013 Kris Buytaert
- 2. Kris Buytaert ● I used to be a Dev, ● Then Became an Op ● Chief Trolling Officer and Open Source Consultant @inuits.eu ● Everything is an effing DNS Problem ● Building Clouds since before the bookstore ● Some books, some papers, some blogs ● Evangelizing devops
- 3. Do you know what your children do at 5 am in the morning ? ● Sleeping ? ● Crashing a Party ? ● Why are the cops at your front door ? ● What happened ? ● How long have they been gone ?
- 4. devops ● Culture ● (Lean) ● Automation ● Measurement ● Sharing Damon Edwards and John Willis Gene Kim
- 5. Monitoring is usually an aftertought ENOBUDGET, ENOTIME
- 6. A sub movement ● #monitoringsucks trending ● https://github.com/monitoringsucks/ ● 2008 Study :Nagios + Friends ● 2011 Conclusion : Nagios/Icinga are the only automatable alternatives
- 7. #monitoringsucks Monitoring is AWESOME. Metrics are AWESOME. I love it. Here's what I don't love: ● Having my hands tied with the model of host and service bindings. ● Having to set up "fake" hosts just to group arbitrary metrics together ● Having to either collect metrics twice - once for alerting and another for trending ● Only being able to see my metrics in 5 minute intervals ● Having to chose between shitty interface but great monitoring or shitty monitoring but great interface ● Dealing with a monitoring system that thinks IT is the system of truth for my environment ● Not actually having any real choices John Vincent (@lusis) on his blog http://lusislog.blogspot.com/2011/06/why- monitoring-sucks.html
- 9. Let's forget about ● GroundWork ● Cacti ● Zenoss ● Munin ● Zabbix ● BigBrother ● Hyperic
- 10. What's different in the cloud ? ● Scale ● Velocity ● Change
- 14. Challenges ● Reproducability ● Speed ● Auditing ● Keeping stuff in sync
- 15. Thank god Luke for Puppet
- 16. And Mark for Cfengine
- 17. } #monitoringsucks 2013
- 18. What we want ● Small , wel suited components • Collect • Transport / Mangle • Analyse / Act • Visualize
- 20. Collectd ● Collects ● Zillion Plugins • Nginx,apache, mysql, disk ● Graphite Carbon Plugin ● Send metrics to graphite ● https://github.com/KrisBuytaert/pupp et-collectd
- 21. ● Collect from anywhere ● Filter ● Send anywhere
- 22. Puppet and Logstash ● https://github.com/KrisBuytaert/vagrant-puppet-logstash ● Includes Logstash / Kibana / Grok / ES modules as submodules ! git clone ● git submodule init ● git submodule update ● vagrant up
- 23. Graphite ● Graphing at Scale ● Graphing at Ease ● Any metric is a graph ● echo "somestring $somevalue $timestamp" | nc <%= graphitehost %> 2003
- 24. Graphite Composer x
- 25. Graphite API
- 26. Gdash In action
- 27. Puppet and Graphite ● https://github.com/KrisBuytaert/vagrant-graphite/ ● Includes Graphite / Gdash / Jmxtrans / Logster / Collectd / Statsd / Tattle and more modules as submodules ! ● git clone ● git submodule init ● git submodule update ● vagrant up
- 29. Configuring a Service ● Package ● Config ● Service <- we pray status is correct
- 30. Apache Example:
- 31. Definition of Done: Done means : being monitored
- 32. Icinga ? • Isn't nagios dead ? • Vibrant Community • Throw great parties in Nurnberg • Nobody can pronounce it anyhow • https://github.com/Inuits/puppet-icinga/
- 33. Monitoring Baseline ● Deploy a host, ● Add it to the monitoring ● Add collection tools ● Add check definitions ● Update the monitoring tool config
- 34. Stored Configs
- 35. Collection and Export Export : Collect: @@resource { Resource <<| query |>> ... } Clean out nodes that dissapear puppet node clean
- 38. It is not in production if it is not monitored
- 39. Triggers on Graphs ● Export Java Metrics ● Collect JMX Exports on JMXTransNode ● JMXTrans ● Graph Em ● Export JMXConfigs Collect Nagios Configs ● Configure NRPE Check on Nagios Server ● Export NagiosCheck
- 42. Conclusion: ● Reproducable monitoring setup ● Dynamically generated monitoring config ● Code is available at github.com/KrisBuytaert/snippets
- 43. But, (pronounced in Ozzy dialect)
- 45. #monitoringlove
- 46. Homework Sensu Riemann Extrememon Esper ...
- 47. Contact Kris.Buytaert@inuits.eu Further Reading @krisbuytaert http://www.krisbuytaert.be/blog/ http://www.inuits.eu/ Inuits Duboistraat 50 2060 Antwerpen Belgium 891.514.231 +32 475 961221