Continuous monitoring
0 likes113 views
The document discusses the requirements for a new monitoring solution, including that it must be automatic, lightweight, able to run on-site, and compatible with various platforms. It also describes how Sensu can meet these requirements by automatically collecting, storing, and displaying metrics and events using its publish-subscribe model and transport via RabbitMQ. Sensu converts data to line protocol for storage in InfluxDB and visualization in Grafana.
![#
~
{
"checks":{
"cpu_metrics": {
"command": "/etc/sensu/plugins/metrics/cpu-metrics.rb",
"interval": 20,
"handlers": [ "influxdb" ]
}
}
}
cat /etc/sensu/conf.d/cpu_metrics.json
$](https://image.slidesharecdn.com/jz2016-continuousmonitoring-211208170231/85/Continuous-monitoring-39-320.jpg)
Continuous monitoring
- 1. truls jørgensen johnny horvi og monitoring continuous
- 2. truls jørgensen johnny horvi javazone 2016 og monitoring continuous
- 3. monitoring s “å observere, ta opp eller oppdage en hendelse eller tilstand med et instrument uten å påvirke den.”
- 4. monitoring “å observere, ta opp eller oppdage en hendelse eller tilstand med et instrument uten å påvirke den.”
- 5. 40 80 120 160 200 0 10 20 30 40 50 60 6 5 4 3 2
- 8. 114
- 9. truls johnny
- 10. Prod
- 12. ! dev " # app
- 16. automatisk må kunne settes opp
- 17. tilgjengelig må være for alle i alle miljøer automatisk
- 20. automatisk tilgjengelig lettvekt on site egne metrikker må være enkelt for applikasjoner å lage
- 21. automatisk tilgjengelig lettvekt on site egne metrikker komponere må være enkelt å og bytte med andre verktøy
- 23. automatisk tilgjengelig lettvekt on site egne metrikker komponerbar open source ulike plattformer må kunne brukes på
- 27. samle lagre vise
- 33. $ $ $ $ $ … … 4000 6
- 36. & push me # $ sensu client applikasjon
- 37. # $ SensuClient.sendEvent( "nySykmelding.event", ImmutableMap.of("value", 1) ); sensu client applikasjon
- 38. $ # & pull me sensu client applikasjon
- 39. # ~ { "checks":{ "cpu_metrics": { "command": "/etc/sensu/plugins/metrics/cpu-metrics.rb", "interval": 20, "handlers": [ "influxdb" ] } } } cat /etc/sensu/conf.d/cpu_metrics.json $
- 40. # ~ /etc/sensu/plugins/metrics/cpu-metrics.rb $ cpu.user 0.40 1472720920 cpu.nice 0.00 1472720920 cpu.system 0.20 1472720920 cpu.idle 99.20 1472720920 cpu.iowait 0.20 1472720920 cpu.irq 0.00 1472720920 cpu.softirq 0.00 1472720920 cpu.steal 0.00 1472720920 cpu.guest 0.00 1472720920
- 41. # cpu.user 0.40 1472720920 cpu.nice 0.00 1472720920 cpu.system 0.20 1472720920 cpu.idle 99.20 1472720920 cpu.iowait 0.20 1472720920 cpu.irq 0.00 1472720920 cpu.softirq 0.00 1472720920 cpu.steal 0.00 1472720920 cpu.guest 0.00 1472720920 ~ /etc/sensu/plugins/metrics/cpu-metrics.rb $ '
- 42. cpu.user 0.40 1472720920 cpu.nice 0.00 1472720920 cpu.system 0.20 1472720920 cpu.idle 99.20 1472720920 cpu.iowait 0.20 1472720920 cpu.irq 0.00 1472720920 cpu.softirq 0.00 1472720920 cpu.steal 0.00 1472720920 # $ "tags": { "environment": "p", "application": "syfoservice", "hostname": "a00069.adeo.no" } '
- 43. cpu.user 0.40 1472720920 cpu.nice 0.00 1472720920 cpu.system 0.20 1472720920 cpu.idle 99.20 1472720920 cpu.iowait 0.20 1472720920 cpu.irq 0.00 1472720920 cpu.softirq 0.00 1472720920 cpu.steal 0.00 1472720920 "tags": { "environment": "p", "application": "syfoservice", "hostname": "a00069.adeo.no" ' # $
- 44. $ $ $ $ … … ' ' ' ' $
- 45. cpu.user 0.40 1472720920 cpu.nice 0.00 1472720920 cpu.system 0.20 1472720920 cpu.idle 99.20 1472720920 cpu.iowait 0.20 1472720920 cpu.irq 0.00 1472720920 cpu.softirq 0.00 1472720920 cpu.steal 0.00 1472720920 cpu.guest 0.00 1472720920 "tags": { "environment": "p", "application": "syfoservice", "hostname": "a00069.adeo.no" ' $ …
- 46. $ cpu.user 0.40 1472720920 cpu.nice 0.00 1472720920 cpu.system 0.20 1472720920 cpu.idle 99.20 1472720920 cpu.iowait 0.20 1472720920 cpu.irq 0.00 1472720920 cpu.softirq 0.00 1472720920 cpu.steal 0.00 1472720920 cpu.guest 0.00 1472720920 "tags": { "environment": "p", "application": "syfoservice", "hostname": "a00069.adeo.no" } '
- 47. $ cpu.user 0.40 1472720920 cpu.nice 0.00 1472720920 cpu.system 0.20 1472720920 cpu.idle 99.20 1472720920 cpu.iowait 0.20 1472720920 cpu.irq 0.00 1472720920 cpu.softirq 0.00 1472720920 cpu.steal 0.00 1472720920 cpu.guest 0.00 1472720920 ( github.com/jhrv/sensu-influxdb-extension "tags": { "environment": "p", "application": "syfoservice", "hostname": "a00069.adeo.no" }
- 48. $ cpu.user 0.40 1472720920 cpu.nice 0.00 1472720920 cpu.system 0.20 1472720920 cpu.idle 99.20 1472720920 cpu.iowait 0.20 1472720920 cpu.irq 0.00 1472720920 cpu.user 0.40 1472720920 "tags": { "environment": "p", "application": "syfoservice", "hostname": "a00069.adeo.no" } ( influxdb lineprotocol fields key tags timestamp
- 49. $ cpu.user 0.40 1472720920 cpu.nice 0.00 1472720920 cpu.system 0.20 1472720920 cpu.idle 99.20 1472720920 cpu.iowait 0.20 1472720920 cpu.irq 0.00 1472720920 cpu.user 0.40 1472720920 "tags": { "environment": "p", "application": "syfoservice", "hostname": "a00069.adeo.no" } environment application hostname p a00069.adeo.no ( konvertering til lineprotocol syfoservice , fields key tags timestamp
- 50. , $ cpu.user 0.40 1472720920 cpu.nice 0.00 1472720920 cpu.system 0.20 1472720920 cpu.idle 99.20 1472720920 cpu.iowait 0.20 1472720920 cpu.irq 0.00 1472720920 ( konvertering til lineprotocol fields timestamp key tags cpu.user 0.40 1472720920 , value= "tags": { "environment": "p", "application": "syfoservice", "hostname": "a00069.adeo.no" } environment application hostname p syfoservice a00069.adeo.no
- 51. , $ cpu.user 0.40 1472720920 cpu.nice 0.00 1472720920 cpu.system 0.20 1472720920 cpu.idle 99.20 1472720920 cpu.iowait 0.20 1472720920 cpu.irq 0.00 1472720920 ( konvertering til lineprotocol fields timestamp key tags cpu.user 0.40 1472720920 environment application hostname p syfoservice a00069.adeo.no , = , = , = value= "tags": { "environment": "p", "application": "syfoservice", "hostname": "a00069.adeo.no" }
- 52. $ ( sammenstilling cpu.user 0.40 1472720920 environment application hostname p syfoservice a00069.adeo.no , = , = , = value= ….… 2000 nySykmelding.event,environment=p,application=syfoservice,hostname=a06969.adeo.no value=1 147272099 cpu.nice,environment=p,application=syfoservice,hostname=a06969.adeo.no value=0.00 1472720920 cpu.idle,environment=p,application=syfoservice,hostname=a06969.adeo.no value=0.00 1472720920 cpu.system,environment=p,application=syfoservice,hostname=a06969.adeo.no value=0.00 1472720920
- 53. $ ' http cpu.user 0.40 1472720920 environment application hostname p syfoservice a00069.adeo.no , = , = , = value= ….…2000 nySykmelding.event,environment=p,application=syfoservice,hostname=a06969.adeo.no value=1 147272099 cpu.nice,environment=p,application=syfoservice,hostname=a06969.adeo.no value=0.00 1472720920 cpu.idle,environment=p,application=syfoservice,hostname=a06969.adeo.no value=0.00 1472720920 cpu.system,environment=p,application=syfoservice,hostname=a06969.adeo.no value=0.00 1472720920
- 54. ….… 2000 nySykmelding.event,environment=p,application=syfoservice,hostname=a06969.adeo.no value=1 147272099 cpu.nice,environment=p,application=syfoservice,hostname=a06969.adeo.no value=0.00 1472720920 cpu.idle,environment=p,application=syfoservice,hostname=a06969.adeo.no value=0.00 1472720920 cpu.system,environment=p,application=syfoservice,hostname=a06969.adeo.no value=0.00 1472720920 cpu.user 0.40 1472720920 environment application hostname p syfoservice a00069.adeo.no = value= $ ' http
- 55. $ ' ' ' ~700 000 000 datapunkter/dag $ $ $ $ … … ~150GB /år
- 56. $ /query?q=select mean(‘value’) from nySykmelding.event … … /write?db=metrics…
- 57. $
- 58. $
- 59. 114
- 61. 114
- 62. 114
- 63. 114
- 69. automatisk tilgjengelig lettvekt on site egne metrikker komponerbar open source ulike plattformer det som var viktig for oss
- 70. det som var viktig for oss $ $ $ $ … …
- 74. #