SlideShare a Scribd company logo

Integrating icinga2 and the HashiCorp suite

Bram Vogelaar, profile picture
Bram Vogelaar

This document discusses integrating various HashiCorp tools like Packer, Vagrant, Terraform, Consul, and Vault with Icinga monitoring. Packer is used to build machine images while Vagrant provisions virtual machines. Terraform models infrastructure as code and can integrate with Icinga to provision hosts, checks, and notifications. Consul provides service discovery and can trigger Icinga config deployment. Vault manages secrets and certificates that could be used for authentication in Icinga. The presenter demonstrates using these tools together for infrastructure as code and monitoring workflows.

Technology
1 of 46
Download to read offline
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
Integrating icinga2 and the HashiCorp suite Bram Vogelaar
~$ whoami~$ whoami ● I used to be a Molecular Biologist,I used to be a Molecular Biologist, ● Then became a Dev,Then became a Dev, ● Now an Ops.Now an Ops. ● Open Source Consultant @Open Source Consultant @inuits.euinuits.eu
Integrating icinga2 and the HashiCorp suite
PackerPacker ● Open Source tool to make OS imagesOpen Source tool to make OS images ● Supports Cloud Providers, Docker, Vbox, …Supports Cloud Providers, Docker, Vbox, … (builders)(builders) ● Has hooks to provision the base imagesHas hooks to provision the base images (provisioners)(provisioners) ● Create artifacts (Post-Processors)Create artifacts (Post-Processors)
PackerPacker {{ "builders": ["builders": [ {{ "type": "azure-arm","type": "azure-arm", ...... }} ],], "provisioners": ["provisioners": [ {{ "scripts": ["scripts": [ "scripts/common/consul_install.sh","scripts/common/consul_install.sh", "scripts/common/puppet_install.sh","scripts/common/puppet_install.sh", "scripts/common/puppet_icinga.sh""scripts/common/puppet_icinga.sh" ],], "type": "shell","type": "shell", }} ]] }} $ packer build template.json$ packer build template.json
VagrantVagrant ● Open Source tool to bootstrap vmsOpen Source tool to bootstrap vms ● Supports many vm Providers, Docker, Vbox, …Supports many vm Providers, Docker, Vbox, … ● Has hooks to provision the base imagesHas hooks to provision the base images (provisioners), Puppet, Chef, Ansible(provisioners), Puppet, Chef, Ansible
VagrantfileVagrantfile # -*- mode: ruby -*-# -*- mode: ruby -*- # vi: set ft=ruby :# vi: set ft=ruby : Vagrant.configure("2") do |config|Vagrant.configure("2") do |config| config.vm.box = "base"config.vm.box = "base" # config.vm.box_check_update = false# config.vm.box_check_update = false # config.vm.network "forwarded_port", guest: 80, host: 8080# config.vm.network "forwarded_port", guest: 80, host: 8080 # config.vm.network "private_network", ip: "192.168.33.10"# config.vm.network "private_network", ip: "192.168.33.10" # config.vm.network "public_network"# config.vm.network "public_network" # config.vm.synced_folder "../data", "/vagrant_data"# config.vm.synced_folder "../data", "/vagrant_data" # config.vm.provider "virtualbox" do |vb|# config.vm.provider "virtualbox" do |vb| # vb.gui = true# vb.gui = true # vb.memory = "1024"# vb.memory = "1024" # end# end # config.vm.provision "shell", inline: <<-SHELL# config.vm.provision "shell", inline: <<-SHELL # apt-get update# apt-get update # apt-get install -y apache2# apt-get install -y apache2 # SHELL# SHELL endend
Try Icinga Yourself!Try Icinga Yourself! https://github.com/Icinga/icinga-vagranthttps://github.com/Icinga/icinga-vagrant
By Icinga, for IcingaBy Icinga, for Icinga ● StandaloneStandalone ● DistributedDistributed ● InfluxDBInfluxDB ● ElasticElastic ● GraylogGraylog
Docs for all!Docs for all!
Puppet WorkflowPuppet Workflow
Exported ResourcesExported Resources
TerraformTerraform ● Open Source Automation ToolOpen Source Automation Tool ● ““cloud” orientedcloud” oriented ● Cloud are API’sCloud are API’s ● API’s orientedAPI’s oriented Terraform is an open source automation toolTerraform is an open source automation tool which can deal with any kind of CRUD api’s –which can deal with any kind of CRUD api’s – including major cloud providersincluding major cloud providers
The Terraform modelThe Terraform model ● You model your infrastructureYou model your infrastructure ● You make a planYou make a plan ● If ok, you apply that planIf ok, you apply that plan ● Current state is saved for future changesCurrent state is saved for future changes
HCLHCL ● Hashicorp Configuration LanguageHashicorp Configuration Language ● Yet another cfgmgmt DSLYet another cfgmgmt DSL ● Desired stateDesired state ● Used by multiple hashicorp tools but also 3rdUsed by multiple hashicorp tools but also 3rd party toolsparty tools
Icinga ProviderIcinga Provider providerprovider "icinga2" {"icinga2" { api_url = "https://icinga.alerting.vagrant:5665/v1"api_url = "https://icinga.alerting.vagrant:5665/v1" api_user = "root"api_user = "root" api_password = "icinga"api_password = "icinga" insecure_skip_tls_verify =insecure_skip_tls_verify = truetrue }}
Icinga HostIcinga Host resourceresource "icinga2_host" "node1" {"icinga2_host" "node1" { hostname = "node1.alerting.vagrant"hostname = "node1.alerting.vagrant" address = "192.168.47.51"address = "192.168.47.51" check_command = "hostalive"check_command = "hostalive" varsvars {{ os = "linux"os = "linux" }} }} resourceresource "icinga2_hostgroup" "linux-nodes" {"icinga2_hostgroup" "linux-nodes" { name = "linux-nodes"name = "linux-nodes" display_name = "All linux nodes"display_name = "All linux nodes" }}
Fast FeedbackFast Feedback resource "aws_instance" "node1" {resource "aws_instance" "node1" { ami = "${var.ami_id}"ami = "${var.ami_id}" instance_type = "${var.ami_size}"instance_type = "${var.ami_size}" key_name = "${var.key_pair_name}"key_name = "${var.key_pair_name}" subnet_id = "${var.private_subnet}"subnet_id = "${var.private_subnet}" vpc_security_group_ids = ["${var.security_group_id}"]vpc_security_group_ids = ["${var.security_group_id}"] }} resourceresource "icinga2_host" "node1" {"icinga2_host" "node1" { hostname = "hostname = "node1"node1" address = "address = "${aws_instance.instance.private_ip}${aws_instance.instance.private_ip}"" check_command = "hostalive"check_command = "hostalive" varsvars {{ os = "linux"os = "linux" }} }}
Check CommandCheck Command resourceresource "icinga2_checkcommand" "apache_status" {"icinga2_checkcommand" "apache_status" { name = "apache_status"name = "apache_status" templates = ["apache-status", "plugin-check-command",templates = ["apache-status", "plugin-check-command", "plugin-check-command", "ipv4-or-ipv6"]"plugin-check-command", "ipv4-or-ipv6"] command =command = "/usr/lib64/nagios/plugins/check_apache_status.pl""/usr/lib64/nagios/plugins/check_apache_status.pl" arguments = {arguments = { "-H" = "$apache_status_address$""-H" = "$apache_status_address$" "-c" = "$apache_status_critical$""-c" = "$apache_status_critical$" "-p" = "$apache_status_port$""-p" = "$apache_status_port$" }} }} resourceresource "icinga2_service" "my-service" {"icinga2_service" "my-service" { name = "ssh"name = "ssh" hostname = "c1-mysql-1"hostname = "c1-mysql-1" check_command = "ssh"check_command = "ssh" }}
NotificationsNotifications resource "icinga2_user" "user" {resource "icinga2_user" "user" { name = "terraform"name = "terraform" email = "terraform@dev.null"email = "terraform@dev.null" }} resourceresource "icinga2_notification" "host-notification" {"icinga2_notification" "host-notification" { hostname = "docker-icinga2"hostname = "docker-icinga2" command = "mail-host-notification"command = "mail-host-notification" users = ["user"]users = ["user"] }} resourceresource "icinga2_notification" "ping-service-notification" {"icinga2_notification" "ping-service-notification" { hostname = "docker-icinga2"hostname = "docker-icinga2" command = "mail-service-notification"command = "mail-service-notification" users = ["user"]users = ["user"] servicename = "ping"servicename = "ping" }}
Other ResourcesOther Resources resourceresource "icinga2_checkcommand" "apache_status" {"icinga2_checkcommand" "apache_status" { name = "apache_status"name = "apache_status" templates = ["apache-status", "plugin-check-command",templates = ["apache-status", "plugin-check-command", "plugin-check-command", "ipv4-or-ipv6"]"plugin-check-command", "ipv4-or-ipv6"] command =command = "/usr/lib64/nagios/plugins/check_apache_status.pl""/usr/lib64/nagios/plugins/check_apache_status.pl" arguments = {arguments = { "-H" = "$apache_status_address$""-H" = "$apache_status_address$" "-c" = "$apache_status_critical$""-c" = "$apache_status_critical$" "-p" = "$apache_status_port$""-p" = "$apache_status_port$" }} }}
DEMO TIMEDEMO TIME
ConsulConsul ● Open Source Service Discovery ToolOpen Source Service Discovery Tool • dig @127.0.0.1 -p 8600 puppetmaster.service.consul ANYdig @127.0.0.1 -p 8600 puppetmaster.service.consul ANY ● Build-in KV storeBuild-in KV store ● Service Mesh toolService Mesh tool
ConsulConsul ::consul::service { 'puppetmaster':::consul::service { 'puppetmaster': port => 8140,port => 8140, }} ::consul::check { 'puppetmaster_tcp':::consul::check { 'puppetmaster_tcp': interval => '60s',interval => '60s', tcp => 'localhost:8140',tcp => 'localhost:8140', notes => 'Puppetmasters listen on port 8140',notes => 'Puppetmasters listen on port 8140', service_id => 'puppetmaster',service_id => 'puppetmaster', }}
Consul~Icinga Exit CodesConsul~Icinga Exit Codes ::consul::service { 'pgsql':::consul::service { 'pgsql': checks => [checks => [ {{ script => '/usr/lib64/nagios/plugins/check_pgsql',script => '/usr/lib64/nagios/plugins/check_pgsql', interval => '10s'interval => '10s' }} ],], port => 5432,port => 5432, }}
New Import SourceNew Import Source
Module.infoModule.info Name: ConsulName: Consul Version: 1.0.0Version: 1.0.0 Depends: directorDepends: director Description: Consul module for Icinga Web 2Description: Consul module for Icinga Web 2 This module provides a Consul import source for Icinga DirectorThis module provides a Consul import source for Icinga Director
run.phprun.php <?php<?php useuse IcingaApplicationIcinga;IcingaApplicationIcinga; $this->provideHook('director/ImportSource');$this->provideHook('director/ImportSource');
library/Consul/ProvidedHook/Director/ImportSource.phplibrary/Consul/ProvidedHook/Director/ImportSource.php <?php<?php namespacenamespace IcingaModuleConsulProvidedHookDirector;IcingaModuleConsulProvidedHookDirector; useuse IcingaModuleDirectorHookImportSourceHook;IcingaModuleDirectorHookImportSourceHook; useuse IcingaModuleDirectorWebFormQuickForm;IcingaModuleDirectorWebFormQuickForm; classclass ImportSourceImportSource extendsextends ImportSourceHookImportSourceHook {{ public functionpublic function getName()getName() {{ returnreturn 'HashiCorp Consul';'HashiCorp Consul'; }} public functionpublic function fetchData() {}fetchData() {} public functionpublic function listColumns() {}listColumns() {} public static functionpublic static function getDefaultKeyColumnName() {}getDefaultKeyColumnName() {} }}
Little bit of ConfigLittle bit of Config public static function getDefaultKeyColumnName()public static function getDefaultKeyColumnName() {{ return 'Node';return 'Node'; }} public static function addSettingsFormFields(QuickForm $form)public static function addSettingsFormFields(QuickForm $form) {{ $form->addElement('text', 'consul_url', array($form->addElement('text', 'consul_url', array( 'label' => 'HTTP API URL','label' => 'HTTP API URL', 'required' => true,'required' => true, 'value' => 'http://127.0.0.1:8500','value' => 'http://127.0.0.1:8500', ));)); return;return; }}
We are not cave men!We are not cave men! composer require sensiolabs/consul-php-sdkcomposer require sensiolabs/consul-php-sdk <?php<?php useuse IcingaApplicationIcinga;IcingaApplicationIcinga; require_oncerequire_once __DIR____DIR__ . '/vendor/autoload.php';. '/vendor/autoload.php'; $this->provideHook('director/ImportSource');$this->provideHook('director/ImportSource');
Query ConsulQuery Consul use SensioLabsConsulServiceFactory;use SensioLabsConsulServiceFactory; public functionpublic function fetchData()fetchData() {{ $sf = new ServiceFactory($sf = new ServiceFactory( arrayarray('base_uri' => $this->getSetting('consul_url'))('base_uri' => $this->getSetting('consul_url')) );); $agent = $sf->get('catalog');$agent = $sf->get('catalog'); return json_decode($agent->nodes()->getBody());return json_decode($agent->nodes()->getBody()); }} public functionpublic function listColumns()listColumns() {{ return array_keys((array) current($this->fetchData()));return array_keys((array) current($this->fetchData())); }}
Adding a sync sourceAdding a sync source
Adding a sync ruleAdding a sync rule
Add sync propertiesAdd sync properties
Plumbing it togetherPlumbing it together
Manual is for AnimalsManual is for Animals [Unit][Unit] Description=Director Job runnerDescription=Director Job runner [Service][Service] Type=simpleType=simple ExecStart=/usr/bin/icingacli director jobs run foreverExecStart=/usr/bin/icingacli director jobs run forever Restart=on-successRestart=on-success
Fast FeedbackFast Feedback ::consul::watch { 'director_import':::consul::watch { 'director_import': type => 'service',type => 'service', handler => '/usr/local/bin/director_sync_deploy.sh',handler => '/usr/local/bin/director_sync_deploy.sh', service => 'node_exporter',service => 'node_exporter', passingonly =>passingonly => truetrue,, require => File['Director Sync and Deploy'],require => File['Director Sync and Deploy'], }}
director_sync_deploy.shdirector_sync_deploy.sh #!/usr/bin/env bash#!/usr/bin/env bash setset -x-x icingacliicingacli director importsource run --id 1director importsource run --id 1 icingacliicingacli director syncrule run --id 1director syncrule run --id 1 icingacliicingacli director config deploydirector config deploy
DEMO TIMEDEMO TIME
Github.comGithub.com attachmentgenie/icingaweb2-module-consul.gitattachmentgenie/icingaweb2-module-consul.git attachmentgenie/vagrant-alerting.gitattachmentgenie/vagrant-alerting.git
VaultVault ● Open Source tool to do secrets managementOpen Source tool to do secrets management ● Secure, store and tightly control access toSecure, store and tightly control access to tokens, passwords, certificates, encryption keystokens, passwords, certificates, encryption keys for protecting secrets and other sensitive datafor protecting secrets and other sensitive data using a UI, CLI, or HTTP API.using a UI, CLI, or HTTP API. ● Certificate managementCertificate management ● Password rotatationPassword rotatation
Pesky PasswordsPesky Passwords $ vault unseal$ vault unseal $ vault write kv/my-secret value="s3c(eT"$ vault write kv/my-secret value="s3c(eT" $ vault read kv/mysecret$ vault read kv/mysecret Key ValueKey Value --- -------- ----- refresh_interval 768hrefresh_interval 768h mysecret s3c(eTmysecret s3c(eT https://forge.puppet.com/jsok/vaulthttps://forge.puppet.com/jsok/vault
NomadNomad ● Open Source tool to do dynamic workloadOpen Source tool to do dynamic workload schedulingscheduling ● Batch, containerized, and non-containerizedBatch, containerized, and non-containerized applications.applications. ● Has native Consul and Vault integrations.Has native Consul and Vault integrations.
Monitoring Nomad with Prometheus andMonitoring Nomad with Prometheus and IcingaIcinga OSMC, Nuremberg, 6th nov 2019OSMC, Nuremberg, 6th nov 2019
ContactContact Bram VogelaarBram Vogelaar +31 6 46 62 60 78+31 6 46 62 60 78 bram.vogelaar@inuits.eubram.vogelaar@inuits.eu @attachmentgenie@attachmentgenie Github.com/attachmentgenieGithub.com/attachmentgenie Inuits BEInuits BE Essensteenweg 31Essensteenweg 31 2930 Brasschaat2930 Brasschaat BelgiumBelgium Inuits NLInuits NL Maashaven Zuidzijde 2Maashaven Zuidzijde 2 3081 AE Rotterdam3081 AE Rotterdam NetherlandsNetherlands

More Related Content

PDF
Ansible2.9 ネットワーク対応のアップデート #ansiblejp
akira6592
 
PDF
Linux directory structure by jitu mistry
JITU MISTRY
 
PPTX
BigtopでHadoopをビルドする（Open Source Conference 2021 Online/Spring 発表資料）
NTT DATA Technology & Innovation
 
PDF
Tomcatx performance-tuning
Vladimir Khokhryakov
 
PDF
Db2 v11.5.4 高可用性構成 & HADR 構成パターンご紹介
IBM Analytics Japan
 
PDF
RFC5277(NETCONF Event Notifications)の勉強資料
Tetsuya Hasegawa
 
PPTX
Haute Disponibilité et Tolérance de Panne
Elior Boukhobza
 
PDF
PostgreSQL High Availability in a Containerized World
Jignesh Shah
 
Ansible2.9 ネットワーク対応のアップデート #ansiblejp
akira6592
 
Linux directory structure by jitu mistry
JITU MISTRY
 
BigtopでHadoopをビルドする（Open Source Conference 2021 Online/Spring 発表資料）
NTT DATA Technology & Innovation
 
Tomcatx performance-tuning
Vladimir Khokhryakov
 
Db2 v11.5.4 高可用性構成 & HADR 構成パターンご紹介
IBM Analytics Japan
 
RFC5277(NETCONF Event Notifications)の勉強資料
Tetsuya Hasegawa
 
Haute Disponibilité et Tolérance de Panne
Elior Boukhobza
 
PostgreSQL High Availability in a Containerized World
Jignesh Shah
 

What's hot (20)

PPT
Iptables in linux
Mandeep Singh
 
PPT
OpenDaylight Integration with OpenStack Neutron: A Tutorial
mestery
 
PDF
Understanding Open vSwitch
YongKi Kim
 
PDF
ブロケード FC ファブリックスイッチ オペレーション講座（後編）
Brocade
 
PPT
virtio勉強会 #1 「virtioの基本的なところ(DRAFT版)」
Naoya Kaneko
 
PDF
インターネットの仕組みとISPの構造
Taiji Tsuchiya
 
PDF
ネットワークOS野郎 ~ インフラ野郎Night 20160414
Kentaro Ebisawa
 
PPS
DSS ITSEC Webinars 2013 - Network Access Control + Mobile Security (Forescout)
Andris Soroka
 
PDF
2022のShowNetに向けて_ShowNet2021_conf_mini_5_2022_stm
Interop Tokyo ShowNet NOC Team
 
PDF
1000 Ccna Questions And Answers
CCNAResources
 
PPTX
Process Monitor の使い方
彰 村地
 
PDF
JavaScript for Hackers.pdf
nafees40
 
PDF
Junos SpaceによるJunos機器の運用管理
Juniper Networks (日本)
 
PDF
Squid, SquidGuard, and Lightsquid on pfSense 2.3 & 2.4 - pfSense Hangout Janu...
Netgate
 
PDF
Wakamonog6 “ISPのネットワーク”って どんなネットワーク？
Satoshi Matsumoto
 
PDF
ONOS SDN-IP: Tutorial and Use Case for SDX
APNIC
 
PDF
Ironic
Haomeng Wang
 
PDF
Provisioning Bare Metal with OpenStack
Devananda Van Der Veen
 
PDF
Apache Bigtop3.2 (仮)（Open Source Conference 2022 Online/Hiroshima 発表資料）
NTT DATA Technology & Innovation
 
PPTX
Understanding the Value and Architecture of Apache Drill
DataWorks Summit
 
Iptables in linux
Mandeep Singh
 
OpenDaylight Integration with OpenStack Neutron: A Tutorial
mestery
 
Understanding Open vSwitch
YongKi Kim
 
ブロケード FC ファブリックスイッチ オペレーション講座（後編）
Brocade
 
virtio勉強会 #1 「virtioの基本的なところ(DRAFT版)」
Naoya Kaneko
 
インターネットの仕組みとISPの構造
Taiji Tsuchiya
 
ネットワークOS野郎 ~ インフラ野郎Night 20160414
Kentaro Ebisawa
 
DSS ITSEC Webinars 2013 - Network Access Control + Mobile Security (Forescout)
Andris Soroka
 
2022のShowNetに向けて_ShowNet2021_conf_mini_5_2022_stm
Interop Tokyo ShowNet NOC Team
 
1000 Ccna Questions And Answers
CCNAResources
 
Process Monitor の使い方
彰 村地
 
JavaScript for Hackers.pdf
nafees40
 
Junos SpaceによるJunos機器の運用管理
Juniper Networks (日本)
 
Squid, SquidGuard, and Lightsquid on pfSense 2.3 & 2.4 - pfSense Hangout Janu...
Netgate
 
Wakamonog6 “ISPのネットワーク”って どんなネットワーク？
Satoshi Matsumoto
 
ONOS SDN-IP: Tutorial and Use Case for SDX
APNIC
 
Ironic
Haomeng Wang
 
Provisioning Bare Metal with OpenStack
Devananda Van Der Veen
 
Apache Bigtop3.2 (仮)（Open Source Conference 2022 Online/Hiroshima 発表資料）
NTT DATA Technology & Innovation
 
Understanding the Value and Architecture of Apache Drill
DataWorks Summit
 
Ad

Similar to Integrating icinga2 and the HashiCorp suite (20)

ODP
Integrating icinga2 and the HashiCorp suite
Bram Vogelaar
 
ODP
Puppet and the HashiCorp Suite
Bram Vogelaar
 
PPTX
Streamline Hadoop DevOps with Apache Ambari
Alejandro Fernandez
 
ODP
Bootstrap your Cloud Infrastructure using puppet and hashicorp stack
Bram Vogelaar
 
PDF
Burn down the silos! Helping dev and ops gel on high availability websites
Lindsay Holmwood
 
PDF
Dev ninja -> vagrant + virtualbox + chef-solo + git + ec2
Yros
 
PDF
Why favour Icinga over Nagios @ FrOSCon 2015
Icinga
 
PDF
NetDevOps Developer Environments with Vagrant @ SCALE16x
Hank Preston
 
PDF
Null Bachaav - May 07 Attack Monitoring workshop.
Prajal Kulkarni
 
PDF
Practical Chef and Capistrano for Your Rails App
SmartLogic
 
PDF
Icinga2 Hacking Session 2014-10-10
Icinga
 
PDF
Infrastructure-as-code: bridging the gap between Devs and Ops
Mykyta Protsenko
 
PDF
Postgres the hardway
Dave Pitts
 
PDF
Automating complex infrastructures with Puppet
Kris Buytaert
 
PDF
Vagrant for real
Codemotion
 
PDF
Vagrant for real (codemotion rome 2016)
Michele Orselli
 
PDF
EC2 AMI Factory with Chef, Berkshelf, and Packer
George Miranda
 
PDF
infra-as-code
Itamar Hassin
 
PDF
ELK: a log management framework
Giovanni Bechis
 
PDF
PuppetDB: A Single Source for Storing Your Puppet Data - PUG NY
Puppet
 
Integrating icinga2 and the HashiCorp suite
Bram Vogelaar
 
Puppet and the HashiCorp Suite
Bram Vogelaar
 
Streamline Hadoop DevOps with Apache Ambari
Alejandro Fernandez
 
Bootstrap your Cloud Infrastructure using puppet and hashicorp stack
Bram Vogelaar
 
Burn down the silos! Helping dev and ops gel on high availability websites
Lindsay Holmwood
 
Dev ninja -> vagrant + virtualbox + chef-solo + git + ec2
Yros
 
Why favour Icinga over Nagios @ FrOSCon 2015
Icinga
 
NetDevOps Developer Environments with Vagrant @ SCALE16x
Hank Preston
 
Null Bachaav - May 07 Attack Monitoring workshop.
Prajal Kulkarni
 
Practical Chef and Capistrano for Your Rails App
SmartLogic
 
Icinga2 Hacking Session 2014-10-10
Icinga
 
Infrastructure-as-code: bridging the gap between Devs and Ops
Mykyta Protsenko
 
Postgres the hardway
Dave Pitts
 
Automating complex infrastructures with Puppet
Kris Buytaert
 
Vagrant for real
Codemotion
 
Vagrant for real (codemotion rome 2016)
Michele Orselli
 
EC2 AMI Factory with Chef, Berkshelf, and Packer
George Miranda
 
infra-as-code
Itamar Hassin
 
ELK: a log management framework
Giovanni Bechis
 
PuppetDB: A Single Source for Storing Your Puppet Data - PUG NY
Puppet
 
Ad

More from Bram Vogelaar (20)

PPTX
Terraforming your Platform Engineering organisation.pptx
Bram Vogelaar
 
PDF
Secure second days operations with Boundary and Vault.pdf
Bram Vogelaar
 
PDF
Cost reconciliation in a post CMDB world
Bram Vogelaar
 
PDF
Self scaling Multi cloud nomad workloads
Bram Vogelaar
 
PDF
Scraping metrics for fun and profit
Bram Vogelaar
 
PDF
10 things i learned building nomad-packs
Bram Vogelaar
 
PDF
10 things I learned building Nomad packs
Bram Vogelaar
 
PDF
Easy Cloud Native Transformation with Nomad
Bram Vogelaar
 
PDF
Uncomplicated Nomad
Bram Vogelaar
 
PDF
Observability; a gentle introduction
Bram Vogelaar
 
PDF
Running Trusted Payload with Nomad and Waypoint
Bram Vogelaar
 
PDF
Easy Cloud Native Transformation using HashiCorp Nomad
Bram Vogelaar
 
PDF
Securing Prometheus exporters using HashiCorp Vault
Bram Vogelaar
 
PDF
CICD using jenkins and Nomad
Bram Vogelaar
 
PDF
Bootstrapping multidc observability stack
Bram Vogelaar
 
PDF
Running trusted payloads with Nomad and Waypoint
Bram Vogelaar
 
PDF
Gamification of Chaos Testing
Bram Vogelaar
 
PDF
Puppet and the HashiStack
Bram Vogelaar
 
PDF
Bootstrapping multidc observability stack
Bram Vogelaar
 
PPTX
Creating Reusable Puppet Profiles
Bram Vogelaar
 
Terraforming your Platform Engineering organisation.pptx
Bram Vogelaar
 
Secure second days operations with Boundary and Vault.pdf
Bram Vogelaar
 
Cost reconciliation in a post CMDB world
Bram Vogelaar
 
Self scaling Multi cloud nomad workloads
Bram Vogelaar
 
Scraping metrics for fun and profit
Bram Vogelaar
 
10 things i learned building nomad-packs
Bram Vogelaar
 
10 things I learned building Nomad packs
Bram Vogelaar
 
Easy Cloud Native Transformation with Nomad
Bram Vogelaar
 
Uncomplicated Nomad
Bram Vogelaar
 
Observability; a gentle introduction
Bram Vogelaar
 
Running Trusted Payload with Nomad and Waypoint
Bram Vogelaar
 
Easy Cloud Native Transformation using HashiCorp Nomad
Bram Vogelaar
 
Securing Prometheus exporters using HashiCorp Vault
Bram Vogelaar
 
CICD using jenkins and Nomad
Bram Vogelaar
 
Bootstrapping multidc observability stack
Bram Vogelaar
 
Running trusted payloads with Nomad and Waypoint
Bram Vogelaar
 
Gamification of Chaos Testing
Bram Vogelaar
 
Puppet and the HashiStack
Bram Vogelaar
 
Bootstrapping multidc observability stack
Bram Vogelaar
 
Creating Reusable Puppet Profiles
Bram Vogelaar
 

Recently uploaded (20)

PDF
cuic standard and advanced reporting.pdf
SimoneTitaniumTomase
 
PDF
How UI/UX Design Impacts User Retention in Mobile Apps.pdf
SynapseIndia
 
PDF
The Rise and Fall of 3GPP – Time for a Sabbatical?
3G4G
 
PDF
Peak of Data & AI Encore- AI for Metadata and Smarter Workflows
Safe Software
 
PDF
Advanced methodologies resolving dimensionality complications for autism neur...
IAESIJAI
 
PPTX
PA Analog/Digital System: The Backbone of Modern Surveillance and Communication
AVTRON Technologies LLC
 
PDF
Build a system with the filesystem maintained by OSTree @ COSCUP 2025
Jian-Hong Pan
 
PPTX
KOM of Painting work and Equipment Insulation REV00 update 25-dec.pptx
muhammadmuneebnaeem7
 
PPTX
Cloud computing and distributed systems.
kkkkkhan
 
PDF
Mobile App Security Testing_ A Comprehensive Guide.pdf
flufftailshop
 
PDF
NewMind AI Weekly Chronicles - August'25 Week I
NewMind AI
 
PDF
KodekX | Application Modernization Development
KodekX
 
DOCX
The AUB Centre for AI in Media Proposal.docx
GAONE9
 
PDF
7 ChatGPT Prompts to Help You Define Your Ideal Customer Profile.pdf
SOFTTECHHUB
 
PDF
Diabetes mellitus diagnosis method based random forest with bat algorithm
IAESIJAI
 
PDF
Electronic commerce courselecture one. Pdf
OmerMohamed64
 
PDF
NewMind AI Monthly Chronicles - July 2025
NewMind AI
 
PDF
Architecting across the Boundaries of two Complex Domains - Healthcare & Tech...
Peter Tan
 
PDF
Review of recent advances in non-invasive hemoglobin estimation
IAESIJAI
 
PPTX
20250228 LYD VKU AI Blended-Learning.pptx
DatVoNgoc
 
cuic standard and advanced reporting.pdf
SimoneTitaniumTomase
 
How UI/UX Design Impacts User Retention in Mobile Apps.pdf
SynapseIndia
 
The Rise and Fall of 3GPP – Time for a Sabbatical?
3G4G
 
Peak of Data & AI Encore- AI for Metadata and Smarter Workflows
Safe Software
 
Advanced methodologies resolving dimensionality complications for autism neur...
IAESIJAI
 
PA Analog/Digital System: The Backbone of Modern Surveillance and Communication
AVTRON Technologies LLC
 
Build a system with the filesystem maintained by OSTree @ COSCUP 2025
Jian-Hong Pan
 
KOM of Painting work and Equipment Insulation REV00 update 25-dec.pptx
muhammadmuneebnaeem7
 
Cloud computing and distributed systems.
kkkkkhan
 
Mobile App Security Testing_ A Comprehensive Guide.pdf
flufftailshop
 
NewMind AI Weekly Chronicles - August'25 Week I
NewMind AI
 
KodekX | Application Modernization Development
KodekX
 
The AUB Centre for AI in Media Proposal.docx
GAONE9
 
7 ChatGPT Prompts to Help You Define Your Ideal Customer Profile.pdf
SOFTTECHHUB
 
Diabetes mellitus diagnosis method based random forest with bat algorithm
IAESIJAI
 
Electronic commerce courselecture one. Pdf
OmerMohamed64
 
NewMind AI Monthly Chronicles - July 2025
NewMind AI
 
Architecting across the Boundaries of two Complex Domains - Healthcare & Tech...
Peter Tan
 
Review of recent advances in non-invasive hemoglobin estimation
IAESIJAI
 
20250228 LYD VKU AI Blended-Learning.pptx
DatVoNgoc
 

Integrating icinga2 and the HashiCorp suite

  • 1. Integrating icinga2 and the HashiCorp suite Bram Vogelaar
  • 2. ~$ whoami~$ whoami ● I used to be a Molecular Biologist,I used to be a Molecular Biologist, ● Then became a Dev,Then became a Dev, ● Now an Ops.Now an Ops. ● Open Source Consultant @Open Source Consultant @inuits.euinuits.eu
  • 4. PackerPacker ● Open Source tool to make OS imagesOpen Source tool to make OS images ● Supports Cloud Providers, Docker, Vbox, …Supports Cloud Providers, Docker, Vbox, … (builders)(builders) ● Has hooks to provision the base imagesHas hooks to provision the base images (provisioners)(provisioners) ● Create artifacts (Post-Processors)Create artifacts (Post-Processors)
  • 5. PackerPacker {{ "builders": ["builders": [ {{ "type": "azure-arm","type": "azure-arm", ...... }} ],], "provisioners": ["provisioners": [ {{ "scripts": ["scripts": [ "scripts/common/consul_install.sh","scripts/common/consul_install.sh", "scripts/common/puppet_install.sh","scripts/common/puppet_install.sh", "scripts/common/puppet_icinga.sh""scripts/common/puppet_icinga.sh" ],], "type": "shell","type": "shell", }} ]] }} $ packer build template.json$ packer build template.json
  • 6. VagrantVagrant ● Open Source tool to bootstrap vmsOpen Source tool to bootstrap vms ● Supports many vm Providers, Docker, Vbox, …Supports many vm Providers, Docker, Vbox, … ● Has hooks to provision the base imagesHas hooks to provision the base images (provisioners), Puppet, Chef, Ansible(provisioners), Puppet, Chef, Ansible
  • 7. VagrantfileVagrantfile # -*- mode: ruby -*-# -*- mode: ruby -*- # vi: set ft=ruby :# vi: set ft=ruby : Vagrant.configure("2") do |config|Vagrant.configure("2") do |config| config.vm.box = "base"config.vm.box = "base" # config.vm.box_check_update = false# config.vm.box_check_update = false # config.vm.network "forwarded_port", guest: 80, host: 8080# config.vm.network "forwarded_port", guest: 80, host: 8080 # config.vm.network "private_network", ip: "192.168.33.10"# config.vm.network "private_network", ip: "192.168.33.10" # config.vm.network "public_network"# config.vm.network "public_network" # config.vm.synced_folder "../data", "/vagrant_data"# config.vm.synced_folder "../data", "/vagrant_data" # config.vm.provider "virtualbox" do |vb|# config.vm.provider "virtualbox" do |vb| # vb.gui = true# vb.gui = true # vb.memory = "1024"# vb.memory = "1024" # end# end # config.vm.provision "shell", inline: <<-SHELL# config.vm.provision "shell", inline: <<-SHELL # apt-get update# apt-get update # apt-get install -y apache2# apt-get install -y apache2 # SHELL# SHELL endend
  • 8. Try Icinga Yourself!Try Icinga Yourself! https://github.com/Icinga/icinga-vagranthttps://github.com/Icinga/icinga-vagrant
  • 9. By Icinga, for IcingaBy Icinga, for Icinga ● StandaloneStandalone ● DistributedDistributed ● InfluxDBInfluxDB ● ElasticElastic ● GraylogGraylog
  • 10. Docs for all!Docs for all!
  • 13. TerraformTerraform ● Open Source Automation ToolOpen Source Automation Tool ● ““cloud” orientedcloud” oriented ● Cloud are API’sCloud are API’s ● API’s orientedAPI’s oriented Terraform is an open source automation toolTerraform is an open source automation tool which can deal with any kind of CRUD api’s –which can deal with any kind of CRUD api’s – including major cloud providersincluding major cloud providers
  • 14. The Terraform modelThe Terraform model ● You model your infrastructureYou model your infrastructure ● You make a planYou make a plan ● If ok, you apply that planIf ok, you apply that plan ● Current state is saved for future changesCurrent state is saved for future changes
  • 15. HCLHCL ● Hashicorp Configuration LanguageHashicorp Configuration Language ● Yet another cfgmgmt DSLYet another cfgmgmt DSL ● Desired stateDesired state ● Used by multiple hashicorp tools but also 3rdUsed by multiple hashicorp tools but also 3rd party toolsparty tools
  • 16. Icinga ProviderIcinga Provider providerprovider "icinga2" {"icinga2" { api_url = "https://icinga.alerting.vagrant:5665/v1"api_url = "https://icinga.alerting.vagrant:5665/v1" api_user = "root"api_user = "root" api_password = "icinga"api_password = "icinga" insecure_skip_tls_verify =insecure_skip_tls_verify = truetrue }}
  • 17. Icinga HostIcinga Host resourceresource "icinga2_host" "node1" {"icinga2_host" "node1" { hostname = "node1.alerting.vagrant"hostname = "node1.alerting.vagrant" address = "192.168.47.51"address = "192.168.47.51" check_command = "hostalive"check_command = "hostalive" varsvars {{ os = "linux"os = "linux" }} }} resourceresource "icinga2_hostgroup" "linux-nodes" {"icinga2_hostgroup" "linux-nodes" { name = "linux-nodes"name = "linux-nodes" display_name = "All linux nodes"display_name = "All linux nodes" }}
  • 18. Fast FeedbackFast Feedback resource "aws_instance" "node1" {resource "aws_instance" "node1" { ami = "${var.ami_id}"ami = "${var.ami_id}" instance_type = "${var.ami_size}"instance_type = "${var.ami_size}" key_name = "${var.key_pair_name}"key_name = "${var.key_pair_name}" subnet_id = "${var.private_subnet}"subnet_id = "${var.private_subnet}" vpc_security_group_ids = ["${var.security_group_id}"]vpc_security_group_ids = ["${var.security_group_id}"] }} resourceresource "icinga2_host" "node1" {"icinga2_host" "node1" { hostname = "hostname = "node1"node1" address = "address = "${aws_instance.instance.private_ip}${aws_instance.instance.private_ip}"" check_command = "hostalive"check_command = "hostalive" varsvars {{ os = "linux"os = "linux" }} }}
  • 19. Check CommandCheck Command resourceresource "icinga2_checkcommand" "apache_status" {"icinga2_checkcommand" "apache_status" { name = "apache_status"name = "apache_status" templates = ["apache-status", "plugin-check-command",templates = ["apache-status", "plugin-check-command", "plugin-check-command", "ipv4-or-ipv6"]"plugin-check-command", "ipv4-or-ipv6"] command =command = "/usr/lib64/nagios/plugins/check_apache_status.pl""/usr/lib64/nagios/plugins/check_apache_status.pl" arguments = {arguments = { "-H" = "$apache_status_address$""-H" = "$apache_status_address$" "-c" = "$apache_status_critical$""-c" = "$apache_status_critical$" "-p" = "$apache_status_port$""-p" = "$apache_status_port$" }} }} resourceresource "icinga2_service" "my-service" {"icinga2_service" "my-service" { name = "ssh"name = "ssh" hostname = "c1-mysql-1"hostname = "c1-mysql-1" check_command = "ssh"check_command = "ssh" }}
  • 20. NotificationsNotifications resource "icinga2_user" "user" {resource "icinga2_user" "user" { name = "terraform"name = "terraform" email = "terraform@dev.null"email = "terraform@dev.null" }} resourceresource "icinga2_notification" "host-notification" {"icinga2_notification" "host-notification" { hostname = "docker-icinga2"hostname = "docker-icinga2" command = "mail-host-notification"command = "mail-host-notification" users = ["user"]users = ["user"] }} resourceresource "icinga2_notification" "ping-service-notification" {"icinga2_notification" "ping-service-notification" { hostname = "docker-icinga2"hostname = "docker-icinga2" command = "mail-service-notification"command = "mail-service-notification" users = ["user"]users = ["user"] servicename = "ping"servicename = "ping" }}
  • 21. Other ResourcesOther Resources resourceresource "icinga2_checkcommand" "apache_status" {"icinga2_checkcommand" "apache_status" { name = "apache_status"name = "apache_status" templates = ["apache-status", "plugin-check-command",templates = ["apache-status", "plugin-check-command", "plugin-check-command", "ipv4-or-ipv6"]"plugin-check-command", "ipv4-or-ipv6"] command =command = "/usr/lib64/nagios/plugins/check_apache_status.pl""/usr/lib64/nagios/plugins/check_apache_status.pl" arguments = {arguments = { "-H" = "$apache_status_address$""-H" = "$apache_status_address$" "-c" = "$apache_status_critical$""-c" = "$apache_status_critical$" "-p" = "$apache_status_port$""-p" = "$apache_status_port$" }} }}
  • 23. ConsulConsul ● Open Source Service Discovery ToolOpen Source Service Discovery Tool • dig @127.0.0.1 -p 8600 puppetmaster.service.consul ANYdig @127.0.0.1 -p 8600 puppetmaster.service.consul ANY ● Build-in KV storeBuild-in KV store ● Service Mesh toolService Mesh tool
  • 24. ConsulConsul ::consul::service { 'puppetmaster':::consul::service { 'puppetmaster': port => 8140,port => 8140, }} ::consul::check { 'puppetmaster_tcp':::consul::check { 'puppetmaster_tcp': interval => '60s',interval => '60s', tcp => 'localhost:8140',tcp => 'localhost:8140', notes => 'Puppetmasters listen on port 8140',notes => 'Puppetmasters listen on port 8140', service_id => 'puppetmaster',service_id => 'puppetmaster', }}
  • 25. Consul~Icinga Exit CodesConsul~Icinga Exit Codes ::consul::service { 'pgsql':::consul::service { 'pgsql': checks => [checks => [ {{ script => '/usr/lib64/nagios/plugins/check_pgsql',script => '/usr/lib64/nagios/plugins/check_pgsql', interval => '10s'interval => '10s' }} ],], port => 5432,port => 5432, }}
  • 26. New Import SourceNew Import Source
  • 27. Module.infoModule.info Name: ConsulName: Consul Version: 1.0.0Version: 1.0.0 Depends: directorDepends: director Description: Consul module for Icinga Web 2Description: Consul module for Icinga Web 2 This module provides a Consul import source for Icinga DirectorThis module provides a Consul import source for Icinga Director
  • 29. library/Consul/ProvidedHook/Director/ImportSource.phplibrary/Consul/ProvidedHook/Director/ImportSource.php <?php<?php namespacenamespace IcingaModuleConsulProvidedHookDirector;IcingaModuleConsulProvidedHookDirector; useuse IcingaModuleDirectorHookImportSourceHook;IcingaModuleDirectorHookImportSourceHook; useuse IcingaModuleDirectorWebFormQuickForm;IcingaModuleDirectorWebFormQuickForm; classclass ImportSourceImportSource extendsextends ImportSourceHookImportSourceHook {{ public functionpublic function getName()getName() {{ returnreturn 'HashiCorp Consul';'HashiCorp Consul'; }} public functionpublic function fetchData() {}fetchData() {} public functionpublic function listColumns() {}listColumns() {} public static functionpublic static function getDefaultKeyColumnName() {}getDefaultKeyColumnName() {} }}
  • 30. Little bit of ConfigLittle bit of Config public static function getDefaultKeyColumnName()public static function getDefaultKeyColumnName() {{ return 'Node';return 'Node'; }} public static function addSettingsFormFields(QuickForm $form)public static function addSettingsFormFields(QuickForm $form) {{ $form->addElement('text', 'consul_url', array($form->addElement('text', 'consul_url', array( 'label' => 'HTTP API URL','label' => 'HTTP API URL', 'required' => true,'required' => true, 'value' => 'http://127.0.0.1:8500','value' => 'http://127.0.0.1:8500', ));)); return;return; }}
  • 31. We are not cave men!We are not cave men! composer require sensiolabs/consul-php-sdkcomposer require sensiolabs/consul-php-sdk <?php<?php useuse IcingaApplicationIcinga;IcingaApplicationIcinga; require_oncerequire_once __DIR____DIR__ . '/vendor/autoload.php';. '/vendor/autoload.php'; $this->provideHook('director/ImportSource');$this->provideHook('director/ImportSource');
  • 32. Query ConsulQuery Consul use SensioLabsConsulServiceFactory;use SensioLabsConsulServiceFactory; public functionpublic function fetchData()fetchData() {{ $sf = new ServiceFactory($sf = new ServiceFactory( arrayarray('base_uri' => $this->getSetting('consul_url'))('base_uri' => $this->getSetting('consul_url')) );); $agent = $sf->get('catalog');$agent = $sf->get('catalog'); return json_decode($agent->nodes()->getBody());return json_decode($agent->nodes()->getBody()); }} public functionpublic function listColumns()listColumns() {{ return array_keys((array) current($this->fetchData()));return array_keys((array) current($this->fetchData())); }}
  • 33. Adding a sync sourceAdding a sync source
  • 34. Adding a sync ruleAdding a sync rule
  • 35. Add sync propertiesAdd sync properties
  • 37. Manual is for AnimalsManual is for Animals [Unit][Unit] Description=Director Job runnerDescription=Director Job runner [Service][Service] Type=simpleType=simple ExecStart=/usr/bin/icingacli director jobs run foreverExecStart=/usr/bin/icingacli director jobs run forever Restart=on-successRestart=on-success
  • 38. Fast FeedbackFast Feedback ::consul::watch { 'director_import':::consul::watch { 'director_import': type => 'service',type => 'service', handler => '/usr/local/bin/director_sync_deploy.sh',handler => '/usr/local/bin/director_sync_deploy.sh', service => 'node_exporter',service => 'node_exporter', passingonly =>passingonly => truetrue,, require => File['Director Sync and Deploy'],require => File['Director Sync and Deploy'], }}
  • 39. director_sync_deploy.shdirector_sync_deploy.sh #!/usr/bin/env bash#!/usr/bin/env bash setset -x-x icingacliicingacli director importsource run --id 1director importsource run --id 1 icingacliicingacli director syncrule run --id 1director syncrule run --id 1 icingacliicingacli director config deploydirector config deploy
  • 42. VaultVault ● Open Source tool to do secrets managementOpen Source tool to do secrets management ● Secure, store and tightly control access toSecure, store and tightly control access to tokens, passwords, certificates, encryption keystokens, passwords, certificates, encryption keys for protecting secrets and other sensitive datafor protecting secrets and other sensitive data using a UI, CLI, or HTTP API.using a UI, CLI, or HTTP API. ● Certificate managementCertificate management ● Password rotatationPassword rotatation
  • 43. Pesky PasswordsPesky Passwords $ vault unseal$ vault unseal $ vault write kv/my-secret value="s3c(eT"$ vault write kv/my-secret value="s3c(eT" $ vault read kv/mysecret$ vault read kv/mysecret Key ValueKey Value --- -------- ----- refresh_interval 768hrefresh_interval 768h mysecret s3c(eTmysecret s3c(eT https://forge.puppet.com/jsok/vaulthttps://forge.puppet.com/jsok/vault
  • 44. NomadNomad ● Open Source tool to do dynamic workloadOpen Source tool to do dynamic workload schedulingscheduling ● Batch, containerized, and non-containerizedBatch, containerized, and non-containerized applications.applications. ● Has native Consul and Vault integrations.Has native Consul and Vault integrations.
  • 45. Monitoring Nomad with Prometheus andMonitoring Nomad with Prometheus and IcingaIcinga OSMC, Nuremberg, 6th nov 2019OSMC, Nuremberg, 6th nov 2019
  • 46. ContactContact Bram VogelaarBram Vogelaar +31 6 46 62 60 78+31 6 46 62 60 78 bram.vogelaar@inuits.eubram.vogelaar@inuits.eu @attachmentgenie@attachmentgenie Github.com/attachmentgenieGithub.com/attachmentgenie Inuits BEInuits BE Essensteenweg 31Essensteenweg 31 2930 Brasschaat2930 Brasschaat BelgiumBelgium Inuits NLInuits NL Maashaven Zuidzijde 2Maashaven Zuidzijde 2 3081 AE Rotterdam3081 AE Rotterdam NetherlandsNetherlands