SlideShare a Scribd company logo
1
CHEF
INDUSTRIALIZE & AUTOMATE YOUR INFRASTRUCTURE
Created by Michael Lopez - © Maisons du Monde
2
TO DO WHAT ?
Con gure a machine in minutes without action
Deploy software or application
Ensure that all your machines are identical
Gain time !
3 . 1
HOW DOES IT WORKS ?
HOW DOES IT WORKS ?
3 . 24
CHEF VS PUPPET/ANSIBLE
PUPPET
Language : DSL Ruby/Json
Approach : Execution by dependency and
chained action
Stored data : YAML
Agent : Yes
ANSIBLE
Language : Python
Approach : Hierarchical Execution
Stored data : YAML
Agent : No
CHEF
Language : Full Ruby & DSL
Approach : Hierarchical Execution
Stored data : JSON
Agent : Yes
PUPPET
package { 'openssh-server':
ensure => present,
before => File['/etc/ssh/sshd_config'],
}
file { '/etc/ssh/sshd_config':
ensure => file,
mode => 600,
source => 'puppet:///modules/sshd/sshd_config',
require => Package['openssh-server'],
}
service { 'sshd':
ensure => running,
enable => true,
subscribe => File['/etc/ssh/sshd_config'],
}
5 . 15 . 2
PUPPET
Chained Actions
package { 'openssh-server':
ensure => present,
} -> # and then:
file { '/etc/ssh/sshd_config':
ensure => file,
mode => 600,
source => 'puppet:///modules/sshd/sshd_config',
} ~> # and then:
service { 'sshd':
ensure => running,
enable => true,
}
5 . 3
PUPPET
Virtual Resources
@user {'deploy':
uid => 2004,
comment => 'Deployment User',
group => www-data,
groups => ["enterprise"],
tag => [deploy, web],
}
realize User['deploy'], User['zleslie']
User <| tag == web |>
6 . 1
ANSIBLE
Playbook
---
# This role installs HAProxy and configures it.
- name: Download and install haproxy and socat
apt: name={{ item }} state=present
with_items:
- haproxy
- socat
- name: Configure the haproxy cnf file with hosts
template: src=haproxy.cfg.j2 dest=/etc/haproxy/haproxy.cfg
notify: restart haproxy
- name: Start the haproxy service
service: name=haproxy state=started enabled=yes
6 . 2
ANSIBLE
- apt: name=foo update_cache=yes
- apt: name=foo state=absent
- apt: name=foo=1.00 state=present
- apt: deb=/tmp/mypackage.deb
- file: path=/etc/foo.conf owner=foo group=foo mode=0644
- file: src=/tmp/{{ item.path }} dest={{ item.dest }} state=link
with_items:
- { path: 'x', dest: 'y' }
- { path: 'z', dest: 'k' }
- file: path=/etc/foo.conf state=touch mode="u=rw,g=r,o=r"
7
CHEF VS PUPPET/ANSIBLE
Advantages Disadvantages
Chef Fast & powerful
Dev Oriented => Flexible
Full Ruby
Search
Crypted data
Flexibility -> Complexity
Puppet Mature
Large Community
Lot of tools
Slow
Complex Language
Complex execution order
Ansible Agentless
« No code »
Multilingual plugins
Immaturity: Small community and few tools
Data stored in les
8 . 1
CHEF SERVER
Free/Basic Version
Free & no node limit
In your Infrastructure
› No access to "premium" features
8 . 2
CHEF SERVER
8 . 3
CHEF AUTOMATE
Deliver a continuous deployment pipeline for infrastructure and applications.
Gain insight into operational, compliance, and work ow events.
Identify compliance issues, security risks, and outdated software with customizable
reports
9
NOT ONLY CHEF SERVER...
Chef Solo
No Server
Deploy & Run recipes directly on the node
› No search & dedicated attributes on a node
› Need to push cookbooks on each nodes
› Not compatible with version constraint
Chef Zero
Chef Server instance in memory
Chef Local mode (-z)
embedded chef-zero (faster)
10
INSTALLATION MODES
Package
Gem (if ruby already exists on the system)
Bootstrap (client)
11
KNIFE – CHEF'S SWISS KNIFE
Management of chef environment
Search, SSH (Executing commands in parallel)
Plugins (VMware, solo, spork...)
12
CLIENTS AND NODES
A client is a registered machine with the server
A Node is a client that executes one or more recipe
→ A node is a client but a client is not necessarily a node
13
CHEF REPOSITORY
› git clone git://github.com/chef/chef-repo.git && cd chef-repo
14
THE COOKBOOKS – WHAT WE WILL COOK TODAY ?
Create a cookbook
Structure
Metadata
Recipes
Attributes
Files & Templates
Resources & Providers
LWRP & HWRP
De nitions
Library
15
THE COOKBOOKS - CREATE
Three options to create a cookbook:
Create your own
Get a cookbook from Chef Supermarket
Get a cookbook from Github (or other)
knife cookbook create nginx
chef generate cookbook nginx
knife cookbook site install nginx
16
THE COOKBOOKS - STRUCTURE
$ tree cookbooks/mdm/
cookbooks/mdm/
├── CHANGELOG.md
├── README.md
├── attributes
├── definitions
├── files
│   └── default
├── libraries
├── metadata.rb
├── providers
├── recipes
│   └── default.rb
├── resources
└── templates
└── default
17
THE COOKBOOKS - METADATA
18
THE COOKBOOKS - RECIPES
› cookbook/elasticsearch/recipes/default.rb
include_recipe 'java'
include_recipe 'elasticsearch::repo'
include_recipe 'elasticsearch::sysctl'
package 'elasticsearch'
template '/etc/default/elasticsearch' do
source 'elasticsearch.default.erb'
owner 'root'
group 'root'
mode 0644
notifies :restart, 'service[elasticsearch]', :delayed
end
template node[:elasticsearch][:conf_file] do
source 'elasticsearch.yml.erb'
owner 'root'
group 'root'
mode 0644
notifies :restart, 'service[elasticsearch]', :delayed
end
include_recipe 'elasticsearch::plugins'
service 'elasticsearch' do
supports :status => true, :restart => true
action [:enable, :start]
end
› cookbook/elasticsearch/recipes/repo.rb apt_repository "elasticsearch" do
uri "http://packages.elasticsearch.org/elasticsearch/1.4/debian"
distribution "stable"
components ['main']
arch "amd64"
key "https://packages.elasticsearch.org/GPG-KEY-elasticsearch"
action :add
end
19 . 1
THE COOKBOOKS - ATTRIBUTES
› memcache/attributes/default.rb
default['memcached']['memory'] = 64
default['memcached']['port'] = 11211
default['memcached']['udp_port'] = 11211
default['memcached']['listen'] = '0.0.0.0'
default['memcached']['maxconn'] = 1024
default['memcached']['max_object_size'] = '1m'
default['memcached']['logfilename'] = 'memcached.log'
case node['platform_family']
when 'ubuntu'
default['memcached']['user'] = 'memcache'
default['memcached']['group'] = 'memcache'
when 'debian'
default['memcached']['user'] = 'nobody'
default['memcached']['group'] = 'nogroup'
else
default['memcached']['user'] = 'nobody'
default['memcached']['user'] = 'nogroup'
end
19 . 2
THE COOKBOOKS - ATTRIBUTES
case node['platform']
when "debian"
case
when node['platform_version'].to_f < 6.0 # All 5.X
default['postgresql']['version'] = "8.3"
when node['platform_version'].to_f < 7.0 # All 6.X
default['postgresql']['version'] = "8.4"
else
default['postgresql']['version'] = "9.1"
end
default['postgresql']['client']['packages'] = ["postgresql-client-#{node['postgresql']['version']}","libpq-dev"]
default['postgresql']['server']['packages'] = ["postgresql-#{node['postgresql']['version']}"]
default['postgresql']['contrib']['packages'] = ["postgresql-contrib-#{node['postgresql']['version']}"]
when "fedora"
...
19 . 3
THE COOKBOOKS - AUTOMATIC ATTRIBUTES
19 . 4
THE COOKBOOKS - ATTRIBUTES
20 . 1
THE COOKBOOKS - TEMPLATES
template '/etc/memcached.conf' do
source 'memcached.conf.erb'
owner 'root'
group 'root'
mode '0644'
variables(
:listen => node['memcached']['listen'],
:logfilename => node['memcached']['logfilename'],
:user => node['memcached']['user'],
:port => node['memcached']['port'],
:udp_port => node['memcached']['udp_port'],
:maxconn => node['memcached']['maxconn'],
:memory => node['memcached']['memory'],
:max_object_size => node['memcached']['max_object_size']
)
notifies :restart, 'service[memcached]'
end
20 . 2
THE COOKBOOKS - TEMPLATES
› memcached/templates/default/memcached.conf.erb
# Run memcached as a daemon. This command is implied, and is not needed for the
# daemon to run. See the README.Debian that comes with this package for more
# information.
-d
# Log memcached's output to /var/log/memcached
logfile /var/log/<%= @logfilename %>
# Be verbose
#-v
# Be even more verbose (print client commands as well)
# -vv
# Start with a cap of 64 megs of memory. It's reasonable, and the daemon default
# Note that the daemon will grow to this size, but does not start out holding this much
# memory
-m <%= @memory %>
# Default connection port is 11211
-p <%= @port %>
#-U <%= @udp_port %>
# Run the daemon as root. The start-memcached will default to running as root if no
# -u command is present in this config file
-u <%= @user %>
# Specify which IP address to listen on. The default is to listen on all IP addresses
# This parameter is one of the only security measures that memcached has, so make sure
21 . 1
THE COOKBOOKS - FILES
› cookbook/app/ les/default/application-5.8.3.pm
› cookbook/app/ les/default/application-5.20.1.pm
cookbook_file "application.pm" do
path case node['platform']
when "centos","redhat"
"/usr/lib/version/1.2.3/dir/application.pm"
when "arch"
"/usr/share/version/core_version/dir/application.pm"
else
"/etc/version/dir/application.pm"
end
source "application-#{node['languages']['perl']['version']}.pm"
owner 'root'
group 'root'
mode '0644'
end
21 . 2
THE COOKBOOKS - FILES
1. /host-$fqdn/$source
2. /$platform-$platform_version/$source
3. /$platform/$source
4. /default/$source
5. /$source
host-foo.example.com/apache2_module_conf_generate.pl
ubuntu-10.04/apache2_module_conf_generate.pl
ubuntu-10/apache2_module_conf_generate.pl
ubuntu/apache2_module_conf_generate.pl
default/apache2_module_conf_generate.pl
22
THE COOKBOOKS – RESOURCES & PROVIDERS (LWRP / HWRP)
$ tree cookbooks/mdm/
cookbooks/mdm/
├── CHANGELOG.md
├── README.md
├── attributes
├── definitions
├── files
│   └── default
├── libraries
├── metadata.rb
├── providers
├── recipes
│  └── default.rb
├── resources
└── templates
└── default
Resources : Used to de ne a set of actions and attributes
Providers : Used to say to chef-client what to do foreach
de ned actions
user 'random' do
supports :manage_home => true
comment 'Random User'
uid 1234
gid 'users'
home '/home/random'
shell '/bin/bash'
password '$1$JJsvHslV$szsCjVEroftprNn4JHtDi'
end
23 . 1
THE COOKBOOKS – RESOURCES & PROVIDERS (LWRP)
› cookbooks/nginx/resources/app_site.rb
actions :create, :remove, :enable, :disable
default_action :create
attribute :app_name, :kind_of => String, :name_attribute => true, :required => true
attribute :cookbook, :kind_of => String, :default => "nginx"
attribute :template, :kind_of => String, :default => "reverse"
attribute :server_name, :kind_of => String
attribute :proxy_pass_url, :kind_of => String
attr_accessor :exists
attr_accessor :enabled
23 . 2
THE COOKBOOKS – RESOURCES & PROVIDERS (LWRP)
› cookbooks/nginx/providers/app_site.rb
23 . 3
THE COOKBOOKS – RESOURCES & PROVIDERS (LWRP)
23 . 4
THE COOKBOOKS – RESOURCES & PROVIDERS (LWRP)
› cookbooks/stash/recipes/nginx.rb
node.default['nginx']['default_site_enabled'] = false
include_recipe 'nginx'
nginx_app_site "stash" do
server_name node[:stash][:http_server][:virtual_host_name]
proxy_pass_url "http://localhost:#{node[:stash][:tomcat][:port]}"
cookbook "nginx"
template "reverse"
action [:create, :enable]
notifies :reload, 'service[nginx]', :delayed
end
nginx_app_site "stash-ssl" do
server_name node[:stash][:http_server][:virtual_host_name]
proxy_pass_url "http://localhost:#{node[:stash][:tomcat][:ssl_port]}"
cookbook "nginx"
template "reverse_ssl"
action [:create, :enable]
notifies :reload, 'service[nginx]', :delayed
end
24
THE COOKBOOKS – RESOURCES & PROVIDERS
(HWRP)
Create a resource with pure Ruby
Bypass the limits of Chef DSL
http://tech.yipit.com/2013/05/09/advanced-chef-writing-
heavy-weight-resource-providers-hwrp/
25 . 1
THE COOKBOOKS - DEFINITIONS
Used to de ne a set of actions with or without parameters
(function)
You can call your de nition many times in one or more
recipes
This is the same as a resource (LWRP) except that you may
not notify (trigger) other resources
25 . 2
THE COOKBOOKS - DEFINITIONS
define :host_porter, :port => 4000, :hostname => nil do
params[:hostname] ||= params[:name]
directory "/etc/#{params[:hostname]}" do
recursive true
end
file "/etc/#{params[:hostname]}/#{params[:port]}" do
content "some content"
end
end
host_porter node['hostname'] do
port 4000
end
host_porter "www1" do
port 4001
end
26 . 1
CUSTOM RESOURCES
Providers (LWRP/HWRP)
De nitions
DEPRECATED
26 . 2
CUSTOM RESOURCES
From Chef version 12.5
De nitions are useless. Advise you to use a resources instead.
Custom resources is a provider redesigned to be simpler.
Located only in "resources" directory
26 . 3
CUSTOM RESOURCES
exampleco/resources/site.rb
property :homepage, String, default: '<h1>Hello world!</h1>'
load_current_value do
if ::File.exist?('/var/www/html/index.html')
homepage IO.read('/var/www/html/index.html')
end
end
action :create do
package 'httpd'
service 'httpd' do
action [:enable, :start]
end
file '/var/www/html/index.html' do
content homepage
end
end
action :delete do
package 'httpd' do
action :delete
end
end
exampleco_site 'httpd' do
homepage '<h1>Welcome to the Example Co. website!</h1>'
action :create
end
27 . 1
THE COOKBOOKS - LIBRARIES
Allows to extend the Chef's classes or create your own Ruby lib
Do what Chef does not already do
Do external data processing to use in Chef
27 . 2
THE COOKBOOKS - LIBRARIES
27 . 3
THE COOKBOOKS - LIBRARIES
28
ROLES
Used to group one or more roles, attributes and recipes
{
"name": "ns",
"description": "The DNS Role",
"json_class": "Chef::Role",
"default_attributes": {
"bind": {
"dnssec-validation": "no",
"zone-statistics": "yes",
"zones": [ "company.local", "0.168.192.in-addr.arpa" ],
"acls": {
"slaves": [ "192.168.0.2", "192.168.0.3" ],
"mdm_network": [ "192.168.0.0/24" ]
},
"allow_query": ["localhost", "company_network"],
"allow_recursion": ["localhost", "company_network"],
"allow_query_cache": ["localhost", "company_network"],
"also_notify": true,
}
},
"override_attributes": {
"bind": {
"masters": ["192.168.0.1"],
"allow_transfer": ["slaves"]
},
"ntp": {
"is_server": true,
"servers": ["0.fr.pool.ntp.org", "1.fr.pool.ntp.org", "2.fr.pool.ntp.org", "3.fr.pool.ntp.org"],
"restrictions": ["192.168.0.0 mask 255.255.255.0 nomodify notrap"]
29
ENVIRONNEMENTS
{
"name": "staging",
"description": "The Staging environment",
"cookbook_versions": {
"box": "= 0.3.8",
"nginx": "= 0.5.3",
"web": "= 1.4.29",
"base": "= 0.3.4",
"tools": "= 0.4.2",
"users": "= 1.7.5",
"db": "= 0.3.5",
"ambari": "= 0.7.1"
},
"json_class": "Chef::Environment",
"chef_type": "environment",
"default_attributes": {
"php-fpm": {
"session": {
"save_handler_type": "memcache",
"save_handler_path": "tcp://localhost:11211"
},
"log_level": "alert",
"emergency_restart_threshold": "10",
"emergency_restart_interval": "1m",
"process_control_timeout": "10s"
},
"nginx": {
"keepalive_timeout": "40s",
"client_body_timeout": "20s",
"client_header_timeout": "20s",
"server_tokens": "off",
"disable_access_log": "true",
"mdm_log_format": "true",
"fastcgi_buffers": "8 16k",
"fastcgi_buffer_size": "32k",
Sets behaviors or different attributes for a given environment
include_recipe 'sshd'
include_recipe 'sudo'
include_recipe 'users::system'
include_recipe 'users::admins'
include_recipe 'users::operators'
if node.chef_environment != 'prod'
include_recipe 'users::qa'
include_recipe 'users::devops'
end
case node[:platform_family]
when 'debian'
include_recipe 'apt'
when 'rhel', 'fedora'
include_recipe 'yum'
include_recipe 'yum-epel'
include_recipe 'selinux::disabled'
include_recipe 'iptables::disabled'
end
include_recipe 'timezone-ii'
include_recipe 'ntp'
include_recipe 'postfix'
include_recipe 'snmp'
include_recipe 'locales'
include_recipe 'line'
POLICYFILE
Policies are a new feature of Chef that combine the very best
parts of Roles, Environments and cookbook dependency
resolvers (Berkshelf) into a single easy to use work ow.
It is associated with a group of nodes, cookbooks, and
settings. When these nodes run, they run the recipes
speci ed in the Policy le run-list
Resolves real-world problems of team work ow
30 . 130 . 2
POLICYFILE
De ne in Policy le:
› Run list
› Cookbook dependencies with version constraint and source
› Attributes overriding
name "jenkins-master"
run_list "java", "jenkins::master", "recipe[policyfile_demo]"
default_source :supermarket, "https://mysupermarket.example"
cookbook "policyfile_demo", path: "cookbooks/policyfile_demo"
cookbook "jenkins", "~> 2.1"
cookbook "mysql", github: "chef-cookbooks/mysql", branch: "master"
default['java']['version'] = '8'
30 . 3
POLICYFILE
Create your policy le into your cookbook
$ cd cookbooks/app
$ chef generate policyfile
$ ls -l
-rw-r--r-- 1 mlopez wheel 596 12 oct 21:57 Policyfile.rb
Create your policy le into policies directory
$ ls -l
total 24
-rw-r--r-- 1 mlopez wheel 70 12 oct 22:16 LICENSE
-rw-r--r-- 1 mlopez wheel 1546 12 oct 22:16 README.md
-rw-r--r-- 1 mlopez wheel 1067 12 oct 22:16 chefignore
drwxr-xr-x 4 mlopez wheel 136 12 oct 22:16 cookbooks
drwxr-xr-x 4 mlopez wheel 136 12 oct 22:16 data_bags
drwxr-xr-x 4 mlopez wheel 136 12 oct 23:13 policies
$ chef generate policyfile policies/web
30 . 4
Create the lock le
$ chef install [path/to/policyfile.rb]
Update lock le after modi cation
$ chef update [path/to/policyfile.rb]
Upload cookbook and policy le
$ chef push POLICY_GROUP PATH/TO/POLICYFILE.rb
→ If POLICY_GROUP doesn't exists it will be created
Assign policy name and policy group into client.rb's node or in
node con guration
31 . 1
DATA BAGS
Stock & share data
Read the desired data from a recipe or from your desktop (knife)
Write data collected during a run
$ knife data bag create DATA_BAG_NAME [ITEM]
knife data bag create users toto
knife data bag from file users data_bags/users/toto.json
31 . 2
DATA BAGS
› knife data bag create users myUser
{
"id": "myUser",
"ssh_keys": [
"ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDakMx4mjWYMko2r085yq/vq0Ey2DiVWXeJ
],
"groups": [
"mdm_qa"
],
"uid": 4001,
"shell": "/bin/bash",
"comment": "My User",
"action": "remove"
}
› knife data bag show users myUser –Fj > data_bags/users/myUser.json
31 . 3
DATA BAGS
admins = []
search(:admins, "*:*").each do |admin|
login = admin[‘id’]
admins << login
home = "/home/#{login}"
user login do
uid admin['uid']
gid admin['gid']
shell admin['shell']
home home
comment admin['comment’]
supports :manage_home => true
end
end
31 . 4
Write in a data bag
DATA BAGS
sam = {
"id" => "sam",
"Full Name" => "Sammy",
"shell" => "/bin/zsh"
}
databag_item = Chef::DataBagItem.new
databag_item.data_bag("users")
databag_item.raw_data = sam
databag_item.save
sam = data_bag_item("users", "sam")
sam["Full Name"] = "Samantha"
sam.save
31 . 5
What should I do if I have sensitive data ?
DATA BAGS
→ Encrypt !!
$ knife data bag create passwords postgresql –secret-file <path>/encrypted_data_bag_secret</path>
$ knife data bag show passwords postgresql -Fj
{
"id": "postgresql",
"password": {
"encrypted_data": "nu0GFIaJuzefK1iCgmYxWbRO64tvEezZJA/7iOUT87NLg=n",
"iv": "LWK$u1omaWHHNfzfDcYN45g==n",
"version": 1,
"cipher": "aes-256-cbc"
},
"databases": {
"encrypted_data": "lG4EULs9UQKKwjfzef8/WrccoGilQO2m7O6JNnIeMu199jGIT2l+/MvR+bX6dnk2U/_dwn
"iv": "UmtXyR9m0ornADWbiayPyw==n",
"version": 1,
"cipher": "aes-256-cbc"
}
}
32 . 1
CHEF VAULT
Whithout Chef Vault, nodes share a shared secret key le
→ Not good for security
With Chef Vault, nodes and workstation use their own
keypair to decrypt data bag. Chef administrators de ne
which node or admin can access to the encrypted data
32 . 2
or
CHEF VAULT
gem install chef-vault
Create a Vault
Allow mlopez, vaubert and nodes with web role assigned to decrypt content
$ knife vault create credentials database -A mlopez,vaubert -M client -S ‘roles:web' 
-J '{“db_password”:”some_password”}'
$ knife encrypt create credentials database --json '{“db_password”:”some_password”}' 
--search 'role:web' 
--admins mlopez, vaubert --mode client
32 . 3
or
CHEF VAULT
From an Unauthorized admin's workstation
$ knife vault show credentials database
db_password:
cipher: aes-256-cbc
encrypted_data: dsiBtADAV8Sbis89yKuYBvbdNXPpu8bQfJrS20op7zoysfR8roFlzpVHyoaG2
4yb3
iv: +0siNLzFHHqEkP07k6JhYw==
version: 1
id: database
Authorized users see the decrypted content
$ knife vault show credentials database
$ knife decrypt credentials database --mode client
db_password: some_password
id: database
32 . 4
CHEF VAULT
Content of database_keys
$ knife data bag show credentials database_keys
admins:
mlopez
vaubert
clients:
web-01
web-02
id: database_test_keys
mlopez: SOME KEY
vaubert: SOME KEY
web-01: SOME KEY
web-02: SOME KEY
32 . 5
CHEF VAULT
Add a new admin workstation
$ knife vault update credentials database -A mhue
Rotate all keys
$ knife vault rotate all keys
32 . 6
CHEF VAULT
Use Vault in recipe
chef_gem 'chef-vault' do
compile_time true if respond_to?(:compile_time)
end
require 'chef-vault'
# Or just include chef_vault coobkook
include_recipe 'chef-vault'
case ChefVault::Item.data_bag_item_type('credentials', 'database')
when :normal
...
when :encrypted
...
when :vault
...
end
33 . 1
OHAI
Ohai is a tool that is used to detect attributes on a node, inventory the
system (platform, cpu, memory...), and then provide automatically these
attributes to the chef-client at the start of every chef-client run.
Check Automatic attributes previously seen. You can list node attributes
by running "ohai" command
$ ohai
{
"hostname": "node-01",
"machinename": "node-01",
"fqdn": "node-01.my.local",
"domain": "my.local",
"network": {
"interfaces": {
"lo": {
"encapsulation": "Loopback",
"addresses": {
"127.0.0.1": {
"family": "inet",
"prefixlen": "8",
"netmask": "255.0.0.0",
...
33 . 2
OHAI - CUSTOM PLUGIN
You can create your own Ohai plugin to collect data before
the run and set them into Ohai as an attribute
Use Ohai cookbook and put your plugin into ' les' directory
› cookbooks/ohai/ les/default/plugins/haproxy.rb
# Encoding: utf-8
# Get current version of Haproxy
Ohai.plugin(:Haproxy) do
provides 'haproxy'
collect_data(:linux) do
haproxy Mash.new
[['dpkg-query -W haproxy | awk '{print $2}' | sed 's/(^[1-9].[1-9]).*/1/'',
:installed_version]].each do |cmd, property|
so = shell_out(cmd)
haproxy[property] = so.stdout.delete("n")
end
end
end
34
TEAM WORKING
Git -> Create branches
Test locally before uploading his cookbook
Vagrant / Virtualbox, Kitchen CI...etc.
Chef solo/zero/local
Bump version of your cookbook and upload it into environment
Freeze uploaded version and apply version constraint into environnements
Knife Spork – Plugin allow you to bump, upload and promote a cookbook more easily.
Noti cation plugin, auto git add …
$ knife cookbook upload nginx [--freeze] [--force] [-E <environment>]
$ knife spork omni nginx –l minor –e qa_group</environment>
35 . 1
TEST HIS JOB
Syntax
Logical tests with Foodcritic
Obsolescence of resources used, invalid search queries, syntax best-
practice…
Unit tests with ChefSpec
$ knife cookbook test nginx
package 'foo'
require 'chefspec'
describe 'example::default' do
let(:chef_run) { ChefSpec::SoloRunner.converge(described_recipe) }
it 'installs foo' do
expect(chef_run).to install_package('foo')
end
end
35 . 2
TEST HIS JOB
Integration tests with Kitchen CI
Check if a run runs without errors (converge)
Include unit tests (Rspec, Bats…)
Run many tests suite in one time (client / server) on many platforms
Check which nodes use your cookbook
Simulate an execution of a run
knife preflight web::ws
knife search node -i "recipes:web::ws »
chef-client --why-run
knife ssh ‘name:srv-01.pp’ ‘sudo chef-client –W’
36
DEBUG – WHY IT DOESN'T WORK ?
chef-client –l debug
Generate logs
Chef::Log.debug(« Doesn’t work »)
puts myVariable
Raise Exceptions
Chef::Log.fatal!('Deployment failure...')
raise
37 . 1
ADVANCED
Override a run-list (-o "recipe[]")
Override a community Cookbook
knife ssh 'name:srv-01.dev' 'sudo chef-client –o "recipe[firefox]"'
include_recipe 'nginx'
resources("template[/etc/nginx/nginx.conf]").cookbook 'myNginx')
37 . 2
ADVANCED
Noti cations by chef-handler
Knife diff (/*)
38
SOURCES & BOOKS
Chef Starter
Chef Infrastructure Automation Cookbook
https://docs.chef.io
http://tech.yipit.com/2013/05/09/advanced-chef-writing-heavy-
weight-resource-providers-hwrp/
http://dougireton.com/blog/2013/02/03/knife-tricks
http://www.pburkholder.com/blog/2015/04/23/emulating-
cookbook-semver-build-numbers-with-chef-policy les/
https://yolover.poise.io/
SOURCES & BOOKS
› Chef Vault
https://github.com/chef/chef-vault
https://blog.chef.io/2016/01/21/chef-vault-what-is-it-and-what-can-it-do-for-you/
https://blog.chef.io/2013/09/19/managing-secrets-with-chef-vault/
http://hedge-ops.com/chef-vault-tutorial/

More Related Content

PPTX
Herd your chickens: Ansible for DB2 configuration management
PDF
Medicine show2 Drupal Bristol Camp 2015
PDF
10 Million hits a day with WordPress using a $15 VPS
PPT
Python Deployment with Fabric
PDF
Continuous Infrastructure: Modern Puppet for the Jenkins Project - PuppetConf...
PDF
Ansible 實戰:top down 觀點
PDF
From Dev to DevOps - Codemotion ES 2012
PDF
Continuous Delivery: The Next Frontier
Herd your chickens: Ansible for DB2 configuration management
Medicine show2 Drupal Bristol Camp 2015
10 Million hits a day with WordPress using a $15 VPS
Python Deployment with Fabric
Continuous Infrastructure: Modern Puppet for the Jenkins Project - PuppetConf...
Ansible 實戰:top down 觀點
From Dev to DevOps - Codemotion ES 2012
Continuous Delivery: The Next Frontier

What's hot (20)

PPTX
2019 Chef InSpec Jumpstart Part 2 of 2
PDF
A tour of Ansible
PDF
Ansible Meetup Hamburg / Quickstart
PDF
Configuring Django projects for multiple environments
ODP
Chef training - Day2
PPTX
2019 Chef InSpec Jumpstart Part 1 of 2
PDF
How to Develop Puppet Modules: From Source to the Forge With Zero Clicks
PDF
Ansible : what's ansible & use case by REX
PDF
Zero Downtime Deployment with Ansible
PDF
Take control of your Jenkins jobs via job DSL.
PDF
Drupal Camp Brighton 2015: Ansible Drupal Medicine show
PDF
Common configuration with Data Bags - Fundamentals Webinar Series Part 4
PDF
Beaker: Automated, Cloud-Based Acceptance Testing - PuppetConf 2014
PDF
Portland PUG April 2014: Beaker 101: Acceptance Test Everything
PDF
Zero Downtime Deployment with Ansible
PDF
Designing net-aws-glacier
PDF
Salt conf 2014-installing-openstack-using-saltstack-v02
PDF
Continuous infrastructure testing
KEY
From Dev to DevOps - ApacheCON NA 2011
KEY
From Dev to DevOps - Apache Barcamp Spain 2011
2019 Chef InSpec Jumpstart Part 2 of 2
A tour of Ansible
Ansible Meetup Hamburg / Quickstart
Configuring Django projects for multiple environments
Chef training - Day2
2019 Chef InSpec Jumpstart Part 1 of 2
How to Develop Puppet Modules: From Source to the Forge With Zero Clicks
Ansible : what's ansible & use case by REX
Zero Downtime Deployment with Ansible
Take control of your Jenkins jobs via job DSL.
Drupal Camp Brighton 2015: Ansible Drupal Medicine show
Common configuration with Data Bags - Fundamentals Webinar Series Part 4
Beaker: Automated, Cloud-Based Acceptance Testing - PuppetConf 2014
Portland PUG April 2014: Beaker 101: Acceptance Test Everything
Zero Downtime Deployment with Ansible
Designing net-aws-glacier
Salt conf 2014-installing-openstack-using-saltstack-v02
Continuous infrastructure testing
From Dev to DevOps - ApacheCON NA 2011
From Dev to DevOps - Apache Barcamp Spain 2011
Ad

Viewers also liked (6)

PPTX
Achieving DevOps Success with Chef Automate
PDF
Chef Automate Workflow Demo
PDF
Chef Cookbook Testing and Continuous Integration
PDF
Introduction to Chef: Automate Your Infrastructure by Modeling It In Code
PPTX
AMIS SIG - Introducing Apache Kafka - Scalable, reliable Event Bus & Message ...
PPTX
Jenkins and Chef: Infrastructure CI and Automated Deployment
Achieving DevOps Success with Chef Automate
Chef Automate Workflow Demo
Chef Cookbook Testing and Continuous Integration
Introduction to Chef: Automate Your Infrastructure by Modeling It In Code
AMIS SIG - Introducing Apache Kafka - Scalable, reliable Event Bus & Message ...
Jenkins and Chef: Infrastructure CI and Automated Deployment
Ad

Similar to Chef - industrialize and automate your infrastructure (20)

PDF
Puppet: Eclipsecon ALM 2013
KEY
Cooking with Chef
PDF
From Dev to DevOps
PPTX
PPTX
PHP on Heroku: Deploying and Scaling Apps in the Cloud
PPTX
InSpec Workshop at Velocity London 2018
PDF
Automação do físico ao NetSecDevOps
PPTX
A Fabric/Puppet Build/Deploy System
PDF
DevOps in PHP environment
PDF
infra-as-code
PDF
Ansible new paradigms for orchestration
PDF
Writing & Sharing Great Modules - Puppet Camp Boston
PDF
Linux Desktop Automation
PDF
Practical Chef and Capistrano for Your Rails App
PDF
Automation day red hat ansible
PDF
Melbourne Infracoders: Compliance as Code with InSpec
PDF
PuppetDB: Sneaking Clojure into Operations
PDF
Ansible Introduction
PPTX
BuildStuff.LT 2018 InSpec Workshop
PDF
Puppet getting started by Dirk Götz
Puppet: Eclipsecon ALM 2013
Cooking with Chef
From Dev to DevOps
PHP on Heroku: Deploying and Scaling Apps in the Cloud
InSpec Workshop at Velocity London 2018
Automação do físico ao NetSecDevOps
A Fabric/Puppet Build/Deploy System
DevOps in PHP environment
infra-as-code
Ansible new paradigms for orchestration
Writing & Sharing Great Modules - Puppet Camp Boston
Linux Desktop Automation
Practical Chef and Capistrano for Your Rails App
Automation day red hat ansible
Melbourne Infracoders: Compliance as Code with InSpec
PuppetDB: Sneaking Clojure into Operations
Ansible Introduction
BuildStuff.LT 2018 InSpec Workshop
Puppet getting started by Dirk Götz

Recently uploaded (20)

PPTX
Detection-First SIEM: Rule Types, Dashboards, and Threat-Informed Strategy
PDF
Architecting across the Boundaries of two Complex Domains - Healthcare & Tech...
PPTX
PA Analog/Digital System: The Backbone of Modern Surveillance and Communication
PDF
Building Integrated photovoltaic BIPV_UPV.pdf
PDF
Chapter 3 Spatial Domain Image Processing.pdf
PDF
Modernizing your data center with Dell and AMD
PPT
Teaching material agriculture food technology
PDF
CIFDAQ's Market Insight: SEC Turns Pro Crypto
PDF
Diabetes mellitus diagnosis method based random forest with bat algorithm
PDF
Review of recent advances in non-invasive hemoglobin estimation
PDF
NewMind AI Monthly Chronicles - July 2025
PPTX
KOM of Painting work and Equipment Insulation REV00 update 25-dec.pptx
PDF
Empathic Computing: Creating Shared Understanding
PDF
Reach Out and Touch Someone: Haptics and Empathic Computing
PDF
TokAI - TikTok AI Agent : The First AI Application That Analyzes 10,000+ Vira...
PPTX
Big Data Technologies - Introduction.pptx
PDF
Blue Purple Modern Animated Computer Science Presentation.pdf.pdf
PDF
Machine learning based COVID-19 study performance prediction
PDF
Agricultural_Statistics_at_a_Glance_2022_0.pdf
PDF
Electronic commerce courselecture one. Pdf
Detection-First SIEM: Rule Types, Dashboards, and Threat-Informed Strategy
Architecting across the Boundaries of two Complex Domains - Healthcare & Tech...
PA Analog/Digital System: The Backbone of Modern Surveillance and Communication
Building Integrated photovoltaic BIPV_UPV.pdf
Chapter 3 Spatial Domain Image Processing.pdf
Modernizing your data center with Dell and AMD
Teaching material agriculture food technology
CIFDAQ's Market Insight: SEC Turns Pro Crypto
Diabetes mellitus diagnosis method based random forest with bat algorithm
Review of recent advances in non-invasive hemoglobin estimation
NewMind AI Monthly Chronicles - July 2025
KOM of Painting work and Equipment Insulation REV00 update 25-dec.pptx
Empathic Computing: Creating Shared Understanding
Reach Out and Touch Someone: Haptics and Empathic Computing
TokAI - TikTok AI Agent : The First AI Application That Analyzes 10,000+ Vira...
Big Data Technologies - Introduction.pptx
Blue Purple Modern Animated Computer Science Presentation.pdf.pdf
Machine learning based COVID-19 study performance prediction
Agricultural_Statistics_at_a_Glance_2022_0.pdf
Electronic commerce courselecture one. Pdf

Chef - industrialize and automate your infrastructure

  • 1. 1 CHEF INDUSTRIALIZE & AUTOMATE YOUR INFRASTRUCTURE Created by Michael Lopez - © Maisons du Monde
  • 2. 2 TO DO WHAT ? Con gure a machine in minutes without action Deploy software or application Ensure that all your machines are identical Gain time !
  • 3. 3 . 1 HOW DOES IT WORKS ?
  • 4. HOW DOES IT WORKS ?
  • 5. 3 . 24 CHEF VS PUPPET/ANSIBLE PUPPET Language : DSL Ruby/Json Approach : Execution by dependency and chained action Stored data : YAML Agent : Yes ANSIBLE Language : Python Approach : Hierarchical Execution Stored data : YAML Agent : No CHEF Language : Full Ruby & DSL Approach : Hierarchical Execution Stored data : JSON Agent : Yes
  • 6. PUPPET package { 'openssh-server': ensure => present, before => File['/etc/ssh/sshd_config'], } file { '/etc/ssh/sshd_config': ensure => file, mode => 600, source => 'puppet:///modules/sshd/sshd_config', require => Package['openssh-server'], } service { 'sshd': ensure => running, enable => true, subscribe => File['/etc/ssh/sshd_config'], }
  • 7. 5 . 15 . 2 PUPPET Chained Actions package { 'openssh-server': ensure => present, } -> # and then: file { '/etc/ssh/sshd_config': ensure => file, mode => 600, source => 'puppet:///modules/sshd/sshd_config', } ~> # and then: service { 'sshd': ensure => running, enable => true, }
  • 8. 5 . 3 PUPPET Virtual Resources @user {'deploy': uid => 2004, comment => 'Deployment User', group => www-data, groups => ["enterprise"], tag => [deploy, web], } realize User['deploy'], User['zleslie'] User <| tag == web |>
  • 9. 6 . 1 ANSIBLE Playbook --- # This role installs HAProxy and configures it. - name: Download and install haproxy and socat apt: name={{ item }} state=present with_items: - haproxy - socat - name: Configure the haproxy cnf file with hosts template: src=haproxy.cfg.j2 dest=/etc/haproxy/haproxy.cfg notify: restart haproxy - name: Start the haproxy service service: name=haproxy state=started enabled=yes
  • 10. 6 . 2 ANSIBLE - apt: name=foo update_cache=yes - apt: name=foo state=absent - apt: name=foo=1.00 state=present - apt: deb=/tmp/mypackage.deb - file: path=/etc/foo.conf owner=foo group=foo mode=0644 - file: src=/tmp/{{ item.path }} dest={{ item.dest }} state=link with_items: - { path: 'x', dest: 'y' } - { path: 'z', dest: 'k' } - file: path=/etc/foo.conf state=touch mode="u=rw,g=r,o=r"
  • 11. 7 CHEF VS PUPPET/ANSIBLE Advantages Disadvantages Chef Fast & powerful Dev Oriented => Flexible Full Ruby Search Crypted data Flexibility -> Complexity Puppet Mature Large Community Lot of tools Slow Complex Language Complex execution order Ansible Agentless « No code » Multilingual plugins Immaturity: Small community and few tools Data stored in les
  • 12. 8 . 1 CHEF SERVER Free/Basic Version Free & no node limit In your Infrastructure › No access to "premium" features
  • 13. 8 . 2 CHEF SERVER
  • 14. 8 . 3 CHEF AUTOMATE Deliver a continuous deployment pipeline for infrastructure and applications. Gain insight into operational, compliance, and work ow events. Identify compliance issues, security risks, and outdated software with customizable reports
  • 15. 9 NOT ONLY CHEF SERVER... Chef Solo No Server Deploy & Run recipes directly on the node › No search & dedicated attributes on a node › Need to push cookbooks on each nodes › Not compatible with version constraint Chef Zero Chef Server instance in memory Chef Local mode (-z) embedded chef-zero (faster)
  • 16. 10 INSTALLATION MODES Package Gem (if ruby already exists on the system) Bootstrap (client)
  • 17. 11 KNIFE – CHEF'S SWISS KNIFE Management of chef environment Search, SSH (Executing commands in parallel) Plugins (VMware, solo, spork...)
  • 18. 12 CLIENTS AND NODES A client is a registered machine with the server A Node is a client that executes one or more recipe → A node is a client but a client is not necessarily a node
  • 19. 13 CHEF REPOSITORY › git clone git://github.com/chef/chef-repo.git && cd chef-repo
  • 20. 14 THE COOKBOOKS – WHAT WE WILL COOK TODAY ? Create a cookbook Structure Metadata Recipes Attributes Files & Templates Resources & Providers LWRP & HWRP De nitions Library
  • 21. 15 THE COOKBOOKS - CREATE Three options to create a cookbook: Create your own Get a cookbook from Chef Supermarket Get a cookbook from Github (or other) knife cookbook create nginx chef generate cookbook nginx knife cookbook site install nginx
  • 22. 16 THE COOKBOOKS - STRUCTURE $ tree cookbooks/mdm/ cookbooks/mdm/ ├── CHANGELOG.md ├── README.md ├── attributes ├── definitions ├── files │   └── default ├── libraries ├── metadata.rb ├── providers ├── recipes │   └── default.rb ├── resources └── templates └── default
  • 23. 17 THE COOKBOOKS - METADATA
  • 24. 18 THE COOKBOOKS - RECIPES › cookbook/elasticsearch/recipes/default.rb include_recipe 'java' include_recipe 'elasticsearch::repo' include_recipe 'elasticsearch::sysctl' package 'elasticsearch' template '/etc/default/elasticsearch' do source 'elasticsearch.default.erb' owner 'root' group 'root' mode 0644 notifies :restart, 'service[elasticsearch]', :delayed end template node[:elasticsearch][:conf_file] do source 'elasticsearch.yml.erb' owner 'root' group 'root' mode 0644 notifies :restart, 'service[elasticsearch]', :delayed end include_recipe 'elasticsearch::plugins' service 'elasticsearch' do supports :status => true, :restart => true action [:enable, :start] end › cookbook/elasticsearch/recipes/repo.rb apt_repository "elasticsearch" do uri "http://packages.elasticsearch.org/elasticsearch/1.4/debian" distribution "stable" components ['main'] arch "amd64" key "https://packages.elasticsearch.org/GPG-KEY-elasticsearch" action :add end
  • 25. 19 . 1 THE COOKBOOKS - ATTRIBUTES › memcache/attributes/default.rb default['memcached']['memory'] = 64 default['memcached']['port'] = 11211 default['memcached']['udp_port'] = 11211 default['memcached']['listen'] = '0.0.0.0' default['memcached']['maxconn'] = 1024 default['memcached']['max_object_size'] = '1m' default['memcached']['logfilename'] = 'memcached.log' case node['platform_family'] when 'ubuntu' default['memcached']['user'] = 'memcache' default['memcached']['group'] = 'memcache' when 'debian' default['memcached']['user'] = 'nobody' default['memcached']['group'] = 'nogroup' else default['memcached']['user'] = 'nobody' default['memcached']['user'] = 'nogroup' end
  • 26. 19 . 2 THE COOKBOOKS - ATTRIBUTES case node['platform'] when "debian" case when node['platform_version'].to_f < 6.0 # All 5.X default['postgresql']['version'] = "8.3" when node['platform_version'].to_f < 7.0 # All 6.X default['postgresql']['version'] = "8.4" else default['postgresql']['version'] = "9.1" end default['postgresql']['client']['packages'] = ["postgresql-client-#{node['postgresql']['version']}","libpq-dev"] default['postgresql']['server']['packages'] = ["postgresql-#{node['postgresql']['version']}"] default['postgresql']['contrib']['packages'] = ["postgresql-contrib-#{node['postgresql']['version']}"] when "fedora" ...
  • 27. 19 . 3 THE COOKBOOKS - AUTOMATIC ATTRIBUTES
  • 28. 19 . 4 THE COOKBOOKS - ATTRIBUTES
  • 29. 20 . 1 THE COOKBOOKS - TEMPLATES template '/etc/memcached.conf' do source 'memcached.conf.erb' owner 'root' group 'root' mode '0644' variables( :listen => node['memcached']['listen'], :logfilename => node['memcached']['logfilename'], :user => node['memcached']['user'], :port => node['memcached']['port'], :udp_port => node['memcached']['udp_port'], :maxconn => node['memcached']['maxconn'], :memory => node['memcached']['memory'], :max_object_size => node['memcached']['max_object_size'] ) notifies :restart, 'service[memcached]' end
  • 30. 20 . 2 THE COOKBOOKS - TEMPLATES › memcached/templates/default/memcached.conf.erb # Run memcached as a daemon. This command is implied, and is not needed for the # daemon to run. See the README.Debian that comes with this package for more # information. -d # Log memcached's output to /var/log/memcached logfile /var/log/<%= @logfilename %> # Be verbose #-v # Be even more verbose (print client commands as well) # -vv # Start with a cap of 64 megs of memory. It's reasonable, and the daemon default # Note that the daemon will grow to this size, but does not start out holding this much # memory -m <%= @memory %> # Default connection port is 11211 -p <%= @port %> #-U <%= @udp_port %> # Run the daemon as root. The start-memcached will default to running as root if no # -u command is present in this config file -u <%= @user %> # Specify which IP address to listen on. The default is to listen on all IP addresses # This parameter is one of the only security measures that memcached has, so make sure
  • 31. 21 . 1 THE COOKBOOKS - FILES › cookbook/app/ les/default/application-5.8.3.pm › cookbook/app/ les/default/application-5.20.1.pm cookbook_file "application.pm" do path case node['platform'] when "centos","redhat" "/usr/lib/version/1.2.3/dir/application.pm" when "arch" "/usr/share/version/core_version/dir/application.pm" else "/etc/version/dir/application.pm" end source "application-#{node['languages']['perl']['version']}.pm" owner 'root' group 'root' mode '0644' end
  • 32. 21 . 2 THE COOKBOOKS - FILES 1. /host-$fqdn/$source 2. /$platform-$platform_version/$source 3. /$platform/$source 4. /default/$source 5. /$source host-foo.example.com/apache2_module_conf_generate.pl ubuntu-10.04/apache2_module_conf_generate.pl ubuntu-10/apache2_module_conf_generate.pl ubuntu/apache2_module_conf_generate.pl default/apache2_module_conf_generate.pl
  • 33. 22 THE COOKBOOKS – RESOURCES & PROVIDERS (LWRP / HWRP) $ tree cookbooks/mdm/ cookbooks/mdm/ ├── CHANGELOG.md ├── README.md ├── attributes ├── definitions ├── files │   └── default ├── libraries ├── metadata.rb ├── providers ├── recipes │  └── default.rb ├── resources └── templates └── default Resources : Used to de ne a set of actions and attributes Providers : Used to say to chef-client what to do foreach de ned actions user 'random' do supports :manage_home => true comment 'Random User' uid 1234 gid 'users' home '/home/random' shell '/bin/bash' password '$1$JJsvHslV$szsCjVEroftprNn4JHtDi' end
  • 34. 23 . 1 THE COOKBOOKS – RESOURCES & PROVIDERS (LWRP) › cookbooks/nginx/resources/app_site.rb actions :create, :remove, :enable, :disable default_action :create attribute :app_name, :kind_of => String, :name_attribute => true, :required => true attribute :cookbook, :kind_of => String, :default => "nginx" attribute :template, :kind_of => String, :default => "reverse" attribute :server_name, :kind_of => String attribute :proxy_pass_url, :kind_of => String attr_accessor :exists attr_accessor :enabled
  • 35. 23 . 2 THE COOKBOOKS – RESOURCES & PROVIDERS (LWRP) › cookbooks/nginx/providers/app_site.rb
  • 36. 23 . 3 THE COOKBOOKS – RESOURCES & PROVIDERS (LWRP)
  • 37. 23 . 4 THE COOKBOOKS – RESOURCES & PROVIDERS (LWRP) › cookbooks/stash/recipes/nginx.rb node.default['nginx']['default_site_enabled'] = false include_recipe 'nginx' nginx_app_site "stash" do server_name node[:stash][:http_server][:virtual_host_name] proxy_pass_url "http://localhost:#{node[:stash][:tomcat][:port]}" cookbook "nginx" template "reverse" action [:create, :enable] notifies :reload, 'service[nginx]', :delayed end nginx_app_site "stash-ssl" do server_name node[:stash][:http_server][:virtual_host_name] proxy_pass_url "http://localhost:#{node[:stash][:tomcat][:ssl_port]}" cookbook "nginx" template "reverse_ssl" action [:create, :enable] notifies :reload, 'service[nginx]', :delayed end
  • 38. 24 THE COOKBOOKS – RESOURCES & PROVIDERS (HWRP) Create a resource with pure Ruby Bypass the limits of Chef DSL http://tech.yipit.com/2013/05/09/advanced-chef-writing- heavy-weight-resource-providers-hwrp/
  • 39. 25 . 1 THE COOKBOOKS - DEFINITIONS Used to de ne a set of actions with or without parameters (function) You can call your de nition many times in one or more recipes This is the same as a resource (LWRP) except that you may not notify (trigger) other resources
  • 40. 25 . 2 THE COOKBOOKS - DEFINITIONS define :host_porter, :port => 4000, :hostname => nil do params[:hostname] ||= params[:name] directory "/etc/#{params[:hostname]}" do recursive true end file "/etc/#{params[:hostname]}/#{params[:port]}" do content "some content" end end host_porter node['hostname'] do port 4000 end host_porter "www1" do port 4001 end
  • 41. 26 . 1 CUSTOM RESOURCES Providers (LWRP/HWRP) De nitions DEPRECATED
  • 42. 26 . 2 CUSTOM RESOURCES From Chef version 12.5 De nitions are useless. Advise you to use a resources instead. Custom resources is a provider redesigned to be simpler. Located only in "resources" directory
  • 43. 26 . 3 CUSTOM RESOURCES exampleco/resources/site.rb property :homepage, String, default: '<h1>Hello world!</h1>' load_current_value do if ::File.exist?('/var/www/html/index.html') homepage IO.read('/var/www/html/index.html') end end action :create do package 'httpd' service 'httpd' do action [:enable, :start] end file '/var/www/html/index.html' do content homepage end end action :delete do package 'httpd' do action :delete end end exampleco_site 'httpd' do homepage '<h1>Welcome to the Example Co. website!</h1>' action :create end
  • 44. 27 . 1 THE COOKBOOKS - LIBRARIES Allows to extend the Chef's classes or create your own Ruby lib Do what Chef does not already do Do external data processing to use in Chef
  • 45. 27 . 2 THE COOKBOOKS - LIBRARIES
  • 46. 27 . 3 THE COOKBOOKS - LIBRARIES
  • 47. 28 ROLES Used to group one or more roles, attributes and recipes { "name": "ns", "description": "The DNS Role", "json_class": "Chef::Role", "default_attributes": { "bind": { "dnssec-validation": "no", "zone-statistics": "yes", "zones": [ "company.local", "0.168.192.in-addr.arpa" ], "acls": { "slaves": [ "192.168.0.2", "192.168.0.3" ], "mdm_network": [ "192.168.0.0/24" ] }, "allow_query": ["localhost", "company_network"], "allow_recursion": ["localhost", "company_network"], "allow_query_cache": ["localhost", "company_network"], "also_notify": true, } }, "override_attributes": { "bind": { "masters": ["192.168.0.1"], "allow_transfer": ["slaves"] }, "ntp": { "is_server": true, "servers": ["0.fr.pool.ntp.org", "1.fr.pool.ntp.org", "2.fr.pool.ntp.org", "3.fr.pool.ntp.org"], "restrictions": ["192.168.0.0 mask 255.255.255.0 nomodify notrap"]
  • 48. 29 ENVIRONNEMENTS { "name": "staging", "description": "The Staging environment", "cookbook_versions": { "box": "= 0.3.8", "nginx": "= 0.5.3", "web": "= 1.4.29", "base": "= 0.3.4", "tools": "= 0.4.2", "users": "= 1.7.5", "db": "= 0.3.5", "ambari": "= 0.7.1" }, "json_class": "Chef::Environment", "chef_type": "environment", "default_attributes": { "php-fpm": { "session": { "save_handler_type": "memcache", "save_handler_path": "tcp://localhost:11211" }, "log_level": "alert", "emergency_restart_threshold": "10", "emergency_restart_interval": "1m", "process_control_timeout": "10s" }, "nginx": { "keepalive_timeout": "40s", "client_body_timeout": "20s", "client_header_timeout": "20s", "server_tokens": "off", "disable_access_log": "true", "mdm_log_format": "true", "fastcgi_buffers": "8 16k", "fastcgi_buffer_size": "32k", Sets behaviors or different attributes for a given environment include_recipe 'sshd' include_recipe 'sudo' include_recipe 'users::system' include_recipe 'users::admins' include_recipe 'users::operators' if node.chef_environment != 'prod' include_recipe 'users::qa' include_recipe 'users::devops' end case node[:platform_family] when 'debian' include_recipe 'apt' when 'rhel', 'fedora' include_recipe 'yum' include_recipe 'yum-epel' include_recipe 'selinux::disabled' include_recipe 'iptables::disabled' end include_recipe 'timezone-ii' include_recipe 'ntp' include_recipe 'postfix' include_recipe 'snmp' include_recipe 'locales' include_recipe 'line'
  • 49. POLICYFILE Policies are a new feature of Chef that combine the very best parts of Roles, Environments and cookbook dependency resolvers (Berkshelf) into a single easy to use work ow. It is associated with a group of nodes, cookbooks, and settings. When these nodes run, they run the recipes speci ed in the Policy le run-list Resolves real-world problems of team work ow
  • 50. 30 . 130 . 2 POLICYFILE De ne in Policy le: › Run list › Cookbook dependencies with version constraint and source › Attributes overriding name "jenkins-master" run_list "java", "jenkins::master", "recipe[policyfile_demo]" default_source :supermarket, "https://mysupermarket.example" cookbook "policyfile_demo", path: "cookbooks/policyfile_demo" cookbook "jenkins", "~> 2.1" cookbook "mysql", github: "chef-cookbooks/mysql", branch: "master" default['java']['version'] = '8'
  • 51. 30 . 3 POLICYFILE Create your policy le into your cookbook $ cd cookbooks/app $ chef generate policyfile $ ls -l -rw-r--r-- 1 mlopez wheel 596 12 oct 21:57 Policyfile.rb Create your policy le into policies directory $ ls -l total 24 -rw-r--r-- 1 mlopez wheel 70 12 oct 22:16 LICENSE -rw-r--r-- 1 mlopez wheel 1546 12 oct 22:16 README.md -rw-r--r-- 1 mlopez wheel 1067 12 oct 22:16 chefignore drwxr-xr-x 4 mlopez wheel 136 12 oct 22:16 cookbooks drwxr-xr-x 4 mlopez wheel 136 12 oct 22:16 data_bags drwxr-xr-x 4 mlopez wheel 136 12 oct 23:13 policies $ chef generate policyfile policies/web
  • 52. 30 . 4 Create the lock le $ chef install [path/to/policyfile.rb] Update lock le after modi cation $ chef update [path/to/policyfile.rb] Upload cookbook and policy le $ chef push POLICY_GROUP PATH/TO/POLICYFILE.rb → If POLICY_GROUP doesn't exists it will be created Assign policy name and policy group into client.rb's node or in node con guration
  • 53. 31 . 1 DATA BAGS Stock & share data Read the desired data from a recipe or from your desktop (knife) Write data collected during a run $ knife data bag create DATA_BAG_NAME [ITEM] knife data bag create users toto knife data bag from file users data_bags/users/toto.json
  • 54. 31 . 2 DATA BAGS › knife data bag create users myUser { "id": "myUser", "ssh_keys": [ "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDakMx4mjWYMko2r085yq/vq0Ey2DiVWXeJ ], "groups": [ "mdm_qa" ], "uid": 4001, "shell": "/bin/bash", "comment": "My User", "action": "remove" } › knife data bag show users myUser –Fj > data_bags/users/myUser.json
  • 55. 31 . 3 DATA BAGS admins = [] search(:admins, "*:*").each do |admin| login = admin[‘id’] admins << login home = "/home/#{login}" user login do uid admin['uid'] gid admin['gid'] shell admin['shell'] home home comment admin['comment’] supports :manage_home => true end end
  • 56. 31 . 4 Write in a data bag DATA BAGS sam = { "id" => "sam", "Full Name" => "Sammy", "shell" => "/bin/zsh" } databag_item = Chef::DataBagItem.new databag_item.data_bag("users") databag_item.raw_data = sam databag_item.save sam = data_bag_item("users", "sam") sam["Full Name"] = "Samantha" sam.save
  • 57. 31 . 5 What should I do if I have sensitive data ? DATA BAGS → Encrypt !! $ knife data bag create passwords postgresql –secret-file <path>/encrypted_data_bag_secret</path> $ knife data bag show passwords postgresql -Fj { "id": "postgresql", "password": { "encrypted_data": "nu0GFIaJuzefK1iCgmYxWbRO64tvEezZJA/7iOUT87NLg=n", "iv": "LWK$u1omaWHHNfzfDcYN45g==n", "version": 1, "cipher": "aes-256-cbc" }, "databases": { "encrypted_data": "lG4EULs9UQKKwjfzef8/WrccoGilQO2m7O6JNnIeMu199jGIT2l+/MvR+bX6dnk2U/_dwn "iv": "UmtXyR9m0ornADWbiayPyw==n", "version": 1, "cipher": "aes-256-cbc" } }
  • 58. 32 . 1 CHEF VAULT Whithout Chef Vault, nodes share a shared secret key le → Not good for security With Chef Vault, nodes and workstation use their own keypair to decrypt data bag. Chef administrators de ne which node or admin can access to the encrypted data
  • 59. 32 . 2 or CHEF VAULT gem install chef-vault Create a Vault Allow mlopez, vaubert and nodes with web role assigned to decrypt content $ knife vault create credentials database -A mlopez,vaubert -M client -S ‘roles:web' -J '{“db_password”:”some_password”}' $ knife encrypt create credentials database --json '{“db_password”:”some_password”}' --search 'role:web' --admins mlopez, vaubert --mode client
  • 60. 32 . 3 or CHEF VAULT From an Unauthorized admin's workstation $ knife vault show credentials database db_password: cipher: aes-256-cbc encrypted_data: dsiBtADAV8Sbis89yKuYBvbdNXPpu8bQfJrS20op7zoysfR8roFlzpVHyoaG2 4yb3 iv: +0siNLzFHHqEkP07k6JhYw== version: 1 id: database Authorized users see the decrypted content $ knife vault show credentials database $ knife decrypt credentials database --mode client db_password: some_password id: database
  • 61. 32 . 4 CHEF VAULT Content of database_keys $ knife data bag show credentials database_keys admins: mlopez vaubert clients: web-01 web-02 id: database_test_keys mlopez: SOME KEY vaubert: SOME KEY web-01: SOME KEY web-02: SOME KEY
  • 62. 32 . 5 CHEF VAULT Add a new admin workstation $ knife vault update credentials database -A mhue Rotate all keys $ knife vault rotate all keys
  • 63. 32 . 6 CHEF VAULT Use Vault in recipe chef_gem 'chef-vault' do compile_time true if respond_to?(:compile_time) end require 'chef-vault' # Or just include chef_vault coobkook include_recipe 'chef-vault' case ChefVault::Item.data_bag_item_type('credentials', 'database') when :normal ... when :encrypted ... when :vault ... end
  • 64. 33 . 1 OHAI Ohai is a tool that is used to detect attributes on a node, inventory the system (platform, cpu, memory...), and then provide automatically these attributes to the chef-client at the start of every chef-client run. Check Automatic attributes previously seen. You can list node attributes by running "ohai" command $ ohai { "hostname": "node-01", "machinename": "node-01", "fqdn": "node-01.my.local", "domain": "my.local", "network": { "interfaces": { "lo": { "encapsulation": "Loopback", "addresses": { "127.0.0.1": { "family": "inet", "prefixlen": "8", "netmask": "255.0.0.0", ...
  • 65. 33 . 2 OHAI - CUSTOM PLUGIN You can create your own Ohai plugin to collect data before the run and set them into Ohai as an attribute Use Ohai cookbook and put your plugin into ' les' directory › cookbooks/ohai/ les/default/plugins/haproxy.rb # Encoding: utf-8 # Get current version of Haproxy Ohai.plugin(:Haproxy) do provides 'haproxy' collect_data(:linux) do haproxy Mash.new [['dpkg-query -W haproxy | awk '{print $2}' | sed 's/(^[1-9].[1-9]).*/1/'', :installed_version]].each do |cmd, property| so = shell_out(cmd) haproxy[property] = so.stdout.delete("n") end end end
  • 66. 34 TEAM WORKING Git -> Create branches Test locally before uploading his cookbook Vagrant / Virtualbox, Kitchen CI...etc. Chef solo/zero/local Bump version of your cookbook and upload it into environment Freeze uploaded version and apply version constraint into environnements Knife Spork – Plugin allow you to bump, upload and promote a cookbook more easily. Noti cation plugin, auto git add … $ knife cookbook upload nginx [--freeze] [--force] [-E <environment>] $ knife spork omni nginx –l minor –e qa_group</environment>
  • 67. 35 . 1 TEST HIS JOB Syntax Logical tests with Foodcritic Obsolescence of resources used, invalid search queries, syntax best- practice… Unit tests with ChefSpec $ knife cookbook test nginx package 'foo' require 'chefspec' describe 'example::default' do let(:chef_run) { ChefSpec::SoloRunner.converge(described_recipe) } it 'installs foo' do expect(chef_run).to install_package('foo') end end
  • 68. 35 . 2 TEST HIS JOB Integration tests with Kitchen CI Check if a run runs without errors (converge) Include unit tests (Rspec, Bats…) Run many tests suite in one time (client / server) on many platforms Check which nodes use your cookbook Simulate an execution of a run knife preflight web::ws knife search node -i "recipes:web::ws » chef-client --why-run knife ssh ‘name:srv-01.pp’ ‘sudo chef-client –W’
  • 69. 36 DEBUG – WHY IT DOESN'T WORK ? chef-client –l debug Generate logs Chef::Log.debug(« Doesn’t work ») puts myVariable Raise Exceptions Chef::Log.fatal!('Deployment failure...') raise
  • 70. 37 . 1 ADVANCED Override a run-list (-o "recipe[]") Override a community Cookbook knife ssh 'name:srv-01.dev' 'sudo chef-client –o "recipe[firefox]"' include_recipe 'nginx' resources("template[/etc/nginx/nginx.conf]").cookbook 'myNginx')
  • 71. 37 . 2 ADVANCED Noti cations by chef-handler Knife diff (/*)
  • 72. 38 SOURCES & BOOKS Chef Starter Chef Infrastructure Automation Cookbook https://docs.chef.io http://tech.yipit.com/2013/05/09/advanced-chef-writing-heavy- weight-resource-providers-hwrp/ http://dougireton.com/blog/2013/02/03/knife-tricks http://www.pburkholder.com/blog/2015/04/23/emulating- cookbook-semver-build-numbers-with-chef-policy les/ https://yolover.poise.io/
  • 73. SOURCES & BOOKS › Chef Vault https://github.com/chef/chef-vault https://blog.chef.io/2016/01/21/chef-vault-what-is-it-and-what-can-it-do-for-you/ https://blog.chef.io/2013/09/19/managing-secrets-with-chef-vault/ http://hedge-ops.com/chef-vault-tutorial/