![Idempotence
• Property of a
declarative interface
• f(x) = x
• Eliminates brittleness
• Safe to run over and
over
package "ntp" do
action :install
end
template "/etc/ntp.conf" do
source "ntp.conf.erb"
owner "root"
group "root"
mode 0644
notifies :restart, "service[ntpd]"
end
service "ntpd" do
action [:enable,:start]
end](https://image.slidesharecdn.com/salishanintro-130430073147-phpapp01/85/Introduction-to-Chef-18-320.jpg)
![Nodes have a Run List
% knife node show hadoop-prod.example.com -r
{
"run_list": [
"role[base]",
"role[hadoop-worker]"
]
}](https://image.slidesharecdn.com/salishanintro-130430073147-phpapp01/85/Introduction-to-Chef-29-320.jpg)
![Roles
name "hadoop-worker"
description "Hadoop cluster member”
run_list(
"role[base]",
"recipe[java]",
"recipe[hadoop]",
“recipe[hadoop-config]"
)
default_attributes(
"hadoop-config" => {
"config_path" => “/etc/hadoop”
}
)](https://image.slidesharecdn.com/salishanintro-130430073147-phpapp01/85/Introduction-to-Chef-33-320.jpg)
![Chef Resources
• Have a type.
• Have a name.
• Have parameters.
• Take action to put the resource
in the declared state.
• Can send notifications to other
resources.
package "apache2" do
action :install
end
template "/etc/apache2/apache2.conf" do
source "apache2.conf.erb"
owner "root"
group "root"
mode 0644
notifies :restart, "service[apache2]"
end
service "apache2" do
supports :restart => true
action [:enable, :start]
end](https://image.slidesharecdn.com/salishanintro-130430073147-phpapp01/85/Introduction-to-Chef-35-320.jpg)
![Chef Recipes
• Resources are evaluated in the
order they appear.
package "haproxy" do
action :install
end
template "/etc/haproxy/haproxy.cfg" do
source "haproxy.cfg.erb"
owner "root"
group "root"
mode 0644
notifies :restart, "service[haproxy]"
end
service "haproxy" do
supports :restart => true
action [:enable, :start]
end](https://image.slidesharecdn.com/salishanintro-130430073147-phpapp01/85/Introduction-to-Chef-38-320.jpg)
![• Dynamic configuration
through search.
pool_members = search("node", "role:app_server")
template "/etc/haproxy/haproxy.cfg" do
source "haproxy.cfg.erb"
owner "root"
group "root"
mode 0644
variables :pool_members => pool_members
notifies :restart, "service[haproxy]"
end
Chef Recipes](https://image.slidesharecdn.com/salishanintro-130430073147-phpapp01/85/Introduction-to-Chef-41-320.jpg)
Introduction to Chef
- 1. An Extemporaneous Introduction To Chef Kevin A. Smith Director of Server Engineering
- 2. Who am I? • Director of Server Engineering @ Opscode • Software developer for 17 years • 7 years with Erlang • Alumni of DCRI, SAS, Red Hat, Basho • Erlang In Practice @ PragProg
- 3. Agenda • Infrastructure as Code • Configuration Management • Chef 101 • Chef in Large Environments http://www.flickr.com/photos/koalazymonkey/3590953001/
- 6. Enable the reconstruction of the business from nothing but a source code repository, an application data backup, and bare metal resources. Infrastructure as Code
- 8. The Old Way
- 9. Manual Configuration • Labor intensive • Error prone • Hard to reproduce
- 10. Scripting • Very brittle • Throw away, one off scripts • grep sed awk perl • curl | bash
- 11. File Distribution • NFS mounts • rdist • scp-on-a-for-loop • rsync on cron
- 12. This does not scale! for i in `cat servers.txt` ; do scp ntp.conf root@$i:/etc/ ntpd.conf ; done for i in `cat servers.txt` ; do ssh root@$i /etc/init.d/ntpd restart ; done for i in `cat servers.txt` ; do ssh root@$i chkconfig ntpd on ; done
- 15. Datacenter #1 Load Balancer App Server Database App Server Load Balancer Database Datacenter #2 Load Balancer App Server Database App Server Load Balancer Database Internet There are a lot of nodes!
- 16. A New Way
- 17. Declarative Configuration • Define policy • Say what, not how • Abstract interface to resources
- 18. Idempotence • Property of a declarative interface • f(x) = x • Eliminates brittleness • Safe to run over and over package "ntp" do action :install end template "/etc/ntp.conf" do source "ntp.conf.erb" owner "root" group "root" mode 0644 notifies :restart, "service[ntpd]" end service "ntpd" do action [:enable,:start] end
- 19. Convergence • Running an agent “converges” a system onto desired state • Fights entropy and unauthorized changes • Update function inputs to deal with changing requirements $ echo “boom” > /etc/ntp.conf $ chef-client $ grep server /etc/ntp.conf | head -n 1 us.pool.ntp.org $ ps -e | grep ntp 1799 ? 00:00:00 ntpd $ /etc/init.d/ntpd stop $ chef-client ps -e | grep ntp 1822 ? 00:00:00 ntpd
- 21. The chef-client runs on your systems.
- 22. Clients talk to a Chef server.
- 23. Client server conversations are protected with SSL and RSA signatures.
- 24. Each system running Chef is called a Managed Node.
- 26. Nodes have attributes { "kernel": { "machine": "x86_64", "name": "Darwin", "os": "Darwin", "version": "Darwin Kernel Version 10.4.0: Fri Apr 23 18:28:53 PDT 2010; root:xnu-1504.7.4~1/RELEASE_I386", "release": "10.4.0" }, "platform_version": "10.6.4", "platform": "mac_os_x", "platform_build": "10F569", "domain": "local", "os": "darwin", "current_user": "mray", "ohai_time": 1278602661.60043, "os_version": "10.4.0", "uptime": "18 days 17 hours 49 minutes 18 seconds", "ipaddress": "10.13.37.116", "hostname": "morbo", "fqdn": "morbomorbo.local", "uptime_seconds": 1619358 } Platform Kernel Hostname, etc.
- 27. Node attributes are searchable. $ knife search node ‘platform:mac_os_x’ search(:node, ‘platform:mac_os_x’)
- 28. Nodes have “to do” lists.
- 29. Nodes have a Run List % knife node show hadoop-prod.example.com -r { "run_list": [ "role[base]", "role[hadoop-worker]" ] }
- 30. Nodes can have Roles.
- 31. Aspirational Roles • webserver • database_master • monitoring • hadoop-worker
- 32. Roles have Attributes and a run list.
- 33. Roles name "hadoop-worker" description "Hadoop cluster member” run_list( "role[base]", "recipe[java]", "recipe[hadoop]", “recipe[hadoop-config]" ) default_attributes( "hadoop-config" => { "config_path" => “/etc/hadoop” } )
- 34. chef-client configures resources on managed nodes. cookbook_file template service package deploy git http_request link ruby_block log bash execute remote_file user
- 35. Chef Resources • Have a type. • Have a name. • Have parameters. • Take action to put the resource in the declared state. • Can send notifications to other resources. package "apache2" do action :install end template "/etc/apache2/apache2.conf" do source "apache2.conf.erb" owner "root" group "root" mode 0644 notifies :restart, "service[apache2]" end service "apache2" do supports :restart => true action [:enable, :start] end
- 36. package “hadoop” {yum install hadoop apt-get install hadoop pacman sync hadoop pkg_add -r hadoop Chef Providers
- 37. Recipes are collections of resources.
- 38. Chef Recipes • Resources are evaluated in the order they appear. package "haproxy" do action :install end template "/etc/haproxy/haproxy.cfg" do source "haproxy.cfg.erb" owner "root" group "root" mode 0644 notifies :restart, "service[haproxy]" end service "haproxy" do supports :restart => true action [:enable, :start] end
- 39. Chef Recipes • Recipes can include other recipes. • Included recipes are also evaluated in order. include_recipe "apache2" include_recipe "apache2::mod_rewrite" include_recipe "apache2::mod_deflate" include_recipe "apache2::mod_headers" include_recipe "apache2::mod_php5"
- 40. Chef Recipes • Extend recipes with Ruby. %w{ php5 php5-dev php5-cgi }.each do |pkg| package pkg do action :install end end
- 41. • Dynamic configuration through search. pool_members = search("node", "role:app_server") template "/etc/haproxy/haproxy.cfg" do source "haproxy.cfg.erb" owner "root" group "root" mode 0644 variables :pool_members => pool_members notifies :restart, "service[haproxy]" end Chef Recipes
- 42. Cookbooks are packages for recipes and related files.
- 43. Cookbook Metadata maintainer "Opscode, Inc." maintainer_email "cookbooks@opscode.com" license "Apache 2.0" description "Installs/Configures tomcat" long_description IO.read(File.join(File.dirname(__FILE__), 'README.md')) version "0.10.3" %w{ java jpackage }.each do |cb| depends cb end %w{ debian ubuntu centos redhat fedora }.each do |os| supports os end recipe "tomcat::default", "Installs and configures Tomcat"
- 44. Cookbooks are Source Code % git log commit d640a8c6b370134d7043991894107d806595cc35 Author: jtimberman <joshua@opscode.com> Import nagios version 1.0.0 commit c40c818498710e78cf73c7f71e722e971fa574e7 Author: jtimberman <joshua@opscode.com> installation and usage instruction docs commit 99d0efb024314de17888f6b359c14414fda7bb91 Author: jtimberman <joshua@opscode.com> Import haproxy version 1.0.1 commit c89d0975ad3f4b152426df219fee0bfb8eafb7e4 Author: jtimberman <joshua@opscode.com> add mediawiki cookbook commit 89c0545cc03b9be26f1db246c9ba4ce9d58a6700 Author: jtimberman <joshua@opscode.com> multiple environments in data bag for mediawiki
- 45. OSS & Community Oriented • Apache 2.0 License • Wiki, mailing lists, shared cookbook repos • http://community.opscode.com • Healthy ecosystem • 20k+ users • Hundreds of contributors • Community tooling: Food Critic,Test Kitchen, Berkshelf
- 47. New Server • Ground up rewrite Ruby/C Erlang • Order of magnitude more scalable • 2k nodes 20k+ nodes per server* *Depending on specific work load
- 48. High Scalability Users • Facebook • Cycle Computing • edmunds.com
- 49. Push Execution • Converge infrastructure on demand • Real-timey view of managed infrastructure • Reduces change latency • 4k nodes now, 10k soon
- 50. Network Automation • Network provisioning and configuration • VLANs, QoS, etc. • Partnered w/Arista on PoC (Fall 2012) • More coming soon!
- 51. ThankYou