Chef - Configuration Management for the Cloud
1 like1,563 views
This document discusses using Chef for configuration management in the cloud. Chef allows infrastructure to be defined as code and automatically deployed across cloud providers. It provides tools to build and manage complex, evolving infrastructures through recipes, templates, and reusable cookbooks. Chef handles provisioning, configuration, application deployment, and allows infrastructure to be easily migrated across cloud platforms.
![Ruby!
extra_packages = case node['platform']
when "ubuntu","debian"
%w{
ruby1.8
ruby1.8-dev
rdoc1.8
ri1.8
libopenssl-ruby
}
end
extra_packages.each do |pkg|
package pkg do
action :install
end
end
Tuesday, April 30, 13](https://image.slidesharecdn.com/pdxcloud-chef-130430121503-phpapp02/85/Chef-Configuration-Management-for-the-Cloud-26-320.jpg)
![Or this
search(:users, '*:*') do |u|
user u['id'] do
uid u['uid']
shell u['shell']
home "/home/#{u['id']}"
end
directory "#{home_dir}/.ssh" do
owner u['id']
group u['gid']
mode "0700"
end
template "#{home_dir}/.ssh/authorized_keys" do
source "authorized_keys.erb"
owner u['id']
group u['id']
mode "0600"
variables :ssh_keys => u['ssh_keys']
end
end
Tuesday, April 30, 13](https://image.slidesharecdn.com/pdxcloud-chef-130430121503-phpapp02/85/Chef-Configuration-Management-for-the-Cloud-27-320.jpg)
![pool_members = search("node","role:webserver”)
template "/etc/haproxy/haproxy.cfg" do
source "haproxy-app_lb.cfg.erb"
owner "root"
group "root"
mode 0644
variables :pool_members => pool_members.uniq
notifies :restart, "service[haproxy]"
end
Pass Results to Templates
Tuesday, April 30, 13](https://image.slidesharecdn.com/pdxcloud-chef-130430121503-phpapp02/85/Chef-Configuration-Management-for-the-Cloud-30-320.jpg)
![# Set up application listeners here.
listen application 0.0.0.0:80
balance roundrobin
<% @pool_members.each do |member| -%>
server <%= member[:hostname] %> <%= member[:ipaddress] %>:> weight 1 maxconn 1 check
<% end -%>
<% if node["haproxy"]["enable_admin"] -%>
listen admin 0.0.0.0:22002
mode http
stats uri /
<% end -%>
Pass Results to Templates
Tuesday, April 30, 13](https://image.slidesharecdn.com/pdxcloud-chef-130430121503-phpapp02/85/Chef-Configuration-Management-for-the-Cloud-31-320.jpg)
![Build it somewhere
#EC2
knife ec2 server create -S jamesc -i ~/.ssh/jamesc.pem -x ubuntu -G default -I ami-
a7a97dce -f m1.small -d omnibus -r 'role[base-cloud],role[tc],role[sample]'
#Rackspace
knife rackspace server create --image 110 --flavor 2 -i ~/.ssh/jamesc.pem -d omnibus -r
'role[base-cloud],role[tc],role[sample]'
#CloudStack
knife cs server create -S "small instance" -T "CentOS 5.5(64-bit) no GUI (KVM)" -i
~/.ssh/jamesc.pem -d omnibus -r 'role[base-cloud],role[tc],role[sample]'
#Ubuntu Linux
knife bootstrap test.lab -r 'role[webserver]' -i ~/.ssh/jamesc.pem -x ubuntu --sudo -d
omnibus -r 'role[base-cloud],role[tc],role[sample]'
Tuesday, April 30, 13](https://image.slidesharecdn.com/pdxcloud-chef-130430121503-phpapp02/85/Chef-Configuration-Management-for-the-Cloud-41-320.jpg)
![knife ec2
$ knife ec2
Available ec2 subcommands: (for details, knife SUB-COMMAND --
help)
** EC2 COMMANDS **
knife ec2 flavor list (options)
knife ec2 instance data (options)
knife ec2 server create (options)
knife ec2 server delete SERVER [SERVER] (options)
knife ec2 server list (options)
$ knife ec2 server create -S keypair -i ~/.ssh/id_rsa -x ubuntu
-I ami-4721882e -f m1.small -r 'role[webserver]'
Tuesday, April 30, 13](https://image.slidesharecdn.com/pdxcloud-chef-130430121503-phpapp02/85/Chef-Configuration-Management-for-the-Cloud-42-320.jpg)
![knife openstack
$ knife openstack
Available openstack subcommands: (for details, knife SUB-
COMMAND --help)
** OPENSTACK COMMANDS **
knife openstack flavor list (options)
knife openstack image list (options)
knife openstack server create (options)
knife openstack server delete SERVER [SERVER] (options)
knife openstack server list (options)
$ knife openstack server create -S keypair -i ~/.ssh/id_rsa
-x ubuntu -I 1231 -f standard.small -r 'role[webserver]'
Tuesday, April 30, 13](https://image.slidesharecdn.com/pdxcloud-chef-130430121503-phpapp02/85/Chef-Configuration-Management-for-the-Cloud-43-320.jpg)
Chef - Configuration Management for the Cloud
- 1. Configuration Management in the Cloud with Chef PDX Cloud James Casey james@opscode.com Twitter: jamesc_000 GitHub: jamesc www.opscode.com Tuesday, April 30, 13
- 2. • Instant infrastructure • Unlimited capacity • Autoscaling • No commitment • Immediate replacement Clouds are great ! Tuesday, April 30, 13
- 3. APIs are awesome! • You can provision compute resources in seconds • You can provision storage resources in seconds • You can provision network resources in seconds • That’s cool. http://www.flickr.com/photos/jdhancock/3634246981/ Tuesday, April 30, 13
- 4. The Dark Side of the Cloud Tuesday, April 30, 13
- 5. • Performance • Security • Price • Reliability Why not the Cloud? Tuesday, April 30, 13
- 6. See Node Application Server Tuesday, April 30, 13
- 7. See Nodes Application Server Application Database Tuesday, April 30, 13
- 8. See Nodes Grow Application Server Application Databases Tuesday, April 30, 13
- 9. Application Servers Application Databases See Nodes Grow Tuesday, April 30, 13
- 10. Application Servers Application Databases Load Balancer See Nodes Grow Tuesday, April 30, 13
- 11. See Nodes Grow Application Servers Application Databases Load Balancers Tuesday, April 30, 13
- 12. See Nodes Grow Application Servers Application Database Cache Load Balancers Application Databases Tuesday, April 30, 13
- 13. Tied together with Config Application Servers Application Database Cache Load Balancers Application Databases Tuesday, April 30, 13
- 14. Infrastructure is a Snowflake Application Servers Application Database Cache Load Balancers Floating IP? Application Databases Tuesday, April 30, 13
- 15. Evolving Complexity Load Balancers Application Servers NoSQL Database Slaves ApplicationCache Database Cache Database Tuesday, April 30, 13
- 16. Complexity Grows Quickly DC1 DC3 DC2 Tuesday, April 30, 13
- 17. http://www.flickr.com/photos/16339684@N00/2681435235/ And it Continues to Evolve Tuesday, April 30, 13
- 18. http://www.flickr.com/photos/16339684@N00/2681435235/ And it Continues to Evolve Ok, so I’ve got a problem. What’s the solution ? Tuesday, April 30, 13
- 19. Golden Images are not the answer • Gold is heavy • Hard to transport • Hard to mold • Easy to lose configuration detail http://www.flickr.com/photos/garysoup/2977173063/ Tuesday, April 30, 13
- 20. Configuration Management and Automated Systems Integration is the Answer http://www.flickr.com/photos/philliecasablanca/3354734116/ Tuesday, April 30, 13
- 21. • Turn code and hardware into infrastructure • From bare metal to services in production • Scale applications as needed • Conform to policy • Align to business goals Configuration Management ? Tuesday, April 30, 13
- 22. Chef - Infrastructure as Code http://www.flickr.com/photos/louisb/4555295187/ • Programmatically provision and configure • Treat like any other code base • Reconstruct business from code repository, data backup, and bare metal resources. Tuesday, April 30, 13
- 23. • Chef-Client generates configurations directly on nodes from their run list • Reduce management complexity through abstraction • Store the configuration of your programs in version control http://www.flickr.com/photos/ssoosay/5126146763/ Nodes Tuesday, April 30, 13
- 24. Collections of Resources • Networking • Files • Directories • Symlinks • Mounts • Routes • Users • Groups • Tasks • Packages • Software • Services • Configurations • Other Stuff http://www.flickr.com/photos/stevekeys/3123167585/ Tuesday, April 30, 13
- 25. Declarative Interface to Resources • Define policy • Say what, not how • Pull not Push http://www.flickr.com/photos/bixentro/2591838509/ Tuesday, April 30, 13
- 26. Ruby! extra_packages = case node['platform'] when "ubuntu","debian" %w{ ruby1.8 ruby1.8-dev rdoc1.8 ri1.8 libopenssl-ruby } end extra_packages.each do |pkg| package pkg do action :install end end Tuesday, April 30, 13
- 27. Or this search(:users, '*:*') do |u| user u['id'] do uid u['uid'] shell u['shell'] home "/home/#{u['id']}" end directory "#{home_dir}/.ssh" do owner u['id'] group u['gid'] mode "0700" end template "#{home_dir}/.ssh/authorized_keys" do source "authorized_keys.erb" owner u['id'] group u['id'] mode "0600" variables :ssh_keys => u['ssh_keys'] end end Tuesday, April 30, 13
- 28. Recipes and Cookbooks • Recipes are collections of Resources • Cookbooks contain recipes, templates, files, custom resources, etc • Code re-use and modularity • Hundreds already on Community.opscode.com http://www.flickr.com/photos/shutterhacks/4474421855/ Tuesday, April 30, 13
- 29. http://www.flickr.com/photos/kathycsus/2686772625 • IP addresses • Hostnames • FQDNs • Search for nodes with Roles • Find configuration data Search Tuesday, April 30, 13
- 30. pool_members = search("node","role:webserver”) template "/etc/haproxy/haproxy.cfg" do source "haproxy-app_lb.cfg.erb" owner "root" group "root" mode 0644 variables :pool_members => pool_members.uniq notifies :restart, "service[haproxy]" end Pass Results to Templates Tuesday, April 30, 13
- 31. # Set up application listeners here. listen application 0.0.0.0:80 balance roundrobin <% @pool_members.each do |member| -%> server <%= member[:hostname] %> <%= member[:ipaddress] %>:> weight 1 maxconn 1 check <% end -%> <% if node["haproxy"]["enable_admin"] -%> listen admin 0.0.0.0:22002 mode http stats uri / <% end -%> Pass Results to Templates Tuesday, April 30, 13
- 32. Jboss App Memcache Postgres Slaves Postgres Master So when this NagiosGraphite Tuesday, April 30, 13
- 33. Jboss App Memcache Postgres Slaves Postgres Master NagiosGraphite Becomes this Tuesday, April 30, 13
- 34. Jboss App Memcache Postgres Slaves Postgres Master NagiosGraphite Updates can be automatic Tuesday, April 30, 13
- 35. NagiosGraphite Count the resources Jboss App Memcache Postgres Slaves • Load balancer config • Nagios host ping • Nagios host ssh • Nagios host HTTP • Nagios host app health • Graphite CPU • Graphite Memory • Graphite Disk • Graphite SNMP • Memcache firewall • Postgres firewall • Postgres authZ config • 12+ resource changes for 1 node addition Tuesday, April 30, 13
- 36. http://www.flickr.com/photos/evelynishere/2798236471/ CLONING CANNOT COPE WITH THIS • Chef can. Tuesday, April 30, 13
- 37. Build anything • Simple internal applications • Complex external applications • Workstations • Hadoop clusters • IaaS infrastructure • PaaS infrastructure • SaaS applications • Storage systems • You name it http://www.flickr.com/photos/hyku/245010680/ Tuesday, April 30, 13
- 38. And manage it simply http://www.flickr.com/photos/helico/404640681/ • Automatically reconfigure everything • Linux,Windows, Unixes, BSDs • Load balancers • Metrics collection systems • Monitoring systems • Cloud migrations become trivial Tuesday, April 30, 13
- 39. knife Tuesday, April 30, 13
- 40. Upload your infrastructure knife cookbook upload apt knife cookbook upload chef-client knife cookbook upload java knife cookbook upload jpackage knife cookbook upload ntp knife cookbook upload sudo knife cookbook upload tomcat knife cookbook upload users knife cookbook upload sample knife role from file base-cloud.rb knife role from file tc.rb knife role from file sample.rb knife data bag create users knife data bag from file users jamesc.json Tuesday, April 30, 13
- 41. Build it somewhere #EC2 knife ec2 server create -S jamesc -i ~/.ssh/jamesc.pem -x ubuntu -G default -I ami- a7a97dce -f m1.small -d omnibus -r 'role[base-cloud],role[tc],role[sample]' #Rackspace knife rackspace server create --image 110 --flavor 2 -i ~/.ssh/jamesc.pem -d omnibus -r 'role[base-cloud],role[tc],role[sample]' #CloudStack knife cs server create -S "small instance" -T "CentOS 5.5(64-bit) no GUI (KVM)" -i ~/.ssh/jamesc.pem -d omnibus -r 'role[base-cloud],role[tc],role[sample]' #Ubuntu Linux knife bootstrap test.lab -r 'role[webserver]' -i ~/.ssh/jamesc.pem -x ubuntu --sudo -d omnibus -r 'role[base-cloud],role[tc],role[sample]' Tuesday, April 30, 13
- 42. knife ec2 $ knife ec2 Available ec2 subcommands: (for details, knife SUB-COMMAND -- help) ** EC2 COMMANDS ** knife ec2 flavor list (options) knife ec2 instance data (options) knife ec2 server create (options) knife ec2 server delete SERVER [SERVER] (options) knife ec2 server list (options) $ knife ec2 server create -S keypair -i ~/.ssh/id_rsa -x ubuntu -I ami-4721882e -f m1.small -r 'role[webserver]' Tuesday, April 30, 13
- 43. knife openstack $ knife openstack Available openstack subcommands: (for details, knife SUB- COMMAND --help) ** OPENSTACK COMMANDS ** knife openstack flavor list (options) knife openstack image list (options) knife openstack server create (options) knife openstack server delete SERVER [SERVER] (options) knife openstack server list (options) $ knife openstack server create -S keypair -i ~/.ssh/id_rsa -x ubuntu -I 1231 -f standard.small -r 'role[webserver]' Tuesday, April 30, 13
- 44. Chef for Infrastructure Portability • knife ec2 • knife rackspace • knife hp • knife google • knife azure • knife cloudstack • knife openstack • knife vsphere • ... and many others Tuesday, April 30, 13
- 45. The Chef Community • Apache License,Version 2.0 • 850+ Individual contributors • 150+ Corporate contributors • HP, Dell, Rackspace,VMware, Joyent, Calxeda, Heroku, SUSE and many more • 550+ cookbooks • http://community.opscode.com Tuesday, April 30, 13
- 46. Summary • Every infrastructure is a unique snowflake • You need tools to let you do what you want • You need the power to grow your infrastructure • You need the ability to change your cloud provider • Automated Configuration Management is the solution Tuesday, April 30, 13
- 48. Thanks! James Casey james@opscode.com Twitter: jamesc_000 Github: jamesc www.opscode.com Tuesday, April 30, 13