Achieving Infrastructure Portability with Chef
8 likes2,422 views
The document discusses the benefits and challenges of using cloud infrastructures and emphasizes the importance of Chef for infrastructure portability. Chef allows for programmatic provisioning and configuration of various cloud resources, enabling management simplification through a code-based approach. Key features include resource collections, recipes, cookbooks, and community collaboration to support diverse environments and configurations.
![Ruby!
extra_packages = case node['platform']
when "ubuntu","debian"
%w{
ruby1.8
ruby1.8-dev
rdoc1.8
ri1.8
libopenssl-ruby
}
end
extra_packages.each do |pkg|
package pkg do
action :install
end
end](https://image.slidesharecdn.com/cloudopen2012-120830172922-phpapp01/85/Achieving-Infrastructure-Portability-with-Chef-27-320.jpg)
![Pass Results to Templates
pool_members = search("node","role:webserver”)
template "/etc/haproxy/haproxy.cfg" do
source "haproxy-app_lb.cfg.erb"
owner "root"
group "root"
mode 0644
variables :pool_members => pool_members.uniq
notifies :restart, "service[haproxy]"
end](https://image.slidesharecdn.com/cloudopen2012-120830172922-phpapp01/85/Achieving-Infrastructure-Portability-with-Chef-30-320.jpg)
![Pass Results to Templates
# Set up application listeners here.
listen application 0.0.0.0:80
balance roundrobin
<% @pool_members.each do |member| -%>
server <%= member[:hostname] %> <%= member[:ipaddress] %>:> weight 1 maxconn 1 check
<% end -%>
<% if node["haproxy"]["enable_admin"] -%>
listen admin 0.0.0.0:22002
mode http
stats uri /
<% end -%>](https://image.slidesharecdn.com/cloudopen2012-120830172922-phpapp01/85/Achieving-Infrastructure-Portability-with-Chef-31-320.jpg)
![knife bootstrap
knife bootstrap SERVER -r 'role[webserver]' -i ~/.ssh/id_rsa
• SSH to the machine given existing
credentials
• Install the Chef Client
• Register with the Chef Server
• Run the initial Run List
• Now managed with Chef!](https://image.slidesharecdn.com/cloudopen2012-120830172922-phpapp01/85/Achieving-Infrastructure-Portability-with-Chef-40-320.jpg)
![knife ec2
$ knife ec2
Available ec2 subcommands: (for details, knife SUB-COMMAND --
help)
** EC2 COMMANDS **
knife ec2 flavor list (options)
knife ec2 instance data (options)
knife ec2 server create (options)
knife ec2 server delete SERVER [SERVER] (options)
knife ec2 server list (options)
$ knife ec2 server create -S keypair -i ~/.ssh/id_rsa -x ubuntu
-I ami-4721882e -f m1.small -r 'role[webserver]'](https://image.slidesharecdn.com/cloudopen2012-120830172922-phpapp01/85/Achieving-Infrastructure-Portability-with-Chef-41-320.jpg)
![knife openstack
$ knife openstack
Available openstack subcommands: (for details, knife SUB-
COMMAND --help)
** OPENSTACK COMMANDS **
knife openstack flavor list (options)
knife openstack image list (options)
knife openstack server create (options)
knife openstack server delete SERVER [SERVER] (options)
knife openstack server list (options)
$ knife openstack server create -S keypair -i ~/.ssh/id_rsa
-x ubuntu -I 1231 -f standard.small -r 'role[webserver]'](https://image.slidesharecdn.com/cloudopen2012-120830172922-phpapp01/85/Achieving-Infrastructure-Portability-with-Chef-42-320.jpg)
Achieving Infrastructure Portability with Chef
- 1. Achieving Infrastructure Portability with Chef CloudOpen 2012 Matt Ray matt@opscode.com IRC/Twitter/GitHub: mattray www.opscode.com
- 4. Why the Cloud? • Instant infrastructure • Unlimited capacity • Autoscaling • No commitment • Immediate replacement
- 5. Cloud Differentiation • Each cloud defines themselves against Amazon • Entry into the market is easier • Not a lot of price competition • Feature parity is growing
- 6. The Dark Side of the Cloud
- 7. Why not the Cloud? • Reliability • Performance • Security • Price
- 8. Data Gravity
- 9. Know our escape plan for every infrastructure provider
- 10. Why Chef?
- 12. See Nodes Application Server Application Database
- 13. See Nodes Grow Application Server Application Databases
- 14. See Nodes Grow Application Servers Application Databases
- 15. See Nodes Grow Load Balancer Application Servers Application Databases
- 16. See Nodes Grow Load Balancers Application Servers Application Databases
- 17. See Nodes Grow Load Balancers Application Servers Application Database Cache Application Databases
- 18. Tied together with Config Load Balancers Application Servers Application Database Cache Application Databases
- 19. Infrastructure is a Snowflake Load Balancers Application Servers Application Database Cache Floating IP? Application Databases
- 20. Evolving Complexity Load Balancers ApplicationCache Application Servers NoSQL Database Cache Database Slaves Database
- 21. Complexity Grows Quickly DC2 DC1 DC3
- 22. And it Continues to Evolve That's great and all, but tell me about Chef! http://www.flickr.com/photos/16339684@N00/2681435235/
- 23. Chef is Infrastructure as Code • Programmatically provision and configure • Treat like any other code base • Reconstruct business from code repository, data backup, and bare metal resources. http://www.flickr.com/photos/louisb/4555295187/
- 24. Nodes • Chef-Client generates configurations directly on nodes from their run list • Reduce management complexity through abstraction • Store the configuration of your programs in version control http://www.flickr.com/photos/ssoosay/5126146763/
- 25. Collections of Resources • Networking • Routes • Users • Files • Groups • Directories • Tasks • Symlinks • Packages • Mounts • Software • Services • Configurations • Other Stuff http://www.flickr.com/photos/stevekeys/3123167585/
- 26. Declarative Interface to Resources • Define policy • Say what, not how • Pull not Push http://www.flickr.com/photos/bixentro/2591838509/
- 27. Ruby! extra_packages = case node['platform'] when "ubuntu","debian" %w{ ruby1.8 ruby1.8-dev rdoc1.8 ri1.8 libopenssl-ruby } end extra_packages.each do |pkg| package pkg do action :install end end
- 28. Recipes and Cookbooks • Recipes are collections of Resources • Cookbooks contain recipes, templates, files, custom resources, etc • Code re-use and modularity • Hundreds already on Community.opscode.com http://www.flickr.com/photos/shutterhacks/4474421855/
- 29. Search • Search for nodes with Roles • Find configuration data • IP addresses • Hostnames • FQDNs http://www.flickr.com/photos/kathycsus/2686772625
- 30. Pass Results to Templates pool_members = search("node","role:webserver”) template "/etc/haproxy/haproxy.cfg" do source "haproxy-app_lb.cfg.erb" owner "root" group "root" mode 0644 variables :pool_members => pool_members.uniq notifies :restart, "service[haproxy]" end
- 31. Pass Results to Templates # Set up application listeners here. listen application 0.0.0.0:80 balance roundrobin <% @pool_members.each do |member| -%> server <%= member[:hostname] %> <%= member[:ipaddress] %>:> weight 1 maxconn 1 check <% end -%> <% if node["haproxy"]["enable_admin"] -%> listen admin 0.0.0.0:22002 mode http stats uri / <% end -%>
- 32. So when this Graphite Nagios Jboss App Memcache Postgres Slaves Postgres Master
- 33. Becomes this Graphite Nagios Jboss App Memcache Postgres Slaves Postgres Master
- 34. Updates can be automatic Graphite Nagios Jboss App Memcache Postgres Slaves Postgres Master
- 35. Count the resources • Load balancer config Graphite Nagios • Nagios host ping • Nagios host ssh Jboss App • Nagios host HTTP • Nagios host app health Memcache • Graphite CPU • Graphite Memory Postgres Slaves • Graphite Disk • Graphite SNMP • Memcache firewall • 12+ resource changes for 1 node addition • Postgres firewall Postgres authZ config
- 36. Build anything • Simple internal applications • Complex external applications • Workstations • Hadoop clusters • IaaS infrastructure • PaaS infrastructure • SaaS applications • Storage systems • You name it http://www.flickr.com/photos/hyku/245010680/
- 37. And manage it simply • Automatically reconfigure everything • Linux, Windows, Unixes, BSDs • Load balancers • Metrics collection systems • Monitoring systems • Cloud migrations become trivial http://www.flickr.com/photos/helico/404640681/
- 38. knife
- 39. knife with the Chef Server • knife node • create/delete/edit • list • knife cookbook ... • knife role ... • knife environment ...
- 40. knife bootstrap knife bootstrap SERVER -r 'role[webserver]' -i ~/.ssh/id_rsa • SSH to the machine given existing credentials • Install the Chef Client • Register with the Chef Server • Run the initial Run List • Now managed with Chef!
- 41. knife ec2 $ knife ec2 Available ec2 subcommands: (for details, knife SUB-COMMAND -- help) ** EC2 COMMANDS ** knife ec2 flavor list (options) knife ec2 instance data (options) knife ec2 server create (options) knife ec2 server delete SERVER [SERVER] (options) knife ec2 server list (options) $ knife ec2 server create -S keypair -i ~/.ssh/id_rsa -x ubuntu -I ami-4721882e -f m1.small -r 'role[webserver]'
- 42. knife openstack $ knife openstack Available openstack subcommands: (for details, knife SUB- COMMAND --help) ** OPENSTACK COMMANDS ** knife openstack flavor list (options) knife openstack image list (options) knife openstack server create (options) knife openstack server delete SERVER [SERVER] (options) knife openstack server list (options) $ knife openstack server create -S keypair -i ~/.ssh/id_rsa -x ubuntu -I 1231 -f standard.small -r 'role[webserver]'
- 43. Chef for Infrastructure Portability • knife ec2 • knife cloudstack • knife rackspace • knife openstack • knife hp • knife vsphere • knife google • ... and many • knife azure others
- 44. The Chef Community • Apache License, Version 2.0 • 850+ Individual contributors • 150+ Corporate contributors • HP, Dell, Rackspace, VMware, Joyent, Calxeda, Heroku, SUSE and many more • 550+ cookbooks • http://community.opscode.com
- 45. Desktop, Virtualization, Private & Public Clouds • Vagrant • AWS • VMware • Rackspace • CloudStack • HP • Eucalyptus • Google • OpenStack • Azure • bare metal • many others
- 46. Desktop, Virtualization, Private & Public Clouds • Vagrant • AWS • VMware • Rackspace • CloudStack • HP • Eucalyptus • Google • OpenStack • Azure • bare metal • many others
- 47. Tale of the Tape • Artur Bergman, CEO at Fastly • It's All About Speed • http://youtu.be/qRnTejOMbZU • Jason Stowe, CEO at Cycle Computing • CycleCloud + Chef = 50,000-core Utility Supercomputer for Science • http://youtu.be/cEaQB6e7G0Q
- 48. Chef and Abstractions • Resources and Providers • Cookbooks (may) normalize deployment • Knife treats APIs the same • Chef strives to not be opinionated
- 49. Environments • Lock down versions of cookbooks • stable releases vs. development • Enforce attributes for deployment • Ports, addresses, etc. • Different run lists based on environment • Debugging enabled?
- 50. Environments • Use the same infrastructure code for wherever you deploy • Development, QA, Pre-Production, Prod • Role-based Access Controls to restrict the promotion of deployment code
- 51. TL;DL • Every infrastructure is a unique snowflake • Understand the costs associated with the features of your platform(s) of choice. • Chef enables Infrastructure Portability • "Data Gravity" is the primary concern
- 52. Thanks! Matt Ray matt@opscode.com IRC/Twitter/GitHub: mattray www.opscode.com