Oliver hookins puppetcamp2011
2 likes6,357 views
Oliver Hookins presents on Nokia's use of Puppet for application deployment and automation. Some key points: - Nokia uses Puppet to deploy diverse applications across environments in a consistent way and reduce errors. - The initial Puppet system led to new problems around node definitions and lack of testing. They aimed to improve with BDD, better versioning, and more testing. - Their goals included taking a more traditional software development approach, improving developer tools, and enabling easier deployments. - Current work includes improving testing with Jenkins, moving to an API approach to remove host/role specifics, and developing an ENC.
![Class Introspection
# Prepare a hash to dump out the API
h = {}
h['class'] = {}
h['define'] = {}
# Inspect the PP file
file = 'foo.pp'
Puppet[:manifest] = file
File.open(file, 'r').readlines().each do |l|
if /^(class|define)s+([^({s]+)/
tipe = $~[1] # capture the type of object we found
id = $~[2] # we want the class/define name
# Grab the arguments and class/define name
args = Puppet::Resource::Type.find(id).to_pson_data_hash['arguments']
klass = Puppet::Resource::Type.find(id).to_pson_data_hash['name']
end
end
h[tipe][klass] = args](https://image.slidesharecdn.com/oliverhookinspuppetcamp2011-110504063626-phpapp02/85/Oliver-hookins-puppetcamp2011-18-320.jpg)
![Class Introspection
# Prepare a hash to dump out the API
h = {}
h['class'] = {}
h['define'] = {}
# Inspect the PP file
file = 'foo.pp'
Puppet[:manifest] = file
File.open(file, 'r').readlines().each do |l|
if /^(class|define)s+([^({s]+)/
tipe = $~[1] # capture the type of object we found
id = $~[2] # we want the class/define name
# Grab the arguments and class/define name
args = Puppet::Resource::Type.find(id).to_pson_data_hash['arguments']
klass = Puppet::Resource::Type.find(id).to_pson_data_hash['name']
end
end
h[tipe][klass] = args](https://image.slidesharecdn.com/oliverhookinspuppetcamp2011-110504063626-phpapp02/85/Oliver-hookins-puppetcamp2011-19-320.jpg)
![Class Introspection
# Prepare a hash to dump out the API
h = {}
h['class'] = {}
h['define'] = {}
# Inspect the PP file
file = 'foo.pp'
Puppet[:manifest] = file
File.open(file, 'r').readlines().each do |l|
if /^(class|define)s+([^({s]+)/
tipe = $~[1] # capture the type of object we found
id = $~[2] # we want the class/define name
# Grab the arguments and class/define name
args = Puppet::Resource::Type.find(id).to_pson_data_hash['arguments']
klass = Puppet::Resource::Type.find(id).to_pson_data_hash['name']
end
end
h[tipe][klass] = args](https://image.slidesharecdn.com/oliverhookinspuppetcamp2011-110504063626-phpapp02/85/Oliver-hookins-puppetcamp2011-20-320.jpg)
![Class Introspection
# Prepare a hash to dump out the API
h = {}
h['class'] = {}
h['define'] = {}
# Inspect the PP file
file = 'foo.pp'
Puppet[:manifest] = file
File.open(file, 'r').readlines().each do |l|
if /^(class|define)s+([^({s]+)/
tipe = $~[1] # capture the type of object we found
id = $~[2] # we want the class/define name
# Grab the arguments and class/define name
args = Puppet::Resource::Type.find(id).to_pson_data_hash['arguments']
klass = Puppet::Resource::Type.find(id).to_pson_data_hash['name']
end
end
h[tipe][klass] = args](https://image.slidesharecdn.com/oliverhookinspuppetcamp2011-110504063626-phpapp02/85/Oliver-hookins-puppetcamp2011-21-320.jpg)
![Class Introspection
# Prepare a hash to dump out the API
h = {}
h['class'] = {}
h['define'] = {}
# Inspect the PP file
file = 'foo.pp'
Puppet[:manifest] = file
File.open(file, 'r').readlines().each do |l|
if /^(class|define)s+([^({s]+)/
tipe = $~[1] # capture the type of object we found
id = $~[2] # we want the class/define name
# Grab the arguments and class/define name
args = Puppet::Resource::Type.find(id).to_pson_data_hash['arguments']
klass = Puppet::Resource::Type.find(id).to_pson_data_hash['name']
end
end
h[tipe][klass] = args](https://image.slidesharecdn.com/oliverhookinspuppetcamp2011-110504063626-phpapp02/85/Oliver-hookins-puppetcamp2011-22-320.jpg)
![Environment Change
def get_node_previous_environment(fqdn)
factsdir = File.join(get_node_yamldir(), 'facts')
File.directory?(factsdir) or raise NoFactsDirectoryError,
factsdir
# Inspect the client machine's facts
factsfile = File.join(factsdir, "#{fqdn}.yaml")
if [nil, ''].include?fqdn or not File.exist?(factsfile)
raise(NoClientFactsError,factsfile)
end
# Create a Puppet::Node object from the stored YAML object
y = YAML.load_file(factsfile)
p = Puppet::Node::Facts.new(y.name, y.values)
p.values['environment']
end](https://image.slidesharecdn.com/oliverhookinspuppetcamp2011-110504063626-phpapp02/85/Oliver-hookins-puppetcamp2011-30-320.jpg)
![Environment Change
def get_node_previous_environment(fqdn)
factsdir = File.join(get_node_yamldir(), 'facts')
File.directory?(factsdir) or raise NoFactsDirectoryError,
factsdir
# Inspect the client machine's facts
factsfile = File.join(factsdir, "#{fqdn}.yaml")
if [nil, ''].include?fqdn or not File.exist?(factsfile)
raise(NoClientFactsError,factsfile)
end
# Create a Puppet::Node object from the stored YAML object
y = YAML.load_file(factsfile)
p = Puppet::Node::Facts.new(y.name, y.values)
p.values['environment']
end](https://image.slidesharecdn.com/oliverhookinspuppetcamp2011-110504063626-phpapp02/85/Oliver-hookins-puppetcamp2011-31-320.jpg)
![Environment Change
def get_node_previous_environment(fqdn)
factsdir = File.join(get_node_yamldir(), 'facts')
File.directory?(factsdir) or raise NoFactsDirectoryError,
factsdir
# Inspect the client machine's facts
factsfile = File.join(factsdir, "#{fqdn}.yaml")
if [nil, ''].include?fqdn or not File.exist?(factsfile)
raise(NoClientFactsError,factsfile)
end
# Create a Puppet::Node object from the stored YAML object
y = YAML.load_file(factsfile)
p = Puppet::Node::Facts.new(y.name, y.values)
p.values['environment']
end](https://image.slidesharecdn.com/oliverhookinspuppetcamp2011-110504063626-phpapp02/85/Oliver-hookins-puppetcamp2011-32-320.jpg)
![Environment Change
def get_node_previous_environment(fqdn)
factsdir = File.join(get_node_yamldir(), 'facts')
File.directory?(factsdir) or raise NoFactsDirectoryError,
factsdir
# Inspect the client machine's facts
factsfile = File.join(factsdir, "#{fqdn}.yaml")
if [nil, ''].include?fqdn or not File.exist?(factsfile)
raise(NoClientFactsError,factsfile)
end
# Create a Puppet::Node object from the stored YAML object
y = YAML.load_file(factsfile)
p = Puppet::Node::Facts.new(y.name, y.values)
p.values['environment']
end](https://image.slidesharecdn.com/oliverhookinspuppetcamp2011-110504063626-phpapp02/85/Oliver-hookins-puppetcamp2011-33-320.jpg)
![Environment Change
if node_puppetenvironment != oldenv then
Puppet.notice("Changing environment from #{oldenv} to
#{node_puppetenvironment} for node #{fqdn}")
y = {}
y['parameters'] = {}
y['environment'] = node_puppetenvironment
y['classes'] = {}
y['classes']['puppet::client'] = {}
y['classes']['puppet::client']['puppet_environment'] =
node_puppetenvironment.clone
# YAML catches duplicate references
y['classes']['puppet::client']['puppetmaster'] =
node_puppetmaster
end](https://image.slidesharecdn.com/oliverhookinspuppetcamp2011-110504063626-phpapp02/85/Oliver-hookins-puppetcamp2011-34-320.jpg)
![Environment Change
if node_puppetenvironment != oldenv then
Puppet.notice("Changing environment from #{oldenv} to
#{node_puppetenvironment} for node #{fqdn}")
y = {}
y['parameters'] = {}
y['environment'] = node_puppetenvironment
y['classes'] = {}
y['classes']['puppet::client'] = {}
y['classes']['puppet::client']['puppet_environment'] =
node_puppetenvironment.clone
# YAML catches duplicate references
y['classes']['puppet::client']['puppetmaster'] =
node_puppetmaster
end](https://image.slidesharecdn.com/oliverhookinspuppetcamp2011-110504063626-phpapp02/85/Oliver-hookins-puppetcamp2011-35-320.jpg)
![Environment Change
if node_puppetenvironment != oldenv then
Puppet.notice("Changing environment from #{oldenv} to
#{node_puppetenvironment} for node #{fqdn}")
y = {}
y['parameters'] = {}
y['environment'] = node_puppetenvironment
y['classes'] = {}
y['classes']['puppet::client'] = {}
y['classes']['puppet::client']['puppet_environment'] =
node_puppetenvironment.clone
# YAML catches duplicate references
y['classes']['puppet::client']['puppetmaster'] =
node_puppetmaster
end](https://image.slidesharecdn.com/oliverhookinspuppetcamp2011-110504063626-phpapp02/85/Oliver-hookins-puppetcamp2011-36-320.jpg)
Oliver hookins puppetcamp2011
- 1. A Puppet Approach To Application Deployment And Automation In Nokia Oliver Hookins Principal Engineer – Services & Developer Experience
- 2. Who am I? - Rockstar SysAdmin
- 3. Who am I? - Puppet User since 0.23 - Working in Nokia's Location-Based Services for 1.5 years - Previously at Anchor Systems (manages GitHub infrastructure with Puppet) - Love/Hate Relationship with Ruby
- 4. What do we do? - “Location Based Services” - Diverse collection of applications - Rapidly moving environment, competing directly with well-known giants in the services arena - Large, traditionally-organised company with heavy change control
- 5. What do we do?
- 6. What do we do?
- 7. The Problem Domain - Deployment consistent across environments - Reduce duplication of effort (and as a by-product, human errors) - Increase testing, feedback of errors, reliability.
- 9. The 1.0 Puppet System
- 10. New Problems - Node definitions - Lack of trust built-in - Code constantly evolving - Not enough testing to counter poor Puppet knowledge - Victim of our own success
- 11. BDD
- 12. Goals - Approach more like traditional software development - Better versioning of components - Testing, testing, testing! - Automation using Jenkins - Easier deployments - Better developer tools
- 13. Testing
- 14. Jenkins - CI/build tool - Split testing into stages - simple tests (lint, --parseonly, templates) – take < 2min to run - compile tests (using Faces) – should take < 10min - integration (full-deploy) - ?? - Use in production! (canary) - Eventually: continuous deployment
- 15. API Approach - Expose a well-defined interface - Remove host-, role-, DC- (etc) specifics from modules - Drive configuration “through the front door” - Remove need to look at the code (basically reverse engineering)
- 16. ENC - Write one! … or try Dashboard - Seriously, it unlocks Puppet's potential
- 17. Why not use extlookup? - Equivalent of global variables in programming, bad! - No well-defined interface to classes/defines - Requires that you code-dive to find out how to use a class/define - Well-defined API and general documentation should be sufficient - Can't easily test code in isolation
- 23. Class Introspection Now you have a YAML interface definition: class: postfix: root_email: /dev/null relayhost: inet_interfaces: localhost alias_maps: hash:/etc/aliases mynetworks: ""
- 24. Class Introspection - “Canary” testing - Interface documentation (have a look at the :doc string as well) - Generate web forms
- 25. Module Organisation “Dist” - Provides “platform” services, as well as APIs for application modules to use - Stable, defined release cycle - Should be of sufficient quality that app teams cannot resist using it - Eventually delegate maintenance to specialist teams
- 26. Module Organisation “Dist” - Pull in things from Moduleforge, or just use Moduleforge (in future)
- 27. Module Organisation “App” - Blueprints for how to deploy applications - Uses Dist APIs rather than reimplement functionality - Sets up development environments for application – still unsure of correct approach for this
- 28. Packaging
- 29. Packaging http://hunnur.com/blog/2010/10/dynamic-git-branch-puppet-environments/ - Modules under /etc/puppet/modules - Each has metadata files (version of app, version of dist) - Module path: $confdir/environments/$environment/app: $confdir/environments/$environment/dist - Spec file creates symlinks from actual app and dist to above locations
- 30. Environment Change def get_node_previous_environment(fqdn) factsdir = File.join(get_node_yamldir(), 'facts') File.directory?(factsdir) or raise NoFactsDirectoryError, factsdir # Inspect the client machine's facts factsfile = File.join(factsdir, "#{fqdn}.yaml") if [nil, ''].include?fqdn or not File.exist?(factsfile) raise(NoClientFactsError,factsfile) end # Create a Puppet::Node object from the stored YAML object y = YAML.load_file(factsfile) p = Puppet::Node::Facts.new(y.name, y.values) p.values['environment'] end
- 31. Environment Change def get_node_previous_environment(fqdn) factsdir = File.join(get_node_yamldir(), 'facts') File.directory?(factsdir) or raise NoFactsDirectoryError, factsdir # Inspect the client machine's facts factsfile = File.join(factsdir, "#{fqdn}.yaml") if [nil, ''].include?fqdn or not File.exist?(factsfile) raise(NoClientFactsError,factsfile) end # Create a Puppet::Node object from the stored YAML object y = YAML.load_file(factsfile) p = Puppet::Node::Facts.new(y.name, y.values) p.values['environment'] end
- 32. Environment Change def get_node_previous_environment(fqdn) factsdir = File.join(get_node_yamldir(), 'facts') File.directory?(factsdir) or raise NoFactsDirectoryError, factsdir # Inspect the client machine's facts factsfile = File.join(factsdir, "#{fqdn}.yaml") if [nil, ''].include?fqdn or not File.exist?(factsfile) raise(NoClientFactsError,factsfile) end # Create a Puppet::Node object from the stored YAML object y = YAML.load_file(factsfile) p = Puppet::Node::Facts.new(y.name, y.values) p.values['environment'] end
- 33. Environment Change def get_node_previous_environment(fqdn) factsdir = File.join(get_node_yamldir(), 'facts') File.directory?(factsdir) or raise NoFactsDirectoryError, factsdir # Inspect the client machine's facts factsfile = File.join(factsdir, "#{fqdn}.yaml") if [nil, ''].include?fqdn or not File.exist?(factsfile) raise(NoClientFactsError,factsfile) end # Create a Puppet::Node object from the stored YAML object y = YAML.load_file(factsfile) p = Puppet::Node::Facts.new(y.name, y.values) p.values['environment'] end
- 34. Environment Change if node_puppetenvironment != oldenv then Puppet.notice("Changing environment from #{oldenv} to #{node_puppetenvironment} for node #{fqdn}") y = {} y['parameters'] = {} y['environment'] = node_puppetenvironment y['classes'] = {} y['classes']['puppet::client'] = {} y['classes']['puppet::client']['puppet_environment'] = node_puppetenvironment.clone # YAML catches duplicate references y['classes']['puppet::client']['puppetmaster'] = node_puppetmaster end
- 35. Environment Change if node_puppetenvironment != oldenv then Puppet.notice("Changing environment from #{oldenv} to #{node_puppetenvironment} for node #{fqdn}") y = {} y['parameters'] = {} y['environment'] = node_puppetenvironment y['classes'] = {} y['classes']['puppet::client'] = {} y['classes']['puppet::client']['puppet_environment'] = node_puppetenvironment.clone # YAML catches duplicate references y['classes']['puppet::client']['puppetmaster'] = node_puppetmaster end
- 36. Environment Change if node_puppetenvironment != oldenv then Puppet.notice("Changing environment from #{oldenv} to #{node_puppetenvironment} for node #{fqdn}") y = {} y['parameters'] = {} y['environment'] = node_puppetenvironment y['classes'] = {} y['classes']['puppet::client'] = {} y['classes']['puppet::client']['puppet_environment'] = node_puppetenvironment.clone # YAML catches duplicate references y['classes']['puppet::client']['puppetmaster'] = node_puppetmaster end
- 37. Our ENC
- 38. Our ENC vars: puppet_version: 2.6.7 ntp_servers: - 192.0.2.1 - 192.0.2.2 - 192.0.2.3 repo_server: repo.ny.example.com accounts: - bofh - peon myapp_version: 3.0.0 classes: platform: puppet_version: ${puppet_version} ntp_servers: ${ntp_servers} repo_server: ${repo_server} accounts: ${accounts} myapp: version: ${myapp_version}
- 39. Future
- 40. Future - Deployment strategy not entirely worked out - Developer workflows still could be improved (some promising work in this area, e.g. cloudsmith/geppetto (IDE integration))
- 41. Thank You! oliver.hookins@nokia.com ohookins@gmail.com http://paperairoplane.net/ P.S. We're Hiring!
- 42. Attributions - Photo on slide 8 by dandeluca, available under a Creative Commons Attribution licence - Photo on slide 9 by graemenewcomb, available under a Creative Commons Attribution licence - Photo on slide 11 by seenful, available under a Creative Commons Attribution licence - Photo on slide 13 by f-oxymoron, available under a Creative Commons Attribution licence - Photo on slide 28 by oskay, available under a Creative Commons Attribution licence - Photo on slide 39 by kirrilyrobert, available under a Creative Commons Attribution licence