Puppet Camp Berlin 2014: Manageable puppet infrastructure
18 likes3,630 views
This document discusses setting up a manageable Puppet infrastructure. It outlines common pitfalls like unmaintainable codebases and collaboration difficulties. Quick wins to address pitfalls include moving data to Hiera, implementing code reviews, and refactoring constantly. The document recommends designing around server roles, storing all things data in Hiera, and facilitating deployment and workflow through environments and R10k. It provides examples of setting up roles, using different Hiera data types, and Hiera-related functions to retrieve and generate resources from data.
![Types of Hieradata
Arrays
# hiera/app01.autiplan.com.yaml
---
array: [ item1, item2, item3 ]
otherarray:
- item1
- item2
- item3
Note: Never use tabs in Hiera files!
25 / 44](https://image.slidesharecdn.com/manageablepuppetinfrastructure-140421185701-phpapp02/85/Puppet-Camp-Berlin-2014-Manageable-puppet-infrastructure-25-320.jpg)
![Whatdoes itdo?
Retrieves the first-found value in the
hierarchy. (top-down)
Whatto use itfor?
Basic variable-lookup.
Very easy to create exceptions!
Howto use it?
$smarthost = hiera('smarthost')
ExampleHieradata
# hiera/mail.autiplan.com.yaml
---
smarthost: smtp.myprovider.nl
# hiera/testing.yaml
---
smarthost: testsmtp.autiplan.com
# hiera/common.yaml
---
smarthost: mail.autiplan.com
hiera('key' [,default_value])
29 / 44](https://image.slidesharecdn.com/manageablepuppetinfrastructure-140421185701-phpapp02/85/Puppet-Camp-Berlin-2014-Manageable-puppet-infrastructure-29-320.jpg)
![Whatdoes itdo?
Retrieves an array or hash value
in the hierarchy, concatinates all
found results
Whatto use itfor?
Combining data from all
hierarchy levels.
Howto use it?
$users = hiera_array('users')
ExampleHieradata
# hiera/app01.autiplan.com.yaml
---
users: [ 'user1', 'user2' ]
# hiera/testing.yaml
---
users: [ 'testuser' ]
# hiera/common.yaml
---
users: [ 'user3', 'user4' ]
hiera_array('key' [,default_value]) (andhiera_hash)
30 / 44](https://image.slidesharecdn.com/manageablepuppetinfrastructure-140421185701-phpapp02/85/Puppet-Camp-Berlin-2014-Manageable-puppet-infrastructure-30-320.jpg)
![Whatdoes itdo?
Generates resources from a
HASH.
Whatto use itfor?
Generate any resource based on
data from Hiera.
Can also be used with
hiera_hash to create resources
from all levels!
Howto use it?
create_resources ('apache::vhost', hiera('vhosts', {}))
ExampleHieradata
# hiera/web01.autiplan.com.yaml
---
vhosts:
autiplan.com:
alias: www.autiplan.com
autiplan.dk:
alias: www.autiplan.dk
docroot: /var/www/html/autiplan.dk
autiplan.nl:
alias: www.autiplan.nl
cdn.autiplan.com:
port: 81
docroot: /var/www/html/cdn
create_resources('type',HASH[,default_values])
32 / 44](https://image.slidesharecdn.com/manageablepuppetinfrastructure-140421185701-phpapp02/85/Puppet-Camp-Berlin-2014-Manageable-puppet-infrastructure-32-320.jpg)
Puppet Camp Berlin 2014: Manageable puppet infrastructure
- 1. Manageable Puppet infrastructure ~April 2014 edition~ PuppetCampBerlin Ger Apeldoorn - http://puppetspecialist.nl 1 / 44
- 2. Freelance PuppetConsultant TrainerforPuppetLabs Benelux Who's this? 2 / 44
- 3. Scope Also... why this talk? 3 / 44
- 5. Pitfalls Cause & effect Pitfalls Lots of Workarounds Unmaintainable codebase Collaboration difficulties 5 / 44
- 6. Pitfalls Cause & effect Quick Wins Fix your codebase! Quick wins: Move data to Hiera Implement Code Review Use Puppet-lint in a git-hook REFACTOR CONSTANTLY 6 / 44
- 7. A Manageable Design April 2014 edition 7 / 44
- 8. Requirements Whadda we need 8 / 44
- 9. Our environment should be: Easyto Use Easyto Comprehend Easyto Update and...Safe 9 / 44
- 10. This stuff isn't exactly easy 10 / 44
- 11. But we cán make it safe and manageable 11 / 44
- 12. Requirements Easyto: Use Comprehend Update Safe Safe Useenvironments to test everything Createahugetesting environment UseGit to promoteyour code 12 / 44
- 13. Requirements Easyto: Use Comprehend Update Safe Manageable Manageable Keepaconsistent modulestructure Using roles for abstraction Facilitatecollaboration 13 / 44
- 14. Domains Server Roles All things data Deployment &Workflow 14 / 44
- 16. Software Components Puppet Enterpriseor TheForeman Hieraandhiera-eyaml (HierarchicalDatalookup) Gerrit (Codereview system) Git (what else?) GitFlow, adaptedversionforGerrit R10K (Environment deployment tool) 16 / 44
- 17. Domain#1: Server Roles 17 / 44
- 18. Alayer ofabstraction 18 / 44
- 19. How to do it? Createroles module root@puppet# puppet module generate gerapeldoorn-role Createabase-roleto cover generic settings # modules/role/manifests/base.pp: class role::base { include users include ssh include motd ... 19 / 44
- 20. How to do it? -Cont'd- Put all requiredresources intheclasses # modules/role/manifests/app.pp: class role::app { include apache include tomcat apache::virtualhost { 'default': ... Includeroleinnodedefinition # site.pp: node 'app01.autiplan.com' { include role::base include role::app } 20 / 44
- 21. Domain#2: All things Data 21 / 44
- 22. Hiera Hierarchical data lookup tool 22 / 44
- 23. ConfiguredHierarchy: #/etc/puppet/hiera.yaml: :hierarchy: - "%{::clientcert}" - "%{::environment}" - common Node app01.autiplan.com: environment: testing Hieradata # hiera/app01.autiplan.com.yaml --- examplekey: value for app01.autiplan.com # hiera/testing.yaml --- examplekey: value for nodes in testing environment # hiera/common.yaml --- examplekey: value for all nodes It's all about Hierarchy What will bein$test? $test = hiera('examplekey') 23 / 44
- 24. Types of Hieradata Regular values # hiera/app01.autiplan.com.yaml --- examplekey: value 24 / 44
- 25. Types of Hieradata Arrays # hiera/app01.autiplan.com.yaml --- array: [ item1, item2, item3 ] otherarray: - item1 - item2 - item3 Note: Never use tabs in Hiera files! 25 / 44
- 26. Types of Hieradata Hashes # hiera/app01.autiplan.com.yaml --- hash: key1: value key2: value 26 / 44
- 27. Types of Hieradata Combinations # hiera/app01.autiplan.com.yaml --- hash: key1: value key2: value key3: - arrayvalue1 - arrayvalue2 key4: subhashkey1: value subhashkey2: value 27 / 44
- 28. Hiera-relatedfunctions ...and what to use them for 28 / 44
- 29. Whatdoes itdo? Retrieves the first-found value in the hierarchy. (top-down) Whatto use itfor? Basic variable-lookup. Very easy to create exceptions! Howto use it? $smarthost = hiera('smarthost') ExampleHieradata # hiera/mail.autiplan.com.yaml --- smarthost: smtp.myprovider.nl # hiera/testing.yaml --- smarthost: testsmtp.autiplan.com # hiera/common.yaml --- smarthost: mail.autiplan.com hiera('key' [,default_value]) 29 / 44
- 30. Whatdoes itdo? Retrieves an array or hash value in the hierarchy, concatinates all found results Whatto use itfor? Combining data from all hierarchy levels. Howto use it? $users = hiera_array('users') ExampleHieradata # hiera/app01.autiplan.com.yaml --- users: [ 'user1', 'user2' ] # hiera/testing.yaml --- users: [ 'testuser' ] # hiera/common.yaml --- users: [ 'user3', 'user4' ] hiera_array('key' [,default_value]) (andhiera_hash) 30 / 44
- 31. Whatdoes itdo? Includes all classes listed in the array that is loaded from Hiera. Takes elements from ALL hierarchy levels. Whatto use itfor? Lightweight ENC. Put all classes / roles in Hiera. Howto use it? node default { hiera_include('roles') } ExampleHieradata # hiera/web01.autiplan.com.yaml --- roles: - role::web # hiera/common.yaml --- roles: - role::base hiera_include('classes') 31 / 44
- 32. Whatdoes itdo? Generates resources from a HASH. Whatto use itfor? Generate any resource based on data from Hiera. Can also be used with hiera_hash to create resources from all levels! Howto use it? create_resources ('apache::vhost', hiera('vhosts', {})) ExampleHieradata # hiera/web01.autiplan.com.yaml --- vhosts: autiplan.com: alias: www.autiplan.com autiplan.dk: alias: www.autiplan.dk docroot: /var/www/html/autiplan.dk autiplan.nl: alias: www.autiplan.nl cdn.autiplan.com: port: 81 docroot: /var/www/html/cdn create_resources('type',HASH[,default_values]) 32 / 44
- 33. Databindings Auto-loading of Hiera data for parameterized classes. 33 / 44
- 34. Whatdoes itdo? Automatically loads class parameters from Hiera. Whatto use itfor? Specify all class parameters in Hiera. Use all hierarchical benefits for class parameters. Simplify the use of parameterized classes. Howto use it? include mysql::server ExampleHieradata # hiera/web01.autiplan.com.yaml --- mysql::server::root_password: m0ars3cr3t # hiera/common.yaml --- mysql::server::root_password: t0ps3cr3t mysql::server::package_name: mysql-server mysql::server::restart: true Data bindings 34 / 44
- 35. Putting it all together Anything node-specific should be in Hiera! 35 / 44
- 36. APuppet Run:What calls what? 36 / 44
- 37. Domain#3: Deployment & Workflow 37 / 44
- 38. Environments Keeping the environmentalists happy 38 / 44
- 39. Environments What is anenvironment? Seperate modulepaths/site.pp. Common environments: development, testing, production. Nodes request a specific environment. Why? Essential to prevent mistakes. NEVER edit code in production! The workflow helps us to 'promote' our code to production. 39 / 44
- 40. Demo! 40 / 44
- 42. Final remarks Keeppublic modules as-is,wherever possible Create wrapper classes in company-module. Create fork if needed, submit pull request for fixes. Add forked module (gitrepo) to Puppetfile. Thinkahead Always try to anticipate future applications. If it feels overly complicated, yer doin it wrong. Refactor! 42 / 44
- 44. Freelance PuppetConsultant TrainerforPuppetLabs Benelux Thank you! A howto of setting up this environment (and the workflow!) is available on my blog: http://puppetspecialist.nl/mpi 44 / 44