Puppet: From 0 to 100 in 30 minutes
1 like1,565 views
The document provides a comprehensive overview of Puppet, a configuration management tool that helps manage applications, systems, and cloud resources using a declarative DSL. It covers the components of Puppet, including the puppet agent, puppet master, and how resources are defined and applied, along with concepts like roles, profiles, and environments in Puppet's setup. Additionally, it introduces the example42 control repository for modern Puppet practices and encourages customization and community involvement.
![Classes
• Puppet classes expose parameters which define
what and how are the managed resources
class postgresql::server (
$postgres_password = undef,
$package_ensure = $postgresql::params::package_ensure,
$service_ensure = $postgresql::params::service_ensure,
$service_enable = $postgresql::params::service_enable,
) inherits postgresql::params {
[ ... ]
package { 'postgresql':
ensure => $package_ensure, # [...]
}
service { 'postgresql':
ensure => $service_ensure, # [...]
enable => $service_enable, # [...]
}
}
• Class parameters can be set via Hiera, for example
using the yaml backend
postgresql::server::postgres_password: 'my_cleartext_password'
postgresql::server::package_ensure: absent
# Secrets can be encrypted using the hiera-eyaml backend. They look like:
postgresql::server::postgres_password: 'ENC[PKCS7,MIIISA ...]](https://image.slidesharecdn.com/from0to100-160704130057/85/Puppet-From-0-to-100-in-30-minutes-10-320.jpg)
Puppet: From 0 to 100 in 30 minutes
- 1. Puppet. From 0 to 100 in 30 minutes (intermediary and next steps are on you) Alessandro Franceschi @alvagante
- 3. What is Puppet? • Puppet is a Configuration Management Tool • It manages applications, systems' components, network and cloud resources. • It provides a language that describes the managed IT resources abstracting the implementation details. • It implements paradigms like Infrastructure as Code and Infrastructure as Data
- 4. Puppet components • On managed nodes are executed Puppet agent and Facter. • On the central Master runs the Puppet server, here public modules and our Puppet code and data are deployed with r10k. • Puppet generated data is stored to PuppetDB (it uses PostgreSQL for persistence ;) • Our data to configure classes and manage Infrastructure can be defined using Hiera. • It's also possible to define what classes to use in which nodes, with External Node Classifiers (ENC) like Puppet Enterprise Console or The Foreman.
- 5. A Puppet run in essence • Puppet agent use facter to collect facts about the system and sends them to the Puppet Master • Puppet Master, using nodes' facts and our Puppet code and data builds a catalog of resources to manage • The catalog is sent back to the client. The included resources are applied locally: packages are installed, files changed, services started, databases initialised... • Whatever the number of runs, at the end the system should be at the desired configuration state (idempotency) • A report of what happens on the client is sent back to the Puppet Master and typically stored on PuppetDB
- 6. Puppet language essentials • Puppet represents managed resources on the systems via a declarative Domain Specific Language (DSL) • Puppet language is written in files called manifests, which have a .pp extension • The single unit of configurations are resource types • Puppet resources are grouped in classes or defines • Classes, user defined types (defines), along with relevant configuration files, are shipped in modules • Public modules are released on Puppet Forge or GitHub
- 7. (Resource) types • Puppet types manage resources abstracting the underlying OS, elements of the system. Examples: package { 'postgresql': ensure => present, } service { 'postgresql': ensure => running, enable => true, } file { '/var/lib/pgsql/data/pg_hba.conf': ensure => present, source => puppet:///modules/postgres/pg_hba.conf', } • Common native resources: package, service, file, user, group, host, cron, exec, host, mount...
- 8. Extra resources from modules • Additional resources can be provided by dedicated modules. Examples from puppetlabs-postgresql module: postgresql::server::db { 'mydatabasename': user => 'mydatabaseuser', password => postgresql_password('mydatabaseuser', 'mypassword'), } postgresql::server::role { 'marmot': password_hash => postgresql_password('marmot', 'mypasswd'), } postgresql::server::database_grant { 'test1': privilege => 'ALL', db => 'test1', role => 'marmot', } postgresql::server::table_grant { 'my_table of test2': privilege => 'ALL', table => 'my_table', db => 'test2', role => 'marmot', } postgresql::server::pg_hba_rule { 'allow application network to access app database': description => "Open up PostgreSQL for access from 200.1.2.0/24", type => 'host', database => 'app', user => 'app', address => '200.1.2.0/24', auth_method => 'md5', }
- 9. Extra resources examples • There are modules to manage almost everything: - Specific applications (nginx, postgresql...) - Systems' features (iptables, sysctl, network...) - Network equipment (cisco, arista, cumulus... ) - Storage equipment (netapp, emc...) - Cloud resources (aws, azure, digitalocean...) - Cloud infrastructures (openstack, opennebula...) - Containers (docker, mesos, rkt...) • Each module may provide specific classes and resources to manage with Puppet DSL specific components
- 10. Classes • Puppet classes expose parameters which define what and how are the managed resources class postgresql::server ( $postgres_password = undef, $package_ensure = $postgresql::params::package_ensure, $service_ensure = $postgresql::params::service_ensure, $service_enable = $postgresql::params::service_enable, ) inherits postgresql::params { [ ... ] package { 'postgresql': ensure => $package_ensure, # [...] } service { 'postgresql': ensure => $service_ensure, # [...] enable => $service_enable, # [...] } } • Class parameters can be set via Hiera, for example using the yaml backend postgresql::server::postgres_password: 'my_cleartext_password' postgresql::server::package_ensure: absent # Secrets can be encrypted using the hiera-eyaml backend. They look like: postgresql::server::postgres_password: 'ENC[PKCS7,MIIISA ...]
- 11. Modules • Modules are distributable directories containing manifests, extensions, templates and files to manage. • They have a standard structure: mysql/ # Main module directory mysql/manifests/ # Manifests directory. Puppet code stays here. mysql/lib/ # Plugins directory. Ruby code that extends # Puppet (types, providers, facts...) is here. mysql/templates/ # ERB and EPP Templates directory mysql/files/ # Static files directory include mysql # Main mysql class is placed in: $modulepath/mysql/manifests/init.pp include mysql::server # This class is defined in: $modulepath/mysql/manifests/server.pp mysql::db { ...} # This define is defined in: $modulepath/mysql/manifests/db.pp file { '/etc/motd': content => template('motd/motd.conf.erb'), } # Template is in: $modulepath/motd/templates/motd.conf.erb • This allows some conventions:
- 12. Hiera • Hiera is a key-value lookup tool based on a configurable hierarchy • It can have different backends to store data in different places (yaml, eyaml, json, redis, mysql...) • Hiera is configured in hiera.yaml --- :backends: - yaml :hierarchy: - "hostname/%{::trusted.certname}" - "role/%{::role}-%{::env}" - "role/%{::role}" - common :yaml: :datadir: "/etc/puppetlabs/code/environments/%{environment}/hieradata"
- 13. Roles and profiles • A popular pattern to manage what resources should be manages on which nodes • A role is assigned to nodes (using different methods: custom facts, ENC...) • A role represents what a node does and can include one or more profiles • In profiles resources from modules are used to configure local systems in the desired way
- 14. Puppet environments • A Puppet environment is directory under /etc/puppetlabs/code/environments/ where are is placed our Puppet manifests, modules and Hiera data • Puppet environments can can be distributed as git control-repos. They contain: manifest directory for common manifests modules directory for modules hieradata directory for Hiera data, if used Puppetfile to configure r10k for public modules
- 15. to 100 Introducing example42 Puppet control repo
- 16. example42 control-repo • The result of several years of Puppet experience. A git repository with the content of a Puppet 4 environment featuring: • Data and code for a state of the art modern Puppet setup • Fabric integration for common operations • Puppet 4 optimised code following updated design principles • Customisable Vagrant environments to locally test code • Docker integrations to test code or build images • Classification based on Roles and Profiles pattern • In green fields: whatever is needed to start a new Puppet project from scratch • In brown fields: source of inspiration • ... and, yes, a lot more.
- 17. Setup • Download the repo from GitHub • Setup environment via a shell script ... • Or via Fabric: • Install git hooks for pre-commit syntax checks git clone https://github.com/example42/control-repo cd control-repo bin/setup.sh fab puppet.setup fab git.install_hooks
- 18. Explore • Give a look to the following files and directories: # The first manifest parsed by Puppet server manifests/site.pp # r10k Puppetfile and directory for public modules Puppetfile modules/ # Directory containing local site modules site/ # Sample Hiera configuration file and data directory hiera.yaml hieradata/ # Directories for Vagrant and Docker operations vagrant/ docker/ # Blueprint directory for new Puppet 4 modules skeleton/
- 19. Test local code with Vagrant Different Vagrant environments available: fab vagrant.all_status Single roles can be tested in relevant VMs: fab vagrant.up:vm=dev-local-log-01 fab vagrant.provision:vm=dev-local-log-01 # All Linux servers use this class of common resources: # site/profile/manifests/base/linux.pp # Common settings are in: # hieradata/common.yaml # For role "log" specific Hiera data is in # hieradata/role/log.yaml Vagrant environments can be customised: # vi vagrant/environments/$vagrant_environment/config.yaml vi vagrant/environments/puppetinfra/config.yaml
- 20. Test local code with Docker Build multiOS Docker images: fab docker.build_multios # Images are buid based on the data in: # hieradata/role/docker_multios_build.yaml Test a role on a given OS: fab docker.test_role:log,ubuntu-14.04 # Available images: ubuntu-12.04, ubuntu-14.04, ubuntu-14.06 # centos-7, debian-7, debian-8, alpine-3.3 Build an image based on a role (WIP): fab docker.build_role:log,ubuntu-14.04 Build multiple OS images based on a role (WIP): fab docker.build_role_multios:log
- 21. next steps learn Puppet and customise your control-repo
- 22. Learning Puppet Check the Official docs http://docs.puppet.com Ask and get involved in the Community https://puppet.com/community Look for modules of Puppet Forge http://forge.puppet.com Give a look to Tiny Puppet (used in example42/control-repo) http://www.tiny-puppet.com
- 23. Customise • The control-repo is just the starting point for a greenfield modern Puppet setup • Define a way to set your nodes' roles • Select the public modules to use and add them to Puppetfile • Write local profiles in site/profile/manifests • Review hiera.yaml logic and customise data in hieradata/ • Customise your Vagrant environment • Customise the skeleton to use for custom modules
- 24. Contribute • example42's control repo has a lot of room for enhancements • Provide profiles to manage applications • Send bug and features reports • Use and improve our code • Spread the word! #example42
- 25. From 0 to 100 in 30 minutes? Thank You (feel free to ask me anything) Alessandro Franceschi @alvagante