Migrating Puppet 3 to 4 -- Code Changes
0 likes660 views
The document provides a summary of the most common code changes needed when migrating Puppet code from version 3 to version 4. It outlines several key differences including: 1) Numbers being treated as numbers rather than strings, requiring file modes to be quoted. 2) Only undefined and false being treated as false, not empty strings. 3) Variable names needing to start with lowercase letters. 4) Other minor changes like hyphens not being allowed in class names and ERB variables requiring @ prefixes. The document recommends using tools like puppet parser validate, puppet-lint, catalog_diff and catalog_preview to automatically check for issues when migrating code to Puppet 4. Overall,
![Don't panic.
New features are opt-in.
Puppet 3 Agents abide Puppet 4 Masters.
Okay. Where do I start?
Our official docs have enormous lists of changes. You are not affected by most of these. I'm about to tell you what matters.
Things like iteration, or specifying the type of your class parameters are opt-in. Your Puppet 3 code can keep leaving them out.
Since code is compiled on the Master, you can keep using old Agents. They may get a catalog with extra data, but they'll ignore
it.
Okay. Let's get into some of the really simple things you're going to need to look out for in your code.
Numbers are Numbers
file { '/etc/motd':
ensure => file,
owner => 'root',
group => 'root',
mode => '0644',
}
Warning: Non-string values for the file mode property are
deprecated. It must be a string, either a symbolic mode
like 'o+w,a+r' or an octal representation like '0644' or
'755'.
Error: Parameter mode failed on File[/etc/motd]: The file
mode specification must be a string, not 'Fixnum' at line
1
3
4
Puppet 3 tends to pass values around as strings, so the provider would just incidentally get '0644' passed to it.
Puppet 4 preserves more about the types of objects. One of those types, is numbers.
The most common case where this difference causes issues is with file resource modes.
Puppet 4 sees an unquoted number as an actual (octal) number.
The provider requires a string. Puppet 3 with future parser will issue a deprecation warning. Puppet 4 will fail because of the
type.
Only undef and false are actually false in Puppet 4.
Some code relies on empty strings to evaluate to false.
Try puppetlabs/stdlib str2bool( ) function.
Booleans
if ( '' ) {
notify { 'An empty string is true in Puppet 4': }
}
if ( str2bool('') == false ) {
notify { 'str2bool() always says an empty string is false': }
}
It used to be Puppet 3 would treat an empty string as false. Now only undefined or an actual boolean false are.
A common example where this causes trouble, is a custom fact that returns an empty string to indicate, generally, "not
applicable"
In their code they would simply evaluate the truth of the variable to decide if they needed to apply its value to something.](https://image.slidesharecdn.com/threetofourcodechanges-160802231032/85/Migrating-Puppet-3-to-4-Code-Changes-2-320.jpg)
![Lowercase your Variable Names
Puppet 3 allows variable names to start with capital letters.
Puppet 4 requires they start with lowercase alphabetical letter.
$Pkg_name = 'awesome-server'
package { $Pkg_name:
ensure => installed,
}
Error: Illegal variable name, The given name 'Pkg_name'
does not conform to the naming rule /^((::)?[a-z]
w*)*((::)?[a-z_]w*)$/ at /tmp/variables.pp:1:1
I don't see this one all that much, but it's out there, and will cause your catalog to fail compilation.
A Little More Esoteric
Regexps on Numbers
Class names with hyphens
Unquoted cases
Relative class declaration
ERB variables lacking @
class configure-me {
if ( $::memorysize_mb =~ /^1ddd/ ) {
notify { 'Looks like a small instance size.': }
}
case $::operatingsystem {
centos,redhat: { notify { 'I am RedHat-ish.': } }
debian: { include debian } # configure-me::debian
default: { notify { 'I do not have a hat.': } }
}
}
<%= @my_variable %>
Not so common, but worth mentioning, are a few more Puppet 4 differences. Here's a heads-up on five of them.
Passing things around internally as strings used to mean you could do a regexp on a number. No longer. This if statement would
fail.
(Though it's interesting to note that some facts that look like numbers, may be strings. For instance, version numbers like 6.5.1.)
Make sure you haven't used hyphens in class names. They are no longer allowed. This example class is no longer valid.
Also, the old behavior of an include first searching for a subclass under the current class no longer works. This is a good thing.
Not pictured here, you'll now get a warning (and likely the template generation will misbehave) if you don't have @variable.
Node inheritance no longer works.
Vestiges of Puppet 2 -- Nodes
node 'linux_base' {
include ssh
include selinux
include sudoers
}
node 'linux_exposed' inherits 'linux_base' {
include iptables
}
node 'web001.puppet.com' inherits 'linux_exposed' {
include apache
include apache::mod::php
}
It used to be a common design pattern to create node definitions that served only as templates that others would add on to.
For instance, this example is somewhat common. All Linux nodes inherit a fictitious node called linux_base.
This sometimes resulted in chains of inheritance that composed a node from an aggregation of node definitions.
This is no longer allowed. For an alternative, look at the Roles & Profiles design pattern.](https://image.slidesharecdn.com/threetofourcodechanges-160802231032/85/Migrating-Puppet-3-to-4-Code-Changes-3-320.jpg)
![Importing other manifests no longer works.
Vestiges of Puppet 2 -- Import
# site.pp
import nodes/*.pp
# environments/$environment/environment.conf
manifest = nodes
It used to be common that when site.pp got too large, you'd break it into smaller files and have site.pp import those.
Import no longer works.
However, for some time now, the "manifest" setting has been willing to take a whole directory as its setting.
And it defaults to reading the whole "manifests" directory, which on a base install just contains the site.pp file.
You likely won't need to change much, as it will recurse into subdirectories, which is where folks have be storing these already.
You can override where Puppet looks for manifests by changing an environment's "manifest" setting.
Structured facts
Typed facts
Old-style facts
System Gems
Facter 3
os => {
architecture => "x86_64",
family => "RedHat",
hardware => "x86_64",
name => "CentOS",
release => {
full => "7.2.1511",
major => "7",
minor => "2"
}
}
notify { "${::os['family']}": }
notify { "${::osfamily}": }
Facter 3, which is the one that the Puppet 4 installer will give you, is a little fancier than you might be used to.
Structured facts, for instance, are actual hashes of keys and values. To index a sub-item, use normal array indexing notation.
Facter 2 tended to represent facts as strings. Facter 3 is more aware of types. Test some agents with "stringify_facts" set to
false.
Structured facts are handy, but tons of code uses the old Facter 2 built-in facts. For the most part, 3 will return these if asked.
Note that if you have custom facts use Ruby gem/library, you'll need to make sure the new vendored Ruby has them as well.
By the way, Facter 3 is much faster than 2.
puppet parser validate
puppet-lint and plug-ins
zack/catalog_diff
puppetlabs/catalog_preview
Automatic Code Checks
It's valuable to know if Puppet 4 is going to compile your catalog at all, and also that it will compile a similar one to Puppet 3.
The most basic of test -- will it parse -- is easy to check on the command line with `puppet parser validate` .. probably in a for
loop.
One step up is to make sure the basic checklist (quoted modes, booleans) is done. There are puppet-lint plug-ins that will do
this.
Next up is to actually compile catalogs, for actual nodes in your environment. You've got two front-running choices here.
Zack's catalog_diff tool will actually talk to two masters, have them compile catalogs from nodes' last facts, and compare them.
The PuppetLabs catalog_preview tool uses a 3.8 Master, compiling catalogs in two environments where one has parser = future.
Please note. Neither of these tools actually applies a catalog to a node. So, for instance, a broken provider will not be revealed.](https://image.slidesharecdn.com/threetofourcodechanges-160802231032/85/Migrating-Puppet-3-to-4-Code-Changes-4-320.jpg)
Migrating Puppet 3 to 4 -- Code Changes
- 1. From Puppet 3 to 4: Code Changes A quick survey of the most common code issues in the field, when migrating Puppet code from version three to four Gabriel M Schuyler Professional Services, Puppet, Inc. @gabe_sky Introductions Pro Services at Puppet Assist in migrations Dislike over-planning Me You Probably in operations ... yes? Any level of experience Need a quick heads-up Hi there, I'm Gabe Schuyler. I work in Professional Services at Puppet, Inc. I travel all around the world -- training, getting folks started, consulting. Lately I've been doing Puppet 3 to 4 migrations. I'm in a hurry ... so I don't like analysis paralysis. Look ahead at what you're about to do, but don't go crazy with the what-if. Show of hands -- who identifies themselves as "operations?" Does anyone still have "devops" on their business card? For this talk, you don't need to be super-experienced. We're not going to look at anything all that complex. You need to do things now. You're not doing anything fancy -- you just want some tips before you get started. Puppet Server All-In-One Agent packaging Facter 3 Wait. Why am I upgrading? There's not a lot of point upgrading something if it doesn't provide any advantage. Here are a few advantages. The new "puppet server" for the Master vastly improves performance. Our tests say it handles 2.5x the Agents that Ruby-based can. "All in one" Agent packaging means no more vying with the system's version of components. For instance, it has Ruby 2.1. Facter 3 is much faster, and returns structured facts, rather than just strings. These are hashes, and also are more strongly typed.
- 2. Don't panic. New features are opt-in. Puppet 3 Agents abide Puppet 4 Masters. Okay. Where do I start? Our official docs have enormous lists of changes. You are not affected by most of these. I'm about to tell you what matters. Things like iteration, or specifying the type of your class parameters are opt-in. Your Puppet 3 code can keep leaving them out. Since code is compiled on the Master, you can keep using old Agents. They may get a catalog with extra data, but they'll ignore it. Okay. Let's get into some of the really simple things you're going to need to look out for in your code. Numbers are Numbers file { '/etc/motd': ensure => file, owner => 'root', group => 'root', mode => '0644', } Warning: Non-string values for the file mode property are deprecated. It must be a string, either a symbolic mode like 'o+w,a+r' or an octal representation like '0644' or '755'. Error: Parameter mode failed on File[/etc/motd]: The file mode specification must be a string, not 'Fixnum' at line 1 3 4 Puppet 3 tends to pass values around as strings, so the provider would just incidentally get '0644' passed to it. Puppet 4 preserves more about the types of objects. One of those types, is numbers. The most common case where this difference causes issues is with file resource modes. Puppet 4 sees an unquoted number as an actual (octal) number. The provider requires a string. Puppet 3 with future parser will issue a deprecation warning. Puppet 4 will fail because of the type. Only undef and false are actually false in Puppet 4. Some code relies on empty strings to evaluate to false. Try puppetlabs/stdlib str2bool( ) function. Booleans if ( '' ) { notify { 'An empty string is true in Puppet 4': } } if ( str2bool('') == false ) { notify { 'str2bool() always says an empty string is false': } } It used to be Puppet 3 would treat an empty string as false. Now only undefined or an actual boolean false are. A common example where this causes trouble, is a custom fact that returns an empty string to indicate, generally, "not applicable" In their code they would simply evaluate the truth of the variable to decide if they needed to apply its value to something.
- 3. Lowercase your Variable Names Puppet 3 allows variable names to start with capital letters. Puppet 4 requires they start with lowercase alphabetical letter. $Pkg_name = 'awesome-server' package { $Pkg_name: ensure => installed, } Error: Illegal variable name, The given name 'Pkg_name' does not conform to the naming rule /^((::)?[a-z] w*)*((::)?[a-z_]w*)$/ at /tmp/variables.pp:1:1 I don't see this one all that much, but it's out there, and will cause your catalog to fail compilation. A Little More Esoteric Regexps on Numbers Class names with hyphens Unquoted cases Relative class declaration ERB variables lacking @ class configure-me { if ( $::memorysize_mb =~ /^1ddd/ ) { notify { 'Looks like a small instance size.': } } case $::operatingsystem { centos,redhat: { notify { 'I am RedHat-ish.': } } debian: { include debian } # configure-me::debian default: { notify { 'I do not have a hat.': } } } } <%= @my_variable %> Not so common, but worth mentioning, are a few more Puppet 4 differences. Here's a heads-up on five of them. Passing things around internally as strings used to mean you could do a regexp on a number. No longer. This if statement would fail. (Though it's interesting to note that some facts that look like numbers, may be strings. For instance, version numbers like 6.5.1.) Make sure you haven't used hyphens in class names. They are no longer allowed. This example class is no longer valid. Also, the old behavior of an include first searching for a subclass under the current class no longer works. This is a good thing. Not pictured here, you'll now get a warning (and likely the template generation will misbehave) if you don't have @variable. Node inheritance no longer works. Vestiges of Puppet 2 -- Nodes node 'linux_base' { include ssh include selinux include sudoers } node 'linux_exposed' inherits 'linux_base' { include iptables } node 'web001.puppet.com' inherits 'linux_exposed' { include apache include apache::mod::php } It used to be a common design pattern to create node definitions that served only as templates that others would add on to. For instance, this example is somewhat common. All Linux nodes inherit a fictitious node called linux_base. This sometimes resulted in chains of inheritance that composed a node from an aggregation of node definitions. This is no longer allowed. For an alternative, look at the Roles & Profiles design pattern.
- 4. Importing other manifests no longer works. Vestiges of Puppet 2 -- Import # site.pp import nodes/*.pp # environments/$environment/environment.conf manifest = nodes It used to be common that when site.pp got too large, you'd break it into smaller files and have site.pp import those. Import no longer works. However, for some time now, the "manifest" setting has been willing to take a whole directory as its setting. And it defaults to reading the whole "manifests" directory, which on a base install just contains the site.pp file. You likely won't need to change much, as it will recurse into subdirectories, which is where folks have be storing these already. You can override where Puppet looks for manifests by changing an environment's "manifest" setting. Structured facts Typed facts Old-style facts System Gems Facter 3 os => { architecture => "x86_64", family => "RedHat", hardware => "x86_64", name => "CentOS", release => { full => "7.2.1511", major => "7", minor => "2" } } notify { "${::os['family']}": } notify { "${::osfamily}": } Facter 3, which is the one that the Puppet 4 installer will give you, is a little fancier than you might be used to. Structured facts, for instance, are actual hashes of keys and values. To index a sub-item, use normal array indexing notation. Facter 2 tended to represent facts as strings. Facter 3 is more aware of types. Test some agents with "stringify_facts" set to false. Structured facts are handy, but tons of code uses the old Facter 2 built-in facts. For the most part, 3 will return these if asked. Note that if you have custom facts use Ruby gem/library, you'll need to make sure the new vendored Ruby has them as well. By the way, Facter 3 is much faster than 2. puppet parser validate puppet-lint and plug-ins zack/catalog_diff puppetlabs/catalog_preview Automatic Code Checks It's valuable to know if Puppet 4 is going to compile your catalog at all, and also that it will compile a similar one to Puppet 3. The most basic of test -- will it parse -- is easy to check on the command line with `puppet parser validate` .. probably in a for loop. One step up is to make sure the basic checklist (quoted modes, booleans) is done. There are puppet-lint plug-ins that will do this. Next up is to actually compile catalogs, for actual nodes in your environment. You've got two front-running choices here. Zack's catalog_diff tool will actually talk to two masters, have them compile catalogs from nodes' last facts, and compare them. The PuppetLabs catalog_preview tool uses a 3.8 Master, compiling catalogs in two environments where one has parser = future. Please note. Neither of these tools actually applies a catalog to a node. So, for instance, a broken provider will not be revealed.
- 5. And that's all I've got. Despite enormous lists of "breaking changes" my field experience says only a few of them usually crop up in the real world. In just a few afternoons, you can get a pretty thorough impression of what it's going to take to migrate your code. And I think you'll be surprised by how easy it's going to be. I appreciate your attention, and now I welcome your questions.