Cooking Perl with Chef

Cooking Perl with Chef
David Golden
@xdg

http://perlchef.com/

July 2012

Configuration
Management
(e.g. chef, puppet, cfengine, ...)

Unknown state

↓
Target state

New machine

↓
Deployed app

Infrastructure as code
automated!
repeatable!
testable!

(no manual steps, checklist, etc.)

One tool to deploy
the whole stack
(DB, caching, messaging, ...)

But wait!
Isn't that hard to do for Perl apps?

CHI
DateTime
DBI
JSON
App = perl + CPAN + your code
Moose
Plack
POE
Try::Tiny
...

CHI
DateTime
DBI
JSON
App = perl + CPAN + your code
Moose
Plack
POE
Try::Tiny
...

your application is the versioned set of all its compontents

CHI
DateTime
DBI
JSON
App
v1.0.0 = perl + CPAN + your code
Moose
Plack
POE
Try::Tiny
...


CHI
DateTime
DBI
JSON
App
v1.0.0 = Perl + CPAN + your code
v5.14.2 Moose
Plack
POE
Try::Tiny
...


0.55
0.76
1.622
2.53
App
v5.14.2 2.0603
0.9989
1.354
0.11
...


0.55
0.76
1.622
2.53
App
v5.14.2 v1.0 2.0603
0.9989
1.354
0.11
...


0.55
0.76
1.622
2.53
App
v5.14.2 v1.0
2.0603
0.9989
1.354
0.11
...

change one piece...

0.55
0.76
1.622
2.53
App
v5.16.0 v1.0
2.0603
0.9989
1.354
0.11
...

change one piece...

0.55
0.76
1.622
2.53
App
v5.16.0 v1.02.0603
0.9989
1.354
0.11
...

… and you have a new version of your application

Repeatable deployment means...


... the same Perl


... the same Perl
... the same modules


... the same Perl
... the same code


... the same Perl
... the same code
... on demand

Use one-size fits all distribution packagers like
apt, yum, etc...?

(How much do you like your system perl?!)

This problem is not
unique to Perl

Chef ❤ Ruby
(written in ruby; various ruby app stacks)

Chef ❤ Python
(virtualenv; pip; django apps)

python → virtualenv + pip
ruby → rvm + Bundler

We've built tools like these, too

Kang-min Liu (gugod)

Matt S Trout (mst)

Tatsuhiko Miyagawa
(and a big community helping them)

perlbrew – multiple perl manager

Matt S Trout (mst)

Tatsuhiko Miyagawa


local::lib – custom @INC manager

Tatsuhiko Miyagawa


local::lib – custom @INC manager

carton – versioned dependency installer

Repeatable deployment in five parts


application-specific Perl

application-specific @INC path

versioned application code

versioned module dependencies

automate the previous four


perlbrew

application-specific @INC path





perlbrew

local::lib





perlbrew

local::lib

git




perlbrew

local::lib

git

carton



perlbrew

local::lib

git

carton

@&$%!

Now...

Chef ❤ Perl
(perlbrew; local::lib; carton)

Time for a quick Chef glossary...

(see http://wiki.opscode.com/)

“Cookbook”

A collection of components to configure
a particular application

Typically includes recipes, providers,
templates, etc.

(CPAN analogy → “distribution”)

“Recipe”

Component applied that deploys an
application or service

Typically declarative, specifying desired
resources and associated configuration

“Resource”

An abstraction of something to be deployed

“Provider”

Platform-specific implementation to deliver a
resource

“Node”

A host computer managed with Chef

Often means the configuration file that
defines recipes, attributes and roles that
define the target state of a host

“Attribute”

A variable used in a recipe and/or provider
that customizes the configuration of a
resource

Attributes have defaults, but can be
customized for nodes or roles

“Role”

A collection of recipes and attributes used to
apply common configuration across multiple
nodes

Summary...

cookbooks include recipes and providers

roles, recipes and attributes get applied to nodes

recipes specify desired resources and customize
them with attributes

providers do the work of deploying resources

I wrote two Perl Chef cookbooks
for the Chef community repository
(which is like CPAN circa 1996 or so)

http://community.opscode.com/

1. perlbrew – for managing perls

2. carton – for deploying apps

Also available here: https://github.com/dagolden/perl-chef

perlbrew cookbook resources:

perlbrew_perl – install a perl

perlbrew_lib – create a local::lib

perlbrew_cpanm – install modules to perl or lib

perlbrew_run – run shell commands under a
particular perlbrew and/or lib

carton cookbook resource:

carton_app – deploy an app with carton

– start in directory with the app source
– configure for a specific perlbrew perl
– install versioned dependencies with carton
– create a runit service for the app
– start the app

Time for an example:

Deploying a “Hello World” Plack app

https://github.com/dagolden/zzz-hello-world

Steps for creating Hello World

1. Write the application

2. Use carton to create a carton.lock file with
versioned dependency info

3. Write a simple cookbook for the application

4. Check it all into git

5. Deploy the application with Chef

$ tree
.
├── Changes
├── Makefile.PL
├── app.psgi
├── carton.lock
├── cookbook
│ └── hello-world
│ ├── README.md
│ ├── attributes
│ │ └── default.rb
│ ├── metadata.rb
│ └── recipes
│ └── default.rb
└── lib
└── ZZZ
└── Hello
└── World.pm

use strict;
use warnings;
use ZZZ::Hello::World;
my $app = sub { ZZZ::Hello::World->run_psgi(@_) };

(this Plack app just invokes a simple module)

use 5.008001;
use strict;
use warnings;

package ZZZ::Hello::World;
our $VERSION = "1.0";

use Plack::Request;

sub run_psgi {
my $self = shift;
my $req = Plack::Request->new(shift);
my $res = $req->new_response(200);
$res->content_type('text/html');
$res->body(<<"HERE");
<html>
<head><title>Hello World</title></head>
<body>
<p>Hello World. It is @{[scalar localtime]}</p>
...
</body>
</html>
HERE
return $res->finalize;
}

1;
(the module just returns some dynamic HTML)

use inc::Module::Install;
name 'ZZZ-Hello-World';
version '1.0';

requires 'Plack';
requires 'Starman';

WriteAll;

(the Makefile.PL also includes deployment dependencies like Starman)

During development, carton installs
dependencies locally and creates a versioned
dependency file called carton.lock

$ carton install
# installs dependencies into a local directory
# creates carton.lock if it doesn't exist
# carton.lock is a JSON file of dependency info

During deployment, carton installs dependencies
from carton.lock and runs the app with them

$ carton install
# installs dependencies into a local directory

$ carton exec Ilib starman p 8080 app.psgi
# runs the app using carton installed deps

# carton.lock JSON
{
"modules" : {
"Class::Inspector" : {
"dist" : "Class-Inspector-1.27",
"module" : "Class::Inspector",
"pathname" :
"A/AD/ADAMK/Class-Inspector-1.27.tar.gz",
...
}.
"Data::Dump" : {
...
},
"Devel::StackTrace" : {
...
},
"Encode::Locale" : {
...
},
...
}
(carton.lock associates module names to specific versions of those module)

# perlbrew to execute with
default['hello-world']['perl_version'] = 'perl-5.16.0'

# Install directory, repo and tag
default['hello-world']['deploy_dir'] = '/opt/hello-world'
default['hello-world']['deploy_repo'] =
'https://github.com/dagolden/zzz-hello-world.git'
default['hello-world']['deploy_tag'] = 'master'

# Service user/group/port
default['hello-world']['user'] = "nobody"
default['hello-world']['group'] = "nogroup"
default['hello-world']['port'] = 8080

(attributes are variables used in the recipe; can be customized per-node during deployment)

include_recipe 'carton'

package 'git-core'

git node['hello-world']['deploy_dir'] do
repository node['hello-world']['deploy_repo']
reference node['hello-world']['deploy_tag']
notifies :restart, "carton_app[hello-world]"
end

carton_app "hello-world" do
perlbrew node['hello-world']['perl_version']
command "starman -p #{node['hello-world']['port']} app.psgi"
cwd node['hello-world']['deploy_dir']
user node['hello-world']['user']
group node['hello-world']['group']
end

action :start
end

(recipe ensures carton and git are available...)


package 'git-core'

end

end

action :start
end

(git resource specifies where application code goes...)


package 'git-core'

end

end

action :start
end

(attributes parameterize the resource statement...)


package 'git-core'

end

end

action :start
end

(carton_app resources installs deps and sets up runit service...)


package 'git-core'

end

end

action :start
end

(again, attributes parameterize the resource...)


package 'git-core'

end

end

action :start
end

(finally, the resource is idempotently started...)

These files – and the Perl Chef
cookbooks – are all you need

Enough code... let's see how to deploy it

Steps for deployment of Hello World

1. Set up a Vagrant virtual machine

2. Prepare Pantry to manage Chef Solo

3. Get Hello World cookbook and dependencies

4. Configure virtual machine for Hello World

5. Deploy

Vagrant is a tool for managing virtual machines

“Can I have a VirtualBox now, please?”

Vagrant is a tool for managing virtual machines

$ vagrant box add base
http://files.vagrantup.com/lucid32.box

$ vagrant init

$ vagrant up

Vagrant is great for testing Chef deployment

(and other things, besides)

Chef Solo is Chef without a central
configuration server

(good for demos and smaller deployments)

Chef – you push config data to Chef Server
– nodes run Chef Client to pull config
from Chef Server and execute it

Chef Solo – you push config data to nodes
– you run Chef Solo remotely

One advantage of Chef Solo...

Your config repo is canonical

(i.e. you don't have to track what you've pushed to the central server)

One dis-advantage of Chef Solo...

Manual rsync/ssh required (yuck!)

Pantry is a tool for automating Chef Solo

Pantry is a tool for automating Chef Solo

$ pantry create node server.example.com

$ pantry apply node server.example.com
--role web --recipe myapp

$ pantry sync node server.example.com

Pantry is written in Perl and available on CPAN

(Similar to pocketknife [Ruby] and littlechef [Python])

Finally, a demonstration...

Screencast available at
http://youtu.be/H93rt-KtwBE

$ vagrant init
# create config file

$ vim Vagrantfile
# edit to forward local port 8080 to
# virtual machine port 8080

$ vagrant up
# launch it

$ vagrant ssh
# check that it's up, then exit

$ pantry init
# create directories to hold Chef Solo config

$ pantry create node vagrant
    host localhost
    port 8080
    user vagrant
# create a node config file

$ sshadd ~/.vagrant.d/insecure_private_key
# allow pantry to ssh to vagrant machine

(Important tip: remove the insecure_private_key after you no longer
need it because Github chokes on it.)

Four cookbooks must be downloaded
and copied to the 'cookbooks' directory

– hello-world
– carton
– perlbrew
– runit

$ pantry apply node vagrant recipe helloworld
# apply recipe to node configuration

$ pantry apply node vagrant default
    helloworld.perl_version=perl5.14.2
# override a default attribute

$ pantry show node vagrant
# see the resulting JSON config file
{
   "helloworld" : {
      "perl_version" : "perl5.14.2"
   },
   "run_list" : [
      "recipe[helloworld]"
   ],
   "name" : "vagrant",
   "pantry_user" : "vagrant",
   "pantry_port" : "2222",
   "pantry_host" : "localhost"
}

$ pantry sync node vagrant
# ... wait for everything to deploy ...
# then load browser and test it!

Don't be afraid. Try it out. Get involved.

tutorial and screencast → http://perlchef.com

mailing list → perl-devops-subscribe@perl.org

Cooking Perl with Chef

More Related Content

What's hot (20)

Similar to Cooking Perl with Chef (20)

More from David Golden (16)

Recently uploaded (20)

Cooking Perl with Chef