Hudson: Your robotic butler

Hudson
Your Robotic Butler

by @stevenmerrill

steven@treehouseagency.com

http://dgo.to/@smerrill

Saturday, July 24, 2010

Hudson

• Java-based continuous integration engine

• Used to automate any regular job

• Send notiﬁcations through a variety of channels


Why Use It?
• Automate tasks that developers should run

• Get detailed console logs for jobs without
inserting into your Drupal database

• Run jobs on any number of servers and chain
them together contingent on the success

• Notify users in a variety of media


Setting Up Hudson
Cheerio!


Getting Hudson

• Hudson is packaged as a .war ﬁle

• Run it on a standard servlet container

• Hudson labs also maintains a .deb ﬁle and an
apt repository


Setup Tips
• Run the Hudson server as a user with umask
002 and with the same primary group as your
web server

• This will make sure that permissions are correct
for ﬁles when DrupalWebTestCase::drupalGet()
or the like invoke your web server

• http://drupal.org/node/838852


Setup, cont’d
• You can also use Apache (or nginx, etc) to
proxy Hudson so that you can run it without
having to use :8080 in the URL

• Use Apache’s mod_proxy and the instructions
at http://wiki.hudson-ci.org/display/HUDSON/
Running+Hudson+behind+Apache


Conﬁguring on Ubuntu

• The simplest way to get up and running with
Hudson is with Ubuntu and their apt repository

• http://pkg.hudson-labs.org/debian/


Conﬁguring on Ubuntu, cont’d
• wget -q -O - http://pkg.hudson-labs.org/debian/
hudson-labs.org.key | sudo apt-key add -

• sudo sh -c "echo 'deb http://pkg.hudson-labs.org/
debian binary/' >> /etc/apt/sources.list"

• sudo apt-get update

• sudo apt-get install hudson

• sudo make me a sandwich (this will take awhile)


Conﬁguring on Ubuntu, cont’d
• sudo usermod -G www-data hudson

• sudo nano -w /etc/init.d/hudson

• Add --umask=002 to DAEMON_ARGS

• sudo service hudson stop

• sudo service hudson start


You now have a robotic butler at your command.


Terminology
• Every task you wish to automate using Hudson will is
called a job

• The directory on the file system where a job’s source
files are checked out it the workspace

• You create numerous build steps to execute
whenever the job is run, which is called a build

• You can take any number of files produced by the
build and save them as artifacts


Terminology, cont’d

• Each build has two status axes

• Failure or success

• Stable or unstable


This is Drizzle’s build farm at http://hudson.drizzle.org/.


Working with Jobs
Credit: http://www.ﬂickr.com/photos/techshownetwork/2943427637/


Job Conﬁguration
• Each job has a unique name

• Name your jobs like Drupal variables -
lowercase and underscores

• On the Ubuntu install, a folder will be created in
/var/lib/hudson/jobs/[job name] .


Source Control Integration
• Hudson can check source control for you and build
your project when there are changes

• Hudson will do a checkout and then grab any available
changes and start a build once it has grabbed them

• Other advantages include being able to have the
notiﬁcation plugins send out a list of changes and the
ability to create Hudson users for each person who
commits to your repository


Source Control, cont’d
• For the job where you’ll be running your unit
tests from, make sure you do not choose the
option to do a clean checkout every time.

• There are source control integration plugins for
nearly every VCS out there, including the open-
source standards cvs, svn, hg, bzr and git, and
proprietary ones such as Perforce, ClearCase
and even Team Foundation Server.


Notifications
• Hudson has built-in notifications via email

• Via user integration, it can also email users who have
broken or fixed the build

• Many plugins exist for other notification media, such
as multi-protocol IM and IRC, suitable for running in
a team IRC room

• If you enter committers IRC nicks / IM handles, the
bot will praise or chastise them


Slave Builds
• Hudson can install a Java-based slave via SSH
on any *nix machine

• The slave will run builds on the local machine
and report build status back to Hudson

• Jobs can be tied to any open slave (better for
jobs like running cron) or to a speciﬁc machine
(often used for deployments)


Build Steps
Credit: http://www.ﬂickr.com/photos/mbgrigby/3427948763


Build Steps
• The meat of any job is the build steps that it
goes through

• When used with *nix machines, these build
steps are shell scripts

• Hudson sets a number of environmental
variables when these jobs are run


Build Steps, cont’d
• $WORKSPACE - the directory with the VCS code

• $BUILD_URL - the URL to this speciﬁc build

• $BUILD_NUMBER - the integer build number

• In addition, you can create parameterized builds
where a default value is set, but users running
builds in the Hudson interface can specify a
different value on a per-build basis


Build Steps, cont’d
• When you look in the console for a build that
runs, you’ll see that it runs sh -xe and then the
scripts that you have put into the build steps

• This option will make it so that if any step in the
process returns a non-zero exit code, the build
will fail


Testing
• The most obvious use for testing is to run
Drupal SimpleTests on every checkin

• To do this, set up a job connected to your
source control and that checks out your site

• Install the site on the machine and make sure
that the SimpleTest module is enabled


Testing, cont’d
• With the SimpleTest module enabled, you can
use the scripts/run-tests.sh script to run
SimpleTests from the command line

• These tests will be run as the Hudson user
when run through Hudson, so that’s where the
umask 002 we set up earlier is important


Testing, cont’d
• The SimpleTest module in Pressﬂow 6 contains
a patch to allow the SimpleTest run-tests.sh
script to output test results in the JUnit XML
format using the --xml switch

• Hudson will show your test successes and
failure and to make a build unstable if tests fail

• http://drupal.org/node/602332


Linking Jobs
Credit: http://www.ﬂickr.com/photos/lopolis/2623060590/


Other Scheduling Options
• In addition to having a build run whenever there
are new changes on a branch in source control,
you can schedule builds:

• Using a cron-like syntax

• To run when another completes successfully

• By hand or with a GET to /job/[job name]/build


Other Scheduling, cont’d

• When you make a chain of builds, one failure in
a build in the chain will stop the whole thing

• You can also select to make a single unstable
build stop the chain


Deployment
Credit: http://www.ﬂickr.com/photos/mateus27_24-25/2760653277/


Deployment
• Hudson is a great tool for automating
deployments

• A common workﬂow when working with DVCS
systems is to have an integration branch or
repository that changes get pushed to

• When new changes appear in the integration
repo, Hudson can check it and run tests


Deployment, cont’d
• Using drush (and optionally its master/slave
support) it is quite easy to automate updating
other environments

• Provided that all important schema updates are
in code, drush can run all these automatically in
a deploy task


Deployment, cont’d
• The following script can be used as a deploy
script on a local or remote machine:

• drush -y updb

• drush -y features-update-all

• drush -y cc all


Other Cool Tricks
Credit: http://www.ﬂickr.com/photos/nowhere77/2571139625/


Run cron

• David Strauss of Four Kitchens similarly
elucidates on why Drush and Hudson rock and
how you can use them together

• http://fourkitchens.com/blog/2010/05/09/drop-
cron-use-hudson-instead


View Status in NetBeans

• NetBeans includes a Hudson plugin in core.


Track Back-End Performance
• Make a build that does some GET requests
against a production site

• Set “XDEBUG_PROFILE” in $_GET, $_POST or
$_COOKIE to get a cachegrind ﬁle

• Run the cachegrind ﬁle through xdebugtoolkit to
get a nice PDF to show where your execution
time goes and track it over time.


Yellow items are called a lot, red items take a lot of time per call.
Magenta items are both.


Track Front-End Performance
• Install ShowSlow from showslow.com

• Use xtightvnc to make a virtual display that
ShowSlow’s integration scripts can direct
Firefox at on a set schedule

• Compare your Google Page Speed and YSlow!
scores over time

Sorry, a magician doesn’t reveal his secrets, and I haven’t had enough time
to write up these instructions yet. But it works - Trust Me™.


Thanks
• The whole @hudsonci team for their work in
creating this amazing tool.

• Steven Jones for his work on JUnit XML output
for SimpleTests.

• The Pressﬂow team for including the patch in
Pressﬂow 6.

Community plumbing, baby.


Questions?


Hudson: Your robotic butler

More Related Content

What's hot (20)

Viewers also liked (14)

Similar to Hudson: Your robotic butler (20)

Recently uploaded (20)

Hudson: Your robotic butler