Hydra: Continuous Integration and Testing for Demanding People: The Details

Hydra: Continuous Integration and Testing for
Demanding People: The Details
Sander van der Burg
Conference Compass
July 15, 2014
Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People

Continuous integration
We want to deliver and test software rapidly
We quickly want to see the impact of changes to the source
code and its dependencies.

Hydra
Hydra: A Nix-based continuous integration server:

Hydra
Hydra: A Nix-based continuous integration server:
Generic. Supports multiple programming language
environments and component technologies.
Deployment. Build and test environments are deployed
automatically and all dependencies are ensured to be present
and correct.
Variability. Multiple versions/variants of dependencies can
safely coexist.
Multi platform support. Builds can be easily delegated to
machines with a diﬀerent operating system.
Scalability. Builds are transparently delegated to any machine
in a cluster capable of building it.

Hydra
How to use Hydra to build or test stuﬀ?

Hydra overview

Hydra overview
Components
Queue runner: Regularly checks what has changed and
what to build
Evaluator: Builds the jobs
Server: Web application making builds and test results
available
Nix: Package mananger responsible for the actual
builds and depedency management

The Nix package manager
A package manager borrowing concepts from purely functional
programming languages.
x = y ⇒ f (x) = f (y)

Nix store
Main idea: store all packages
in isolation from each other:
/nix/store/40awryfqzp46m...
-disnix-0.3
Paths contain a 160-bit
cryptographic hash of all
inputs used to build the
package:
Sources
Libraries
Compilers
Build scripts
. . .
/nix/store
40awryfqzp...-disnix-0.3
bin
disnix-env
disnix-manifest
disnix-service
kjlv4klmra...-getopt-1.1.4
bin
getopt
am13rq9ka...-dbus-glib-0.102
lib
libdbus-glib-1.so.2
94n64qy99...-glibc-2.19
lib
libc.so.6

Nix expressions
disnix.nix
{ stdenv, fetchurl, pkgconfig, dbus_glib
, libxml2, libxslt, getopt, nix, dysnomia }:
stdenv.mkDerivation {
name = "disnix-0.3";
src = fetchurl {
url = http://.../disnix-0.3.tar.bz2;
sha256 = "1jjmzdd7fac6isq5wdaqjbwwnsnzjag5s4...";
};
buildInputs = [ pkgconfig dbus_glib libxml2 libxslt
getopt nix dysnomia ];
buildCommand = ’’
tar xjf $src
./configure --prefix=$out
make; make install
’’;
}

Nix expressions
disnix.nix
{ stdenv, fetchurl, pkgconfig, dbus_glib
, libxml2, libxslt, getopt, nix, dysnomia }:
stdenv.mkDerivation {
name = "disnix-0.3";
src = fetchurl {
url = http://.../disnix-0.3.tar.bz2;
sha256 = "1jjmzdd7fac6isq5wdaqjbwwnsnzjag5s4...";
};
buildCommand = ’’
tar xjf $src
./configure --prefix=$out
make; make install
’’;
}
Environments
Expression defines a function that composes an
environment in which a build is executed
Nearly any type of build can be performed inside it,
e.g. C/C++, Java, Perl, Python...
We can also run tests inside these environments
The buildInputs parameters are used to configure all
settings to make a build find its dependencies, e.g.
setting PATH, PYTHONPATH, CLASSPATH ...
There are also function abstractions for different kinds
of packages
If no buildCommand is given, it executes the default
GNU Autotools build procedure: ./configure;
make; make install.

Nix expressions
all-packages.nix
{system ? builtins.currentSystem}:
rec {
stdenv = ... { inherit system; };
fetchurl = ...;
pkgconfig = ...;
dbus_glib = ...;
libxml2 = ...;
libxslt = ...;
getopt = ...;
nix = callPackage ../pkgs/tools/package-management/nix { };
dysnomia = import ../pkgs/tools/package-management/dysnomia {
inherit stdenv fetchurl getopt;
};
disnix = import ../pkgs/tools/package-management/disnix {
inherit stdenv fetchurl pkgconfig dbus_glib;
inherit libxml2 libxslt getopt nix dysnomia;
};
...
}

Nix expressions
all-packages.nix
{system ? builtins.currentSystem}:
rec {
stdenv = ... { inherit system; };
fetchurl = ...;
pkgconfig = ...;
dbus_glib = ...;
libxml2 = ...;
libxslt = ...;
getopt = ...;
nix = callPackage ../pkgs/tools/package-management/nix { };
dysnomia = import ../pkgs/tools/package-management/dysnomia {
inherit stdenv fetchurl getopt;
};
disnix = import ../pkgs/tools/package-management/disnix {
inherit stdenv fetchurl pkgconfig dbus_glib;
inherit libxml2 libxslt getopt nix dysnomia;
};
...
}
Environments
Composes packages by calling them with the required
function arguments.
Function invocations are lazy – they are only evaluated
if needed.
Previous expression for Disnix that deﬁnes a function
is imported here.
All dependencies are composed in the same expression
as well.

Building Nix expressions
Building a Nix package:
$ nix-build all-packages.nix -A disnix
/nix/store/40awryfqzp46mjzm1rwy9qa8vxscjhgx-disnix-0.3

Building a Nix package:
$ nix-build all-packages.nix -A disnix
/nix/store/40awryfqzp46mjzm1rwy9qa8vxscjhgx-disnix-0.3
The Nix package manager builds disnix and all its
dependencies that have not been built yet.
Hash component is derived from all build inputs used to build
the package.

Finding runtime dependencies
/nix/store
bin
disnix-env
disnix-manifest
disnix-service
bin
getopt
lib
libdbus-glib-1.so.2
94n64qy99...-glibc-2.19
lib
libc.so.6

Finding runtime dependencies
/nix/store
bin
disnix-env
disnix-manifest
disnix-service
bin
getopt
lib
libdbus-glib-1.so.2
94n64qy99...-glibc-2.19
lib
libc.so.6
Contents of
40aw...-disnix-0.3/bin/disnix-service
...
72 74 00 5f 65 6e 64 00 2f 6e 69 78 2f 73 74 6f |rt._end./nix/sto|
72 65 2f 61 6d 31 33 72 71 39 6b 61 7a 6d 31 78 |re/am13rq9kazm1x|
34 30 71 37 67 6b 70 71 77 31 35 62 37 33 32 6d |40q7gkpqw15b732m|
63 62 69 2d 64 62 75 73 2d 67 6c 69 62 2d 30 2e |cbi-dbus-glib-0.|
31 30 32 2f 6c 69 62 2f 6c 69 62 64 62 75 73 2d |102/lib/libdbus-|
67 6c 69 62 2d 31 2e 73 6f 3a 2f 6e 69 78 2f 73 |glib-1.so:/nix/s|
74 6f 72 65 2f 30 39 33 66 61 69 64 32 6d 78 69 |tore/093faid2mxi|
63 72 33 36 38 79 39 36 63 35 61 6a 6b 6c 39 6b |cr368y96c5ajkl9k|
...

During a build of package many side effects are removed:
Most environment variables are initially cleared or set to
dummy values, such as PATH.
Environment variables, such as PATH, are configured to only
contain the specified dependencies.
Nix store paths prevent packages to be implicitly found in
many cases (unlike “traditional” systems using /usr/lib,
/usr/bin or C:WINDOWSSystem32).
Timestamps are set to 1 second after the epoch
Files in the Nix store are made read-only.
Optionally, builds can be performed in a chroot()
environment, improving purity

User environments
Users can have
different sets of
installed applications.
PATH
/nix/.../profiles
current
42
/nix/store
pp56i0a01si5...-user-env
bin
firefox
disnix-env
b9w6q73mqm...-disnix-0.2
bin
disnix-env
mr8f62946...-firefox-30.0
bin
firefox

User environments
Users can have
different sets of
nix-env operations
create new user
environments in the
store.
PATH
/nix/.../profiles
current
42
/nix/store
bin
firefox
disnix-env
bin
disnix-env
mr8f62946...-firefox-30.0
bin
firefox
40awryfq...-disnix-0.3
bin
disnix-env
(nix-env -u disnix)

User environments
Users can have
different sets of
nix-env operations
create new user
environments in the
store.
PATH
/nix/.../profiles
current
42
/nix/store
bin
firefox
disnix-env
bin
disnix-env
mr8f62946...-firefox-30.0
bin
firefox
bin
disnix-env
i3d9vh6d8ip1...-user-env
bin
disnix-env
firefox
(nix-env -u disnix)

User environments
Users can have
different sets of
nix-env operations
create new user
environments in the
store.
PATH
/nix/.../profiles
current
42
43
/nix/store
bin
firefox
disnix-env
bin
disnix-env
mr8f62946...-firefox-30.0
bin
firefox
bin
disnix-env
bin
disnix-env
firefox
(nix-env -u disnix)

User environments
Users can have
different sets of
nix-env operations
create new user
environments in the
store.
We can atomically
switch between them.
PATH
/nix/.../profiles
current
42
43
/nix/store
bin
firefox
disnix-env
bin
disnix-env
mr8f62946...-firefox-30.0
bin
firefox
bin
disnix-env
bin
disnix-env
firefox
(nix-env -u disnix)

User environments
Users can have
different sets of
nix-env operations
create new user
environments in the
store.
We can atomically
These are roots of the
garbage collector.
PATH
/nix/.../profiles
current
43
/nix/store
bin
firefox
disnix-env
bin
disnix-env
mr8f62946...-firefox-30.0
bin
firefox
bin
disnix-env
bin
disnix-env
firefox
(nix-env --remove-generations old)

User environments
Users can have
different sets of
nix-env operations
create new user
environments in the
store.
We can atomically
These are roots of the
garbage collector.
PATH
/nix/.../profiles
current
43
/nix/store
mr8f62946...-firefox-30.0
bin
firefox
bin
disnix-env
bin
disnix-env
firefox
(nix-collect-garbage)

Hydra expression
release.nix
{ nixpkgs ? <nixpkgs>, systems ? [ "x86_64-linux" "x86_64-darwin" ]
, dysnomia ? { outPath = ./.; rev = 1234; } }:
let pkgs = import nixpkgs {}; in
rec {
tarball = pkgs.releaseTools.sourceTarball {
name = "dysnomia-tarball";
version = builtins.readFile ./version;
src = dysnomia;
buildInputs = [ pkgs.getopt ];
};
build = pkgs.lib.genAttrs systems (system:
let pkgs = import nixpkgs { inherit system; }; in
pkgs.releaseTools.nixBuild {
name = "dysnomia";
src = tarball;
});
...
}

Hydra expression
release.nix
, dysnomia ? { outPath = ./.; rev = 1234; } }:
rec {
tarball = pkgs.releaseTools.sourceTarball {
name = "dysnomia-tarball";
src = dysnomia;
};
let pkgs = import nixpkgs { inherit system; }; in
pkgs.releaseTools.nixBuild {
name = "dysnomia";
src = tarball;
});
...
}
Jobs
An Hydra expression is a function returing an attribute
set: rec{attr1 = value1; ...; attrn = valuen; }
Function parameters deﬁne variability points:
Locations of the Dysnomia, Nixpkgs collection Git
repositories
Target system architectures
Each attribute corresponds to a job.
Each value refers to a function performing a build or
test.
File is typically placed in the root folder of a source
package.

Building jobs from the command-line
Building a source tarball:
$ nix-build release.nix -A tarball
Building Dysnomia for 64-bit AMD Linux:
$ nix-build release.nix -A build.x86 64-linux
Building Dysnomia for Mac OS X (Nix delegates the build to
a Mac machine if the build is run on Linux and an external
machine is conﬁgured):
$ nix-build release.nix -A build.x86 64-darwin

Building jobs from Hydra
Create a project:

Create a jobset:

Create a jobset:
Jobs
All (but one) inputs are provided as function
arguments to the release expression.
One input is the package itself (dysnomia) that
contains the release.nix expression

Evaluation results (job names correspond to those deﬁned in
release.nix):

Hydra expression referring to other jobsets
release.nix
, dysnomiaJobset ?
import ../dysnomia/release.nix { inherit nixpkgs systems; }
, disnix ? { outPath = ./.; rev = 1234; } }:
rec {
tarball = ...
let dysnomia = builtins.getAttr system (dysnomiaJobset.build); in
with import nixpkgs { inherit system; };
releaseTools.nixBuild {
name = "disnix";
src = tarball;
};
...
}

Use an input of type: ’Previous Hydra evaluation’:

Hydra jobs I typically write
Source packages. Jobs that assemble tarballs, Zip ﬁles or
other archives containing the source code.
Binary packages. The actual builds for a variety of
architectures, such as i686-linux, x86 64-linux,
x86 64-darwin.
Program manuals. For example, building the manual from
Docbook.
Program documentation catalogs. Generating a
documentation catalog from the source code, e.g. using
javadoc, doxygen or JSDuck.
Unit tests. Running Unit tests, for example with JUnit or
mocha and producing a coverage report.
System integration tests. Composing NixOS Linux VMs with
all environmental dependencies, e.g. DBMS, web server etc,
and run tests inside them.

Nix channel

Nix channel
Adding a channel:
$ nix-channel --add http://localhost/jobset/disnix/disnix-master/channel/latest
$ nix-channel --update
When running:
$ nix-env -i disnix
installing ‘disnix-0.3pre174e883b7b09da822494876d2f297736f33707a7’
these paths will be fetched (0.31 MiB download, 0.91 MiB unpacked):
/nix/store/70rkq38r69fwrz90ayc4fyg823z92nmf-disnix-0.3
fetching path ‘/nix/store/70rkq38r69fwrz90ayc4fyg823z92nmf-disnix-0.3’...
The build gets downloaded from the Hydra server, instead of being
built from source code.

Nix package manager: Exercises
Check it out yourself!!!

Availability
Nix and Hydra are available as free and open source software under
the LGPLv2 and the GPLv3 licenses:
Nix: http://nixos.org/nix
Hydra: http://nixos.org/hydra
NixOS’ Hydra server: http://hydra.nixos.org
Nix can be used on any Linux distribution, NixOS, Mac OS X,
FreeBSD, and Windows (through Cygwin)
Hydra can be used on any Linux distribution

Related work
Using Nix while doing development:
Deploy development packages and composing an environment
in which they can be found
NixOS: http://nixos.org/nixos
Deploy an entire system configuration (Linux distribution) with
Nix.
System integration testing with NixOS
Efficiently compose networks of NixOS machines within a build
in which system integration tests can be performed
Disnix: http://nixos/disnix
(Re)deploy service-oriented systems into networks of machines
NixOps: http://nixos/nixops
Deploy networks of NixOS configurations to physical machines
or into the cloud
Automatically creates VM instances if needed
NiJS: https://www.npmjs.org/package/nijs
Compose Nix packages in JavaScript
Invoke JavaScript functions from Nix expressions
Very primitive stand-alone package manager

Questions

Hydra: Continuous Integration and Testing for Demanding People: The Details

More Related Content

What's hot (20)

Similar to Hydra: Continuous Integration and Testing for Demanding People: The Details (20)

More from Sander van der Burg (17)

Recently uploaded (20)

Hydra: Continuous Integration and Testing for Demanding People: The Details