HYDRA: TOP-NOTCH CONTINUOUS INTEGRATION FOR DEMANDING PEOPLE Ludovic Courtès Service d expérimentation & développement Inria Bordeaux Sud-Ouest 3 November 2011
introduction features practice closing L. Courtès Hydra: continuous integration for demanding people 2
scenario 1: a peaceful development team... L. Courtès Hydra: continuous integration for demanding people 3
scenario 1:... software that works... L. Courtès Hydra: continuous integration for demanding people 4
scenario 1:... and suddenly... L. Courtès Hydra: continuous integration for demanding people 5
scenario 1: who broke it?! L. Courtès Hydra: continuous integration for demanding people 6
scenario 2: works for me user: It s broken. L. Courtès Hydra: continuous integration for demanding people 7
scenario 2: works for me user: It s broken. dev: Weird, it works for me. L. Courtès Hydra: continuous integration for demanding people 8
scenario 2: works for me user: It s broken. dev: Weird, it works for me. Which platform? L. Courtès Hydra: continuous integration for demanding people 9
scenario 2: works for me user: It s broken. dev: Weird, it works for me. Which platform? user: Darwin 10.4. L. Courtès Hydra: continuous integration for demanding people 10
scenario 2: works for me user: It s broken. dev: Weird, it works for me. Which platform? user: Darwin 10.4. On x86 64. L. Courtès Hydra: continuous integration for demanding people 11
scenario 2: works for me user: It s broken. dev: Weird, it works for me. Which platform? user: Darwin 10.4. On x86 64. With Apple GCC 4.0.1, GMP 4.2.4, MPFR 2.4.2, BDB 4.2. L. Courtès Hydra: continuous integration for demanding people 12
scenario 2: works for me user: It s broken. dev: Weird, it works for me. Which platform? user: Darwin 10.4. On x86 64. With Apple GCC 4.0.1, GMP 4.2.4, MPFR 2.4.2, BDB 4.2. Et avec --disable-color. L. Courtès Hydra: continuous integration for demanding people 13
scenario 2: works for me user: It s broken. dev: Weird, it works for me. Which platform? user: Darwin 10.4. On x86 64. With Apple GCC 4.0.1, GMP 4.2.4, MPFR 2.4.2, BDB 4.2. Et avec --disable-color. dev: Well, too bad. I rather test on GNU/Linux, i686 L. Courtès Hydra: continuous integration for demanding people 14
scenario 2: works for me user: It s broken. dev: Weird, it works for me. Which platform? user: Darwin 10.4. On x86 64. With Apple GCC 4.0.1, GMP 4.2.4, MPFR 2.4.2, BDB 4.2. Et avec --disable-color. dev: Well, too bad. I rather test on GNU/Linux, i686, with GCC 4.6.2, GMP 4.2.1, MPFR 2.4.1, GDBM 1.8.2, GTK+ 2.12. Oh, and with --enable-color --enable-gui. L. Courtès Hydra: continuous integration for demanding people 15
scenario 2: works for me user: It s broken. dev: Weird, it works for me. Which platform? user: Darwin 10.4. On x86 64. With Apple GCC 4.0.1, GMP 4.2.4, MPFR 2.4.2, BDB 4.2. Et avec --disable-color. dev: Well, too bad. I rather test on GNU/Linux, i686, with GCC 4.6.2, GMP 4.2.1, MPFR 2.4.1, GDBM 1.8.2, GTK+ 2.12. Oh, and with --enable-color --enable-gui. L. Courtès Hydra: continuous integration for demanding people 16
scenario 2: works for me user: It s broken. dev: Weird, it works for me. Which platform? user: Darwin 10.4. On x86 64. With Apple GCC 4.0.1, GMP 4.2.4, MPFR 2.4.2, BDB 4.2. Et avec --disable-color. dev: Well, too bad. I rather test on GNU/Linux, i686, with GCC 4.6.2, GMP 4.2.1, MPFR 2.4.1, GDBM 1.8.2, GTK+ 2.12. Oh, and with --enable-color --enable-gui. L. Courtès Hydra: continuous integration for demanding people 17
scenario 2: works for me user: It s broken. dev: Weird, it works for me. Which platform? user: Darwin 10.4. On x86 64. With Apple GCC 4.0.1, GMP 4.2.4, MPFR 2.4.2, BDB 4.2. Et avec --disable-color. dev: Well, too bad. I rather test on GNU/Linux, i686, with GCC 4.6.2, GMP 4.2.1, MPFR 2.4.1, GDBM 1.8.2, GTK+ 2.12. Oh, and with --enable-color --enable-gui. L. Courtès Hydra: continuous integration for demanding people 18
scenario 2: works for me user: It s broken. dev: Weird, it works for me. Which platform? user: Darwin 10.4. On x86 64. With Apple GCC 4.0.1, GMP 4.2.4, MPFR 2.4.2, BDB 4.2. Et avec --disable-color. dev: Well, too bad. I rather test on GNU/Linux, i686, with GCC 4.6.2, GMP 4.2.1, MPFR 2.4.1, GDBM 1.8.2, GTK+ 2.12. Oh, and with --enable-color --enable-gui. L. Courtès Hydra: continuous integration for demanding people 19
scenario 2: works for me user: It s broken. dev: Weird, it works for me. Which platform? user: Darwin 10.4. On x86 64. With Apple GCC 4.0.1, GMP 4.2.4, MPFR 2.4.2, BDB 4.2. Et avec --disable-color. dev: Well, too bad. I rather test on GNU/Linux, i686, with GCC 4.6.2, GMP 4.2.1, MPFR 2.4.1, GDBM 1.8.2, GTK+ 2.12. Oh, and with --enable-color --enable-gui. L. Courtès Hydra: continuous integration for demanding people 20
why continuous integration is so great 1. continuous compilation/test L. Courtès Hydra: continuous integration for demanding people 21
why continuous integration is so great 1. continuous compilation/test 2. quick feedback L. Courtès Hydra: continuous integration for demanding people 22
why continuous integration is so great 1. continuous compilation/test 2. quick feedback 3. pristine build environment L. Courtès Hydra: continuous integration for demanding people 23
why continuous integration is so great 1. continuous compilation/test 2. quick feedback 3. pristine build environment 4. different configurations under test L. Courtès Hydra: continuous integration for demanding people 24
why continuous integration is so great 1. continuous compilation/test 2. quick feedback 3. pristine build environment 4. different configurations under test 5. build results published L. Courtès Hydra: continuous integration for demanding people 25
the true cost of doing it yourself while true do rm -rf foo svn co svn+ssh://scm.gforge.inria.fr/svn/foo/trunk foo cd foo make sleep 300 done L. Courtès Hydra: continuous integration for demanding people 26
the true cost of doing it yourself while true do rm -rf foo svn co svn+ssh://scm.gforge.inria.fr/svn/foo/trunk foo cd foo make 2>&1 > build-log \ mail michel.cosnard@inria.fr -s "Build failed!" \ < build-log sleep 300 done L. Courtès Hydra: continuous integration for demanding people 27
the true cost of doing it yourself function svn revision { ( cd "$1" svn info \ grep \^R.vision *: *\([0-9]\+\)$ \ sed -e s/[^0-9]//g ) } function svn update { ( prev rev="$(svn revision $1)" ( cd "$1" ; svn update ) new rev="$(svn revision $1)" test $prev rev -ne $new rev ) } function update html report { local stamp="$(date -u +%Y%m%d-%H%M)" local test log="$builddir/testsuite/log" local failure rate="(tests not run)" gzip --best < "$2" > "build-$stamp.log.gz" scp "build-$stamp.log.gz" "$ssh remote dir/" if [ -f "$test log" ] then i failure rate="$(test failure rate $test log)" $html gzip --best. scp "test-$stamp.log.gz" "$ssh remote dir/" wq fi EOF local result bgcolor if [ $1 -ne 0 ] then } result="<b>fail</b>" bgcolor="lightred" else result="pass" bgcolor="lightgreen" fi local rev="$(svn revision $srcdir)" local html="<tr bgcolor= $bgcolor > <td><a href= $(gforge revision url foo $rev) >$rev< <td>$stamp</td> <td>$failure rate</td> <td> $result [ <a href= build-$stamp.log.gz >build</a> <a href=ẗest-$stamp.log.gz >test</a> ] </td> </tr>" ed "$html report" <<EOF /INSERT BUILD RESULTS +1 < "$builddir/testsuite/log" scp "$html report" "$ssh remote dir/index.html" L. Courtès Hydra: continuous integration for demanding people 28
the true cost of doing it yourself function svn revision { ( cd "$1" svn info \ grep \^R.vision *: *\([0-9]\+\)$ \ sed -e s/[^0-9]//g ) } function svn update { ( prev rev="$(svn revision $1)" ( cd "$1" ; svn update ) new rev="$(svn revision $1)" test $prev rev -ne $new rev ) } function update html report { local stamp="$(date -u +%Y%m%d-%H%M)" local test log="$builddir/testsuite/log" local failure rate="(tests not run)" gzip --best < "$2" > "build-$stamp.log.gz" scp "build-$stamp.log.gz" "$ssh remote dir/" if [ -f "$test log" ] then i failure rate="$(test failure rate $test log)" $html gzip --best. scp "test-$stamp.log.gz" "$ssh remote dir/" wq fi EOF local result bgcolor if [ $1 -ne 0 ] then } result="<b>fail</b>" bgcolor="lightred" else result="pass" bgcolor="lightgreen" fi local rev="$(svn revision $srcdir)" local html="<tr bgcolor= $bgcolor > <td><a href= $(gforge revision url foo $rev) >$rev< <td>$stamp</td> <td>$failure rate</td> <td> $result [ <a href= build-$stamp.log.gz >build</a> <a href=ẗest-$stamp.log.gz >test</a> ] </td> </tr>" ed "$html report" <<EOF /INSERT BUILD RESULTS +1 < "$builddir/testsuite/log" scp "$html report" "$ssh remote dir/index.html" L. Courtès Hydra: continuous integration for demanding people 29
lots of tools! BuildBot Jenkins (formerly Hudson) CruiseControl... Hydra L. Courtès Hydra: continuous integration for demanding people 30
suitability of a continuous integration tool languages & build systems? build notification & visualization? portability? reproducibility? scalability? L. Courtès Hydra: continuous integration for demanding people 31
introduction features practice closing L. Courtès Hydra: continuous integration for demanding people 32
testimonial Thanks to Hydra, my software projects shine more than ever! Ludovic C. L. Courtès Hydra: continuous integration for demanding people 33
Hydra, Nix, & you http://nixos.org/hydra/ Nix build tool focus on controlling the build environment Hydra continuous integration tool based on Nix L. Courtès Hydra: continuous integration for demanding people 34
Hydra, Nix, & you http://nixos.org/hydra/ Nix build tool focus on controlling the build environment Hydra continuous integration tool based on Nix web interface + email notification GNU/Linux, FreeBSD, MacOS X, Cygwin distributed builds L. Courtès Hydra: continuous integration for demanding people 35
Hydra, Nix, & you http://nixos.org/hydra/ Nix build tool focus on controlling the build environment Hydra continuous integration tool based on Nix web interface + email notification GNU/Linux, FreeBSD, MacOS X, Cygwin distributed builds SSH/root access to the build host not needed L. Courtès Hydra: continuous integration for demanding people 36
Hydra, Nix, & you http://nixos.org/hydra/ Nix build tool focus on controlling the build environment Hydra continuous integration tool based on Nix web interface + email notification GNU/Linux, FreeBSD, MacOS X, Cygwin distributed builds SSH/root access to the build host not needed manage the build environment by yourself! L. Courtès Hydra: continuous integration for demanding people 37
a project L. Courtès Hydra: continuous integration for demanding people 38
a job set L. Courtès Hydra: continuous integration for demanding people 39
a build L. Courtès Hydra: continuous integration for demanding people 40
build inputs L. Courtès Hydra: continuous integration for demanding people 41
build input changes L. Courtès Hydra: continuous integration for demanding people 42
view of a successful job set L. Courtès Hydra: continuous integration for demanding people 43
a failed build L. Courtès Hydra: continuous integration for demanding people 44
build input changes that led to a failure L. Courtès Hydra: continuous integration for demanding people 45
build environments vs. reproducibility versions of the dependencies compiler compilation options, and those of dependencies miscellaneous (locale, timezone, etc.) paths L. Courtès Hydra: continuous integration for demanding people 46
build environments vs. reproducibility versions of the dependencies compiler compilation options, and those of dependencies miscellaneous (locale, timezone, etc.) paths -I/path/to/headers -L/path/to/lib $CPATH $LIBRARY PATH L. Courtès Hydra: continuous integration for demanding people 47
build environments vs. reproducibility versions of the dependencies compiler compilation options, and those of dependencies miscellaneous (locale, timezone, etc.) paths -I/path/to/headers -L/path/to/lib $LD LIBRARY PATH RPATH $CPATH $LIBRARY PATH RUNPATH L. Courtès Hydra: continuous integration for demanding people 48
build environments vs. reproducibility versions of the dependencies compiler compilation options, and those of dependencies miscellaneous (locale, timezone, etc.) paths -I/path/to/headers -L/path/to/lib $LD LIBRARY PATH $PYTHONPATH $XML CATALOG FILES $CPATH $LIBRARY PATH RPATH RUNPATH $CLASSPATH $PERL5LIB $GUILE LOAD PATH L. Courtès Hydra: continuous integration for demanding people 49
build environments vs. reproducibility versions of the dependencies compiler compilation options, and those of dependencies miscellaneous (locale, timezone, etc.) paths -I/path/to/headers $CPATH -L/path/to/lib $LIBRARY PATH $LD LIBRARY PATH $PYTHONPATH RPATH Ahem, reproducible builds? $CLASSPATH RUNPATH $XML CATALOG FILES $PERL5LIB $GUILE LOAD PATH L. Courtès Hydra: continuous integration for demanding people 50
build environment control without Nix hello qt4 which compiler? which libc? L. Courtès Hydra: continuous integration for demanding people 51
build environment control without Nix hello qt4 which compiler? which libc? and now your favorite admin upgrades the system... L. Courtès Hydra: continuous integration for demanding people 52
build environment control without Nix hello qt4 which compiler? which libc? and now your favorite admin upgrades the system... Segmentation fault L. Courtès Hydra: continuous integration for demanding people 53
build environment control without Nix hello qt4 uncontrolled build environment = untraceable failures which compiler? which libc? and now your favorite admin upgrades the system... Segmentation fault L. Courtès Hydra: continuous integration for demanding people 54
default-builder.sh ncurses-5.7.drv lzma-4.32.7.drv texinfo-4.13a.drv perl-5.10.0.drv linux-headers-2.6.28.5.drv binutils-2.19.1.drv replace-2.24.drv pcre-7.8.drv gawk-3.1.6.drv lzma-4.32.7.drv gettext-0.17.drv perl-5.10.0.drv gnused-4.1.5.drv bzip2-1.0.5.drv gnum4-1.4.13.drv gnupatch-2.5.4.drv patchelf-0.4.drv gnumake-3.81.drv gnugrep-2.5.4.drv prehook.sh bootstrap-tools.drv builder.sh setup.sh libtool-2.2.6a.drv attr-2.4.43.drv acl-2.2.47.drv no-sys-dirs.patch builder.sh ld-wrapper.sh bootstrap-glibc.drv utils.sh setup-hook.sh add-flags gcc-wrapper.sh setup-hook.sh malloc.patch gettext-fix.patch stdenv-linux-boot.drv builder.sh log.patch impure-dirs.patch diffutils-2.8.1.drv coreutils-7.2.drv bootstrap-gcc.drv prehook.sh builder.sh stdenv-linux-boot.drv nss-skip-unavail.patch glibc-2.9.drv locale-override.patch rpcgen-path.patch gcc-wrapper-4.3.3.drv new-dtags.patch bootstrap-gcc.drv stdenv-linux-boot.drv pass-cxxcpp.patch builder.sh no-sys-dirs.patch linux-2.6.28.5.tar.bz2.drv gnum4-1.4.13.drv gcc-4.3.3.drv gmp-4.3.1.drv mpfr-2.4.1.drv glibc-2.9-20081208.tar.bz2.drv binutils-2.19.1.tar.bz2.drv mpfr-2.4.1.tar.bz2.drv gmp-4.3.1.tar.bz2.drv texinfo-4.13a.tar.lzma.drv ncurses-5.7.tar.gz.drv perl-5.10.0.tar.gz.drv gcc-core-4.3.3.tar.bz2.drv m4-1.4.13.tar.bz2.drv lzma-4.32.7.tar.gz.drv sed-4.1.5.tar.gz.drv gcc-g++-4.3.3.tar.bz2.drv bash40-017.drv attr_2.4.43-1.tar.gz.drv bzip2-1.0.5.tar.gz.drv bash40-016.drv bash40-015.drv coreutils-7.2.tar.gz.drv bash40-010.drv bison-2.3.tar.bz2.drv bash40-006.drv bash-4.0.tar.gz.drv bash40-001.drv bash40-003.drv bash40-012.drv gettext-0.17.tar.gz.drv libtool-2.2.6a.tar.lzma.drv bash40-002.drv bash40-004.drv bash40-009.drv bash40-011.drv replace-2.24-src-11.11.tar.gz.drv bash40-014.drv bash40-013.drv bash40-007.drv bash40-008.drv bash40-005.drv diffutils-2.8.1.tar.gz.drv gawk-3.1.6.tar.bz2.drv acl_2.2.47-1.tar.gz.drv pcre-7.8.tar.bz2.drv grep-2.5.4.tar.bz2.drv patch-2.5.4.tar.gz.drv findutils-4.4.1.tar.gz.drv patchelf-0.4.tar.bz2.drv make-3.81.tar.bz2.drv gzip-1.3.12.tar.gz.drv bison-2.3.drv write-mirror-list.sh gcc-wrapper-4.3.3.drv findutils-path.patch change_echo_path.patch findutils-4.4.1.drv mirrors-list.drv perl-5.10.0.drv stdenv-linux.drv hello-2.3.drv bash-4.0-p17.drv hello-2.3.tar.bz2.drv bzip2 mkdir curl.bz2 download.sh ln cpio sh openssl-0.9.8k.drv curl-7.19.4.drv bootstrap-tools.cpio.bz2.drv connect-timeout.patch zlib-1.2.3.drv unpack-bootstrap-tools.sh gnulib-futimens.patch gzip-1.3.12.drv gnutar-1.22.drv implausible.patch mirrors-list.drv stdenv-linux-boot.drv builder.sh tar-1.22.tar.bz2.drv zlib-1.2.3.tar.gz.drv openssl-0.9.8k.tar.gz.drv curl-7.19.4.tar.bz2.drv build environment control with Nix... down to the compiler s compiler! 1 Except for the kernel, the hardware, the direction of the wind, etc. L. Courtès Hydra: continuous integration for demanding people 55
default-builder.sh ncurses-5.7.drv lzma-4.32.7.drv texinfo-4.13a.drv perl-5.10.0.drv linux-headers-2.6.28.5.drv binutils-2.19.1.drv replace-2.24.drv pcre-7.8.drv gawk-3.1.6.drv lzma-4.32.7.drv gettext-0.17.drv perl-5.10.0.drv gnused-4.1.5.drv bzip2-1.0.5.drv gnum4-1.4.13.drv gnupatch-2.5.4.drv patchelf-0.4.drv gnumake-3.81.drv gnugrep-2.5.4.drv prehook.sh bootstrap-tools.drv builder.sh setup.sh libtool-2.2.6a.drv attr-2.4.43.drv acl-2.2.47.drv no-sys-dirs.patch builder.sh ld-wrapper.sh bootstrap-glibc.drv utils.sh setup-hook.sh add-flags gcc-wrapper.sh setup-hook.sh malloc.patch gettext-fix.patch stdenv-linux-boot.drv builder.sh log.patch impure-dirs.patch diffutils-2.8.1.drv coreutils-7.2.drv bootstrap-gcc.drv prehook.sh builder.sh stdenv-linux-boot.drv nss-skip-unavail.patch glibc-2.9.drv locale-override.patch rpcgen-path.patch gcc-wrapper-4.3.3.drv new-dtags.patch bootstrap-gcc.drv stdenv-linux-boot.drv pass-cxxcpp.patch builder.sh no-sys-dirs.patch linux-2.6.28.5.tar.bz2.drv gnum4-1.4.13.drv gcc-4.3.3.drv gmp-4.3.1.drv mpfr-2.4.1.drv glibc-2.9-20081208.tar.bz2.drv binutils-2.19.1.tar.bz2.drv mpfr-2.4.1.tar.bz2.drv gmp-4.3.1.tar.bz2.drv texinfo-4.13a.tar.lzma.drv ncurses-5.7.tar.gz.drv perl-5.10.0.tar.gz.drv gcc-core-4.3.3.tar.bz2.drv m4-1.4.13.tar.bz2.drv lzma-4.32.7.tar.gz.drv sed-4.1.5.tar.gz.drv gcc-g++-4.3.3.tar.bz2.drv bash40-017.drv attr_2.4.43-1.tar.gz.drv bzip2-1.0.5.tar.gz.drv bash40-016.drv bash40-015.drv coreutils-7.2.tar.gz.drv bash40-010.drv bison-2.3.tar.bz2.drv bash40-006.drv bash-4.0.tar.gz.drv bash40-001.drv bash40-003.drv bash40-012.drv gettext-0.17.tar.gz.drv libtool-2.2.6a.tar.lzma.drv bash40-002.drv bash40-004.drv bash40-009.drv bash40-011.drv replace-2.24-src-11.11.tar.gz.drv bash40-014.drv bash40-013.drv bash40-007.drv bash40-008.drv bash40-005.drv diffutils-2.8.1.tar.gz.drv gawk-3.1.6.tar.bz2.drv acl_2.2.47-1.tar.gz.drv pcre-7.8.tar.bz2.drv grep-2.5.4.tar.bz2.drv patch-2.5.4.tar.gz.drv findutils-4.4.1.tar.gz.drv patchelf-0.4.tar.bz2.drv make-3.81.tar.bz2.drv gzip-1.3.12.tar.gz.drv bison-2.3.drv write-mirror-list.sh gcc-wrapper-4.3.3.drv findutils-path.patch change_echo_path.patch findutils-4.4.1.drv mirrors-list.drv perl-5.10.0.drv stdenv-linux.drv hello-2.3.drv bash-4.0-p17.drv hello-2.3.tar.bz2.drv bzip2 mkdir curl.bz2 download.sh ln cpio sh openssl-0.9.8k.drv curl-7.19.4.drv bootstrap-tools.cpio.bz2.drv connect-timeout.patch zlib-1.2.3.drv unpack-bootstrap-tools.sh gnulib-futimens.patch gzip-1.3.12.drv gnutar-1.22.drv implausible.patch mirrors-list.drv stdenv-linux-boot.drv builder.sh tar-1.22.tar.bz2.drv zlib-1.2.3.tar.gz.drv openssl-0.9.8k.tar.gz.drv curl-7.19.4.tar.bz2.drv build environment control with Nix... down to the compiler s compiler! build environment entirely under control 1 all the build inputs are known to Nix 1 Except for the kernel, the hardware, the direction of the wind, etc. L. Courtès Hydra: continuous integration for demanding people 56
introduction features practice closing L. Courtès Hydra: continuous integration for demanding people 57
hello, world! L. Courtès Hydra: continuous integration for demanding people 58
a job set L. Courtès Hydra: continuous integration for demanding people 59
writing a Hydra job set in Nix job set } { tarball =...; build =...; coverage =...; deb =...; job name L. Courtès Hydra: continuous integration for demanding people 60
writing a Hydra job set in Nix } { tarball =...; build =...; coverage =...; deb =...; make dist make && make check... make CFLAGS=--coverage... qemu... checkinstall... L. Courtès Hydra: continuous integration for demanding people 61
writing Hydra jobs in Nix parameters tarball = { nixpkgs, hellosrc } : function let pkgs = import nixpkgs { }; in pkgs.releasetools.sourcetarball { name = "hello"; src = hellosrc; }; L. Courtès Hydra: continuous integration for demanding people 62
writing Hydra jobs in Nix checkout of the distro checkout of Hello tarball = { nixpkgs, hellosrc } : let pkgs = import nixpkgs { }; in pkgs.releasetools.sourcetarball { name = "hello"; src = hellosrc; }; L. Courtès Hydra: continuous integration for demanding people 63
writing Hydra jobs in Nix result of tarball job x86 64-linux, or x86 64-darwin, etc. build = { nixpkgs, tarball, system } : let pkgs = import nixpkgs { inherit system; }; in pkgs.releasetools.nixbuild { name = "hello"; src = tarball; }; L. Courtès Hydra: continuous integration for demanding people 64
writing Hydra jobs in Nix build = { nixpkgs, tarball, system } : let pkgs = import nixpkgs { inherit system; }; in pkgs.releasetools.nixbuild { name = "hello"; explicit dependency src = tarball; buildinputs = [ pkgs.qt4 ]; hello }; qt4 L. Courtès Hydra: continuous integration for demanding people 65
writing Hydra jobs in Nix build = { nixpkgs, tarball, system } : let pkgs = import nixpkgs { inherit system; }; in pkgs.releasetools.nixbuild { name = "hello"; src = tarball; buildinputs = [ pkgs.qt4 ]; configureflags = "--with-colors"; }; L. Courtès Hydra: continuous integration for demanding people 66
writing Hydra jobs in Nix coverage = { nixpkgs, tarball, system } : let pkgs = import nixpkgs { inherit system; }; in pkgs.releasetools.coverageanalysis { name = "hello"; src = tarball; buildinputs = [ pkgs.qt4 ]; configureflags = "--with-colors"; }; L. Courtès Hydra: continuous integration for demanding people 67
writing Hydra jobs in Nix deb = { nixpkgs, tarball, system } : let pkgs = import nixpkgs { inherit system; }; in pkgs.releasetools.debbuild { name = "hello"; src = tarball; configureflags = "--with-colors"; diskimage = vmtools.diskimages.debian50x86 64; }; L. Courtès Hydra: continuous integration for demanding people 68
filling in the blanks... L. Courtès Hydra: continuous integration for demanding people 69
But Autotools suck, CMake rocks. Anonymous (and cavalier) attendee. L. Courtès Hydra: continuous integration for demanding people 70
building with CMake build = { system, nixpkgs, hellosrc }: let pkgs = import nixpkgs { inherit system; }; in pkgs.releasetools.nixbuild { name = "hello-cmake" ; src = hellosrc; cmakeflags = "-DFOO BAR=on"; buildinputs = [ pkgs.cmake pkgs.qt4 ]; }; L. Courtès Hydra: continuous integration for demanding people 71
All these C-ish things suck anyway; Java is the way. Anonymous attendee. L. Courtès Hydra: continuous integration for demanding people 72
building with Ant build = { nixpkgs, hellosrc }: let pkgs = import nixpkgs { }; in pkgs.releasetools.antbuild { name = "hello-ant" ; src = hellosrc; anttargets = [ "jar" "test" ]; buildinputs = with pkgs; [ ant jdk ]; }; L. Courtès Hydra: continuous integration for demanding people 73
building with Maven build = { nixpkgs, hellosrc }: let pkgs = import nixpkgs { }; in pkgs.releasetools.mvnbuild { name = "hello-maven" ; src = hellosrc; dojavadoc = true; docheckstyle = true; buildinputs = with pkgs; [ maven2 jdk ]; }; L. Courtès Hydra: continuous integration for demanding people 74
building things differently customize existing build functions (nixbuild, etc.) write your own build function L. Courtès Hydra: continuous integration for demanding people 75
building things differently customize existing build functions (nixbuild, etc.) define configureflags, makeflags, etc. define configurephase, buildphase, etc. (shell snippets)... write your own build function pythonbuild anyone? L. Courtès Hydra: continuous integration for demanding people 76
introduction features practice closing L. Courtès Hydra: continuous integration for demanding people 77
Continuous integration pacifies developers! L. Courtès Hydra: continuous integration for demanding people 78
summary Continuous integration pacifies developers! quick feedback to developers source + binaries + doc snapshots for users tests of various configurations L. Courtès Hydra: continuous integration for demanding people 79
summary Continuous integration pacifies developers! quick feedback to developers source + binaries + doc snapshots for users tests of various configurations available today! L. Courtès Hydra: continuous integration for demanding people 80
Hydra @ Inria: what s missing production-ready! L. Courtès Hydra: continuous integration for demanding people 81
Hydra @ Inria: what s missing not quite production-ready L. Courtès Hydra: continuous integration for demanding people 82
Hydra @ Inria: what s missing not quite production-ready no private/proprietary projects yet L. Courtès Hydra: continuous integration for demanding people 83
Hydra @ Inria: what s missing not quite production-ready no private/proprietary projects yet no GPUs, no webcams, no USB percolator (yet) L. Courtès Hydra: continuous integration for demanding people 84
Hydra @ Inria: what s missing not quite production-ready no private/proprietary projects yet no GPUs, no webcams, no USB percolator (yet) some of your dependencies might be missing L. Courtès Hydra: continuous integration for demanding people 85
Hydra @ Inria: what s missing not quite production-ready no private/proprietary projects yet no GPUs, no webcams, no USB percolator (yet) some of your dependencies might be missing no blame list, no favorite Jenkins plug-in here... L. Courtès Hydra: continuous integration for demanding people 86
Hydra @ Inria: what s already there small & growing compile farm: 12 x86 64 cores L. Courtès Hydra: continuous integration for demanding people 87
Hydra @ Inria: what s already there small & growing compile farm: 12 x86 64 cores GNU/Linux, MacOS X, and FreeBSD L. Courtès Hydra: continuous integration for demanding people 88
Hydra @ Inria: what s already there small & growing compile farm: 12 x86 64 cores GNU/Linux, MacOS X, and FreeBSD kernel & distributed tests in virtual machines virtual build environments: Debian, Fedora, opensuse, etc. L. Courtès Hydra: continuous integration for demanding people 89
Hydra @ Inria: what s already there small & growing compile farm: 12 x86 64 cores GNU/Linux, MacOS X, and FreeBSD kernel & distributed tests in virtual machines virtual build environments: Debian, Fedora, opensuse, etc. 6.5 projects being built (C/C++/Fortran, Autotools/CMake) L. Courtès Hydra: continuous integration for demanding people 90
Hydra @ Inria: what s already there small & growing compile farm: 12 x86 64 cores GNU/Linux, MacOS X, and FreeBSD kernel & distributed tests in virtual machines virtual build environments: Debian, Fedora, opensuse, etc. 6.5 projects being built (C/C++/Fortran, Autotools/CMake)... L. Courtès Hydra: continuous integration for demanding people 91
getting started 1. email sed-bordeaux@inria.fr L. Courtès Hydra: continuous integration for demanding people 92
getting started 1. email sed-bordeaux@inria.fr 2. add your recipes to http://gforge.inria.fr/projects/hydra-recipes L. Courtès Hydra: continuous integration for demanding people 93
getting started 1. email sed-bordeaux@inria.fr 2. add your recipes to http://gforge.inria.fr/projects/hydra-recipes 3. watch your project at http://hydra.bordeaux.inria.fr/ L. Courtès Hydra: continuous integration for demanding people 94
getting help copy/paste from http://gforge.inria.fr/projects/hydra-recipes read the fine manual at http://nixos.org/hydra/ email sed-bordeaux@inria.fr L. Courtès Hydra: continuous integration for demanding people 95
ludovic.courtes@inria.fr http://sed.bordeaux.inria.fr/