Continuous Integration @ Inria https://ci.inria.fr

Continuous Integration @ Inria https://ci.inria.fr Damien Martin-Guillerez

Outline 1. Introduction 2. CI@Inria service description 3. CI@Inria usage example 4. Summary Damien Martin-Guillerez Mardi du dév - CI@Inria 2

1 Introduction Damien Martin-Guillerez Mardi du dév - CI@Inria 3

scenario 1: a peaceful development team... Damien Martin-Guillerez Mardi du dév - CI@Inria 4

scenario 1:... software that works... Damien Martin-Guillerez Mardi du dév - CI@Inria 4

scenario 1:... and suddenly... Damien Martin-Guillerez Mardi du dév - CI@Inria 4

scenario 1: who broke it?! Damien Martin-Guillerez Mardi du dév - CI@Inria 4

scenario 2: works for me user: It s broken. Damien Martin-Guillerez Mardi du dév - CI@Inria 5

scenario 2: works for me user: It s broken. dev: Weird, it works for me. Damien Martin-Guillerez Mardi du dév - CI@Inria 5

scenario 2: works for me user: It s broken. dev: Weird, it works for me. Which platform? Damien Martin-Guillerez Mardi du dév - CI@Inria 5

scenario 2: works for me user: It s broken. dev: Weird, it works for me. Which platform? user: Darwin 10.4. Damien Martin-Guillerez Mardi du dév - CI@Inria 5

scenario 2: works for me user: It s broken. dev: Weird, it works for me. Which platform? user: Darwin 10.4. On x86 64. Damien Martin-Guillerez Mardi du dév - CI@Inria 5

scenario 2: works for me user: It s broken. dev: Weird, it works for me. Which platform? user: Darwin 10.4. On x86 64. With Apple GCC 4.0.1, GMP 4.2.4, MPFR 2.4.2, BDB 4.2. Damien Martin-Guillerez Mardi du dév - CI@Inria 5

scenario 2: works for me user: It s broken. dev: Weird, it works for me. Which platform? user: Darwin 10.4. On x86 64. With Apple GCC 4.0.1, GMP 4.2.4, MPFR 2.4.2, BDB 4.2. And with --disable-color. Damien Martin-Guillerez Mardi du dév - CI@Inria 5

scenario 2: works for me user: It s broken. dev: Weird, it works for me. Which platform? user: Darwin 10.4. On x86 64. With Apple GCC 4.0.1, GMP 4.2.4, MPFR 2.4.2, BDB 4.2. And with --disable-color. dev: Well, too bad. I rather test on GNU/Linux, i686 Damien Martin-Guillerez Mardi du dév - CI@Inria 5

scenario 2: works for me user: It s broken. dev: Weird, it works for me. Which platform? user: Darwin 10.4. On x86 64. With Apple GCC 4.0.1, GMP 4.2.4, MPFR 2.4.2, BDB 4.2. And with --disable-color. dev: Well, too bad. I rather test on GNU/Linux, i686, with GCC 4.6.2, GMP 4.2.1, MPFR 2.4.1, GDBM 1.8.2, GTK+ 2.12. Oh, and with --enable-color --enable-gui. Damien Martin-Guillerez Mardi du dév - CI@Inria 5

scenario 2: works for me user: It s broken. dev: Weird, it works for me. Which platform? user: Darwin 10.4. On x86 64. With Apple GCC 4.0.1, GMP 4.2.4, MPFR 2.4.2, BDB 4.2. And with --disable-color. dev: Well, too bad. I rather test on GNU/Linux, i686, with GCC 4.6.2, GMP 4.2.1, MPFR 2.4.1, GDBM 1.8.2, GTK+ 2.12. Oh, and with --enable-color --enable-gui. Damien Martin-Guillerez Mardi du dév - CI@Inria 5

scenario 2: works for me user: It s broken. dev: Weird, it works for me. Which platform? user: Darwin 10.4. On x86 64. With Apple GCC 4.0.1, GMP 4.2.4, MPFR 2.4.2, BDB 4.2. And with --disable-color. dev: Well, too bad. I rather test on GNU/Linux, i686, with GCC 4.6.2, GMP 4.2.1, MPFR 2.4.1, GDBM 1.8.2, GTK+ 2.12. Oh, and with --enable-color --enable-gui. Damien Martin-Guillerez Mardi du dév - CI@Inria 5

scenario 2: works for me user: It s broken. dev: Weird, it works for me. Which platform? user: Darwin 10.4. On x86 64. With Apple GCC 4.0.1, GMP 4.2.4, MPFR 2.4.2, BDB 4.2. And with --disable-color. dev: Well, too bad. I rather test on GNU/Linux, i686, with GCC 4.6.2, GMP 4.2.1, MPFR 2.4.1, GDBM 1.8.2, GTK+ 2.12. Oh, and with --enable-color --enable-gui. Damien Martin-Guillerez Mardi du dév - CI@Inria 5

scenario 2: works for me user: It s broken. dev: Weird, it works for me. Which platform? user: Darwin 10.4. On x86 64. With Apple GCC 4.0.1, GMP 4.2.4, MPFR 2.4.2, BDB 4.2. And with --disable-color. dev: Well, too bad. I rather test on GNU/Linux, i686, with GCC 4.6.2, GMP 4.2.1, MPFR 2.4.1, GDBM 1.8.2, GTK+ 2.12. Oh, and with --enable-color --enable-gui. Damien Martin-Guillerez Mardi du dév - CI@Inria 5

scenario 2: works for me user: It s broken. dev: Weird, it works for me. Which platform? user: Darwin 10.4. On x86 64. With Apple GCC 4.0.1, GMP 4.2.4, MPFR 2.4.2, BDB 4.2. And with --disable-color. dev: Well, too bad. I rather test on GNU/Linux, i686, with GCC 4.6.2, GMP 4.2.1, MPFR 2.4.1, GDBM 1.8.2, GTK+ 2.12. Oh, and with --enable-color --enable-gui. Damien Martin-Guillerez Mardi du dév - CI@Inria 5

why continuous integration is so great 1. continuous compilation/test Damien Martin-Guillerez Mardi du dév - CI@Inria 6

why continuous integration is so great 1. continuous compilation/test 2. quick feedback Damien Martin-Guillerez Mardi du dév - CI@Inria 6

why continuous integration is so great 1. continuous compilation/test 2. quick feedback 3. pristine build environment Damien Martin-Guillerez Mardi du dév - CI@Inria 6

why continuous integration is so great 1. continuous compilation/test 2. quick feedback 3. pristine build environment 4. different configurations under test Damien Martin-Guillerez Mardi du dév - CI@Inria 6

why continuous integration is so great 1. continuous compilation/test 2. quick feedback 3. pristine build environment 4. different configurations under test 5. build results published Damien Martin-Guillerez Mardi du dév - CI@Inria 6

The True Cost of Doing It Yourself while true do rm -rf foo svn co svn+ssh://scm.gforge.inria.fr/svn/foo/trunk foo cd foo make sleep 300 done Damien Martin-Guillerez Mardi du dév - CI@Inria 7

The True Cost of Doing It Yourself while true do rm -rf foo svn co svn+ssh://scm.gforge.inria.fr/svn/foo/trunk foo cd foo make 2>&1 > build-log \ mail CEO@inria.fr -s "Build failed!" \ < build-log sleep 300 done Damien Martin-Guillerez Mardi du dév - CI@Inria 7

the true cost of doing it yourself function svn revision { ( cd "$1" svn info \ grep \^R.vision *: *\([0-9]\+\)$ \ sed -e s/[^0-9]//g ) } function svn update { ( prev rev="$(svn revision $1)" ( cd "$1" ; svn update ) new rev="$(svn revision $1)" test $prev rev -ne $new rev ) } function update html report { local stamp="$(date -u +%Y%m%d-%H%M)" local test log="$builddir/testsuite/log" local failure rate="(tests not run)" gzip --best < "$2" > "build-$stamp.log.gz" scp "build-$stamp.log.gz" "$ssh remote dir/" if [ -f "$test log" ] then failure rate="$(test failure rate $test log)" gzip --best bgcolor="lightred" else result="pass" bgcolor="lightgreen" fi local rev="$(svn revision $srcdir)" local html="<tr bgcolor= $bgcolor > <td><a href= $(gforge revision url foo $rev) >$rev< <td>$stamp</td> <td>$failure rate</td> <td> $result [ <a href= build-$stamp.log.gz >build</a> <a href=ẗest-$stamp.log.gz >test</a> ] </td> </tr>" ed "$html report" <<EOF /INSERT BUILD RESULTS +1 i $html. < "$builddir/testsuite/log" scp "test-$stamp.log.gz" "$ssh remote dir/" wq fi EOF local result bgcolor if [ $1 -ne 0 ] scp "$html report" "$ssh remote dir/index.html" then } result="<b>fail</b>" Damien Martin-Guillerez Mardi du dév - CI@Inria 8

the true cost of doing it yourself function svn revision { ( cd "$1" svn info \ grep \^R.vision *: *\([0-9]\+\)$ \ sed -e s/[^0-9]//g ) } function svn update { ( prev rev="$(svn revision $1)" ( cd "$1" ; svn update ) new rev="$(svn revision $1)" test $prev rev -ne $new rev ) } function update html report { local stamp="$(date -u +%Y%m%d-%H%M)" local test log="$builddir/testsuite/log" local failure rate="(tests not run)" gzip --best < "$2" > "build-$stamp.log.gz" scp "build-$stamp.log.gz" "$ssh remote dir/" if [ -f "$test log" ] then failure rate="$(test failure rate $test log)" gzip --best bgcolor="lightred" else result="pass" bgcolor="lightgreen" fi local rev="$(svn revision $srcdir)" local html="<tr bgcolor= $bgcolor > <td><a href= $(gforge revision url foo $rev) >$rev< <td>$stamp</td> <td>$failure rate</td> <td> $result [ <a href= build-$stamp.log.gz >build</a> <a href=ẗest-$stamp.log.gz >test</a> ] </td> </tr>" ed "$html report" <<EOF /INSERT BUILD RESULTS +1 i $html. < "$builddir/testsuite/log" scp "test-$stamp.log.gz" "$ssh remote dir/" wq fi EOF local result bgcolor if [ $1 -ne 0 ] scp "$html report" "$ssh remote dir/index.html" then } result="<b>fail</b>" Damien Martin-Guillerez Mardi du dév - CI@Inria 8

lots of tools! Buildbot Jenkins (formerly Hudson) Travis Hydra... Damien Martin-Guillerez Mardi du dév - CI@Inria 9

2CI@Inria service description Damien Martin-Guillerez Mardi du dév - CI@Inria 10

Service history 1st version at Lille after the Galaxy ADT - a web-portal - that automatizes creation of Hudson instance Damien Martin-Guillerez Mardi du dév - CI@Inria 11

Service history 1st version at Lille after the Galaxy ADT - a web-portal - that automatizes creation of Hudson instance 2nd version at Lille - both Hudson and Jenkins available - update possible - qualification space Damien Martin-Guillerez Mardi du dév - CI@Inria 11

Service history 1st version at Lille after the Galaxy ADT - a web-portal - that automatizes creation of Hudson instance 2nd version at Lille - both Hudson and Jenkins available - update possible - qualification space D2T selects the project to be national - automn 2011 - Engineer recruited / national project started - summer 2012 - beta-test - early 2013 - service opened Damien Martin-Guillerez Mardi du dév - CI@Inria 11

Service offer Web portal - User administration - Project administration Jenkins instances Compilation slaves - Cloud architecture - GNU/Linux and Windows VMs Damien Martin-Guillerez Mardi du dév - CI@Inria 12

Service architecture Damien Martin-Guillerez Mardi du dév - CI@Inria 13

Service team Development team - Cristophe Demarey @ SED Lille - Julien Garet @ DSI - Baptiste Legrand @ SED Lille (Eng. D2T) Support teams - ci-support@inria.fr: 1 SED / centre + E. Jeanvoine (SED Nancy) - SESI Team @ DSI D2T/DSI Lead - Janet Bertot / Claude Inglebert Damien Martin-Guillerez Mardi du dév - CI@Inria 14

3CI@Inria usage example Damien Martin-Guillerez Mardi du dév - CI@Inria 15

Project creation Step 1/2 Damien Martin-Guillerez Mardi du dév - CI@Inria 16

Project creation Step 2/2 Damien Martin-Guillerez Mardi du dév - CI@Inria 17

Slave creation Step 1/3 Damien Martin-Guillerez Mardi du dév - CI@Inria 18

Slave creation Step 2/3 Damien Martin-Guillerez Mardi du dév - CI@Inria 19

Slave creation Step 3/3 Damien Martin-Guillerez Mardi du dév - CI@Inria 20

Job creation Step 1/5 Damien Martin-Guillerez Mardi du dév - CI@Inria 21

Job creation Step 2/5 Damien Martin-Guillerez Mardi du dév - CI@Inria 22

Job creation Step 3/5 Damien Martin-Guillerez Mardi du dév - CI@Inria 23

Job creation Step 4/5 Damien Martin-Guillerez Mardi du dév - CI@Inria 24

Job creation Step 5/5 Damien Martin-Guillerez Mardi du dév - CI@Inria 25

4Summary Damien Martin-Guillerez Mardi du dév - CI@Inria 26

Summary Continuous integration is good for your sanity. Damien Martin-Guillerez Mardi du dév - CI@Inria 27

Summary Continuous integration is good for your sanity. CI@Inria makes it easy! Damien Martin-Guillerez Mardi du dév - CI@Inria 27

Summary Continuous integration is good for your sanity. CI@Inria makes it easy! So do it! Damien Martin-Guillerez Mardi du dév - CI@Inria 27

damien.martin-guillerez@inria.fr https://ci.inria.fr/