Unit testing using Python nose Luis Pedro Coelho On the web: http://luispedro.org On twitter: @luispedrocoelho European Molecular Biology Laboratory June 11, 2014 Luis Pedro Coelho (luis@luispedro.org) (EMBL) Unit testing June 11, 2014 (1 / 23)
Motivation Scientific code must not just produce nice looking output, but actually be correct. http://bit.ly/testing-science Luis Pedro Coelho (luis@luispedro.org) (EMBL) Unit testing June 11, 2014 (2 / 23)
Motivation (II) Some recent code-related scientific catastrophes: Geoffrey Chang Abortion reduces crime? maybe not so much once you fix the bug Rogoff s Growth in a Time of Debt paper is a famous example (even if the bug itself is only a small part of the counter-argument) East Anglia Climategate Luis Pedro Coelho (luis@luispedro.org) (EMBL) Unit testing June 11, 2014 (3 / 23)
Why do things go wrong?.1 Your code is correct, but input files are wrong/missing/, the network goes down.2 Your code is buggy. Luis Pedro Coelho (luis@luispedro.org) (EMBL) Unit testing June 11, 2014 (4 / 23)
Never fail silently! The worst thing is to fail silently. Fail loudly and clearly (This is partially why Unix tradition is to produce no output when things go well) Luis Pedro Coelho (luis@luispedro.org) (EMBL) Unit testing June 11, 2014 (5 / 23)
Defensive Programming Defensive programming means writing code that will catch bugs early. Luis Pedro Coelho (luis@luispedro.org) (EMBL) Unit testing June 11, 2014 (6 / 23)
Assertions def stddev ( v a l u e s ) : S = stddev ( v a l u e s ) Compute standard d e v i a t i o n a s s e r t l e n ( v a l u e s ) > 0, stddev : got empty l i s t.... Luis Pedro Coelho (luis@luispedro.org) (EMBL) Unit testing June 11, 2014 (7 / 23)
Assertions def stddev ( v a l u e s ) : S = stddev ( v a l u e s ) Compute standard d e v i a t i o n i f l e n ( v a l u e s ) <= 0 : r a i s e A s s e r t i o n E r r o r ( stddev : got empty l i s t. )... Luis Pedro Coelho (luis@luispedro.org) (EMBL) Unit testing June 11, 2014 (8 / 23)
Preconditions In computer programming, a precondition is a condition or predicate that must always be true just prior to the execution of some section of code. (Wikipedia) Luis Pedro Coelho (luis@luispedro.org) (EMBL) Unit testing June 11, 2014 (9 / 23)
Preconditions Other Languages C/C++ #include <assert.h> Java assert pre-condition Matlab assert () (in newer versions) Luis Pedro Coelho (luis@luispedro.org) (EMBL) Unit testing June 11, 2014 (10 / 23)
Assertions Are Not Error Handling! Error handling protects against outside events; assertions protect against programmer mistakes. Assertions should never be false. Luis Pedro Coelho (luis@luispedro.org) (EMBL) Unit testing June 11, 2014 (11 / 23)
Programming by Contract.1 pre-conditions..2 post-conditions..3 invariants. Luis Pedro Coelho (luis@luispedro.org) (EMBL) Unit testing June 11, 2014 (12 / 23)
Pre-condition What must be true before calling a function. Post-condition What is true after calling a function. Luis Pedro Coelho (luis@luispedro.org) (EMBL) Unit testing June 11, 2014 (13 / 23)
Testing Do you test your code? Luis Pedro Coelho (luis@luispedro.org) (EMBL) Unit testing June 11, 2014 (14 / 23)
Unit Testing def test_stddev_const ( ) : a s s e r t stddev ( [ 1 ] * 100 ) < 1e - 3 def t e s t _ s t d d e v _ p o s i t i v e ( ) : a s s e r t stddev ( range ( 20 ) ) > 0. Luis Pedro Coelho (luis@luispedro.org) (EMBL) Unit testing June 11, 2014 (15 / 23)
Nosetest Nose software testing framework: Tests are named test_something. Conditions are asserted. Luis Pedro Coelho (luis@luispedro.org) (EMBL) Unit testing June 11, 2014 (16 / 23)
Software Testing Philosophies.1 Test everything. Test it twice..2 Write tests first..3 Regression testing. Luis Pedro Coelho (luis@luispedro.org) (EMBL) Unit testing June 11, 2014 (17 / 23)
Regression Testing Make sure bugs only appear once! Luis Pedro Coelho (luis@luispedro.org) (EMBL) Unit testing June 11, 2014 (18 / 23)
Practical Session: some preliminaries statistics.py def stddev ( xs ) :... test_statistics.py def test_stddev_const ( ) : a s s e r t stddev ( [ 1 ] * 100 ) < 1e - 3 def t e s t _ s t d d e v _ p o s i t i v e ( ) : a s s e r t stddev ( range ( 20 ) ) > 0. Luis Pedro Coelho (luis@luispedro.org) (EMBL) Unit testing June 11, 2014 (19 / 23)
Practical Session: some preliminaries statistics.py def stddev ( xs ) :... test_statistics.py import s t a t i s t i c s def test_stddev_const ( ) : a s s e r t s t a t i s t i c s. stddev ( [ 1 ] *100 ) < 1e - 3 def t e s t _ s t d d e v _ p o s i t i v e ( ) : a s s e r t s t a t i s t i c s. stddev ( range ( 20 ) ) > 0. Luis Pedro Coelho (luis@luispedro.org) (EMBL) Unit testing June 11, 2014 (20 / 23)
Practical: Python III & Unit testing.1 You can either start from scratch or check the files I give you (or any combination of both)..2 Goal is to write code to do a simple task & test it. Luis Pedro Coelho (luis@luispedro.org) (EMBL) Unit testing June 11, 2014 (21 / 23)
Types of tests Smoke test: just check it runs Corner/edge cases: check complex cases. Case testing: test a known case Regression testing: create a test when you find a bug. Integration test: test that different parts work together. Luis Pedro Coelho (luis@luispedro.org) (EMBL) Unit testing June 11, 2014 (22 / 23)
Goals.1 Download files from https://github.com/luispedro/2014-06-10-cyi-support/testing.2 There is a data file (data.txt).3 See the code in main.py, which loads it..4 Write a function average in a file called robust.py, which computes the average of a sequence of numbers, whilst ignoring the maximum and minimum..5 Write tests for robust.average..6 (If you have the time, you can look at plots.py) Luis Pedro Coelho (luis@luispedro.org) (EMBL) Unit testing June 11, 2014 (23 / 23)