Unit testing with examples in C++

Unit testing with examples in C++ icer Workshop 02/19/2015 Yongjun Choi choiyj@msu.edu Research Specialist, Institute for Cyber- Enabled Research Download the slides: https://wiki.hpcc.msu.edu/x/_ajiaq

Agenda Test-driven development Unit testing and frameworks Unit testing in C++ & Boost.Test

How this workshop works We are going to cover some basics. A few of hands on examples Exercises are denoted by the following icon in this presents:

Test-driven development (TDD) TDD is a software development process which relies on the repetition of a very short development cycle 1. Add a failing test case that defines a desired improvement or new function 2. Run all tests and see if the new one fails 3. Write some code to pass the test 4. Run tests 5. Refactor code

Test-driven development (TDD) Let s assume you want to develop a software module which calculates defint(f(x), a, b) You need to know if your new module works correctly. So, before you start coding, a test case is prepared. eg: defint(f(x), a, b) == 1/2 Now you develop a module to pass this test

TDD workflow http://en.wikipedia.org/wiki/test-driven_development

Software testing Many different forms of tests: unit tests: integration tests: to test if Individual units are combined and functioned as a group. regression tests: to uncover new software bugs in existing functional performance tests: to determine how a system performs in terms of responsiveness and stability under a particular workload.

Unit testing Unit testing is a method by which individual units of source code are tested to determine if they are correctly working. A unit is the smallest testable part of an application. an individual function or procedure

Benefits of unit testing Facilitate changes: unit tests allow programmers to refactor code at later date, and be sure that code still works correctly; Simplify integration: unit tests may reduce uncertainty in the units, and can be used in a bottom-up testing style approach. Living documentation for the system: easy to gain a basic understanding of implemented API

How to organize tests Each test case should test only one thing A test case should be short Test should run fast, so it will possible to run it very often Each test should work independent of other test Tests should NOT be dependent on the order of their execution

Test methods: man in the loop How should a test program report errors? Displaying an error message would be one possible way. if( something_bad_detected ) std::cout << Something bad has been detected << std::endl; requires inspection of the program s output after each run to determine if an error exists. Human inspection of output message is time consuming and unreliable. Unit testing frameworks are designed to automate those tasks.

Test methods: EXIT_SUCCESS/EXIT_FAILURE #include <my_class.hpp> int main( int, char* [] ) { my_class test_object( "qwerty" ); return test_object.is_valid()? EXIT_SUCCESS : EXIT_FAILURE; There are several issues with the test above You need to convert is_valid result in proper result code If exception happens in test_object when is_valid invocation, the program will crash. You won t see any output, if you run it manually.

Test methods: Unit testing frameworks The Unit Test Framework solves all these issues. To integrate it the above program needs to be changed to: #include <my_class.hpp> #define BOOST_TEST_MODULE MyTest #include <boost/test/unit_test.hpp> BOOST_AUTO_TEST_CASE( my_test ) { my_class test_object( "qwerty" ); BOOST_CHECK( test_object.is_valid() );

Test methods: Unit testing frameworks To simplify development of unit tests, unit test frameworks are usually used. Almost any programming language has several unit testing frameworks.

Unit testing frameworks in C++ There are many unit testing frameworks for C++ Boost.Test, Google C++ Testing Framework, etc, etc, etc! Boost.Test Suitable for novice and advanced users Allows organization of test cases into test suites Test cases can be registered automatically and/or manually Fixtures (initialization and cleanup of resources) Large number of assertion/checkers exceptions, equal, not equal, greater, less, floating point numbers comparison etc.

Preparation connect to the MSU hpc open a terminal and do 1. ssh your_user_id@gateway.hpcc.msu.edu 2. ssh dev-intel10 3. module load powertools 4. getexample boostunittests 5. cd boostunittests Now you are ready to run boost unit test samples!

simpleunittest1.cpp #define BOOST_TEST_MODULE Simple testcase // name of this test #include <boost/test/unit_test.hpp> //header file for boost.test // a module that we want to develop int addtwonumbers(int i, int j ) { return 0; //Here is a test BOOST_AUTO_TEST_CASE(addition) { BOOST_CHECK(addTwoNumbers(2, 3) == 5);

simpleunittest1 Now let s build and run it You should be ~/boostunittests. Please do 1. mkdir build 2. cd build 3. cmake../ 4. make 5../simpleUnitTest1 It will give you an error message because we did not implement the module yet.

simpleunittest1 #define BOOST_TEST_MODULE Simple testcases #include <boost/test/unit_test.hpp> int add(int i, int j ) {return 0; BOOST_AUTO_TEST_CASE(addition) { BOOST_CHECK(add(2, 3) == 5); A failed BOOST_CHECK will not exit the test case immediately - the problem is recorded and the case continues. To immediately fail a test, BOOST_REQUIRE is used.

simpleunittest2.cpp #define BOOST_TEST_MODULE Simple testcase //declares name of this test #include <boost/test/unit_test.hpp> // header file for boost.test // a module that we want to develop int addtwonumbers(int i, int j ) { return i - j; // need to be fixed to pass the test //Here is a test BOOST_AUTO_TEST_CASE(addition) { BOOST_CHECK(addTwoNumbers(2, 3) == 5); BOOST_CHECK_MESSAGE(addTwoNumbers(2, 3)== 5, "<addtwonumbers> has some problem!"); BOOST_CHECK_EQUAL(addTwoNumbers(2, 3), 5); BOOST_REQUIRE(addTwoNumbers(2, 3) == 5); BOOST_WARN(addTwoNumbers(2, 3) == 5); //will never reach this check

simpleunittest2 Run and verify the difference between assertions. Fix the bug, and test it again.

Assertions BOOST_AUTO_TEST_CASE(addition) { BOOST_CHECK(addTwoNumbers(2, 3) == 5); BOOST_CHECK_MESSAGE(addTwoNumbers(2, 3)== 5, "<addtwonumbers> has some problem!"); BOOST_CHECK_EQUAL(addTwoNumbers(2, 3), 5); BOOST_REQUIRE(addTwoNumbers(2, 3) == 5); BOOST_WARN(addTwoNumbers(2, 3) == 5); //will never reach this check BOOST_WARN is not checked here, because the test has failed with BOOST_CHECK_REQUIRE

Testing tools/checkers WARN produces warning message check failed, but error counter does not increase and test case continues CHECK reports error and increases error counter when check fails, but test case continues REQUIRE similar to CHECK, but it is not used to report fatal errors.

Testing tools/checkers expression examples BOOST_WARN( sizeof(int) ==sizeof(short)); BOOST_CHECK(i == 1); BOOST_REQUIRE (i > 5); BOOST_REQUIRE( h, s[0]); If the check fails, Boost.Test will report the line in the source code where this occurred and what condition was specified. See the example

More Assertions CHECK, REQUIRE, and WARN (with log_level=all ) BOOST_CHECK_MESSAGE BOOST_CHECK_EQUAL The full list of available assertions can be found http://goo.gl/xqczvr

simpleunittest3.cpp #define BOOST_TEST_MODULE Suites #include <boost/test/unit_test.hpp> int add(int i, int j) { return i + j; BOOST_AUTO_TEST_SUITE(Maths) BOOST_AUTO_TEST_CASE(addition) { BOOST_CHECK(add(2, 2) == 4); BOOST_AUTO_TEST_SUITE_END() BOOST_AUTO_TEST_SUITE(Physics) BOOST_AUTO_TEST_CASE(specialRelativity) { int e = 32; int m = 2; int c = 4; BOOST_CHECK(e == m * c * c); BOOST_AUTO_TEST_SUITE_END() If you have many test cases in one program, then their maintenance could be hard. test suite is available In a normal run, you won t see that the tests have been categorized. With log_level=test_suite or log_level=all, you will.

Fixtures Fixtures are special objects that are used to implement setup and cleanup of data/resources required to execution of unit tests. Separation of code between fixtures and actual test code, simplifies the unit test code, and allows the same initialization code to be used for different test cases and test suites.

Fixtures example struct MyFixture { int* i; MyFixture() i = new int; *i = 0; ~MyFixture() {delete[] i; ; and you can use it in the following way: BOOST_AUTO_TEST_CASE (test_case1) { MyFixture f; // do something with f.i

Boost.Test s fixture macro #define BOOST_TEST_MODULE fixture example #include <boost/test/unit_test.hpp> struct F { int* i; F(): i(1) { ~F() { ; and you can use it following way: BOOST_FIXTURE_TEST_CASE(simple_test, F) { BOOST_CHECK_EQUAL*i, 1);

simpleunittest4.cpp #define BOOST_TEST_MODULE Fixtures #include <boost/test/unit_test.hpp> struct Massive { int m; Massive() : m(3) { BOOST_TEST_MESSAGE("setup mass"); ~Massive() { BOOST_TEST_MESSAGE("teardown mass"); ; The Massive fixture was created, it holds one variable m, and it is directly accessible in our test case. BOOST_FIXTURE_TEST_SUITE(Physics, Massive) BOOST_AUTO_TEST_CASE(specialTheory) { int e = 48; int c = 4; BOOST_CHECK(e == m * c * c); BOOST_AUTO_TEST_CASE(newton2) { int f = 10; int a = 5; BOOST_CHECK(f == m * a); BOOST_AUTO_TEST_SUITE_END()