Unit test guide. Tuesday, October 23, 2012 Compiled by Nikita Astrakhantsev.

Unit test guide Tuesday, October 23, 2012 Compiled by Nikita Astrakhantsev astrakhantsev@ispras.ru

Contents Introduction... 3 Preliminaries... 3 Unit testing tasks... 4 What should be tested... 4 Unit-test one object at a time... 4 Testing conditions... 4 Too simple methods... 4 Test coverage... 5 Where to create test... 5 Test writing pipeline... 5 Setup environment... 5 Choose meaningful test method names... 6 Explain the failure reason in assert calls... 6 One unit test equals one @Test method... 6 Testing exceptions... 7 Custom matchers... 7 Let the test improve the code... 8 Automatic running... 9 2

Introduction This document briefly describes unit testing for Java. It is organized as follows: Firstly the basic notions are introduced. Then unit-testing tasks are discussed in order of their occurrence: what to test, when to stop testing, where to place tests, how to prepare environment, how to write tests themselves, and how to run them automatically. Preliminaries Unit test A unit test examines the behavior of a distinct unit of work. Within a Java application, the distinct unit of work is often (but not always) a single method. By contrast, integration tests and acceptance tests examine how various components interact. A unit of work is a task that is not directly dependent on the completion of any other task. Unit tests often focus on testing whether a method is following the terms of its API contract. API contract A view of an Application Programming Interface (API) as a formal agreement between the caller and the callee. Often the unit tests help define the API contract by demonstrating the expected behavior. All unit testing frameworks should follow: 1. Each unit test must run independently of all other unit tests. 2. The framework must detect and report errors test by test. 3. It must be easy to define which unit tests will run. We will use JUnit 4 1 as a testing framework. Most part of this guide is taken from the book: P. Tahchiev, F. Leme, V. Massol, G. Gregory. JUnit in Action, Second Edition. Manning Publications Co. Greenwich, CT, USA 2010. 1 http://junit.org/ 3

Unit testing tasks This section describes main tasks of unit testing in order of their occurring. What should be tested You have to decide what are you going to test. Briefly, you should test all non-trivial methods of just one object at a time. As a sign of the end you can look at test coverage. Unit-test one object at a time A vital aspect of unit tests is that they are finely grained. A unit test independently examines each object you create, so that you can isolate problems as soon as they occur. If more than one object is put under test, you cannot predict how the objects will interact when changes occur to one or the other. When an object interacts with other complex objects, you can surround the object under test with predictable test objects. Another form of software test, integration testing, examines how working objects interact with each other. Testing conditions 2 Programs should always be tested under two conditions: with expected inputs and with unexpected inputs. This is because while it may work correctly when used correctly, the application will also need to account for errors in case they may occur. Expected inputs are when the client of your application uses it in the exact way it was supposed to. For example, the field requires the user to input a date that is either today s date or a date that occurs in the future. You may test that once you have followed what it is asking you, the program may well work how you intend it to. However, what if you decide to enter a date that is in the past or incorrect date, or just leave the date field empty? Another example is a program for solution of quadratic equations: it may works well for ordinary equations that have roots, but what if the equation is irresolvable - e.g. discriminator is negative number? Or the user may want to solve linear equation and just input zero as a first coefficient - if your program doesn't expect it, it crashes with division by zero. Therefore, it is very important that the application knows how to deal with these kinds of problems and gives the appropriate error message; otherwise you could have some very confused users of your program. Too simple methods If there is a minor utility method, then you might not test it directly. In that case, if the method fail, then tests of the methods that used it would fail. The method would be tested indirectly, but tested nonetheless. If a method is changed so it is not so simple anymore, then you should add a test when that change takes place, but not before. As the JUnit FAQ puts it, The general philosophy is this: if it can t break on its own, it s too simple to break. 2 Taken mostly from http://www.dotnetarchitect.co.uk/blog/12-07-25/unit_testing.aspx 4

Test coverage There are a lot of different metrics for code coverage by tests, see e.g. http://en.wikipedia.org/wiki/code_coverage#coverage_criteria The most basic ones the following: 1. Class coverage 2. Method coverage 3. Line coverage 4. Branch coverage In most cases 100% coverage is not absolutely necessary, but low coverage (usually less than 75%, but it depends) does indicate that tests miss some important logic. There are also a lot of tools for Java code coverage measurement, see http://en.wikipedia.org/wiki/code_coverage#tools_for_java One of the easy, but rather powerful tool is Cobertura 3. It has a plugin for Eclipse and can be easily integrated with Ant, see Appendix. But sure you can use other tools for coverage measurement. Where to create test Put test classes in the same package as the class they test but in a parallel directory structure. You need tests in the same package to allow access to protected methods. You want tests in a separate directory to simplify file management and to clearly delineate test and domain classes. If you need to test a private method, because it is rather complex, then maybe you need to refactor your code and turn this private method into a public method of new class. If the private method is not very complex, you can test it indirectly. Test writing pipeline The general scheme is the following: 1. Instantiate the class to test. 2. Set up the test by placing the environment in a known state (create objects, acquire resources). The pre-test state is referred to as the test fixture. 3. Invoke the method under test. 4. Confirm the result, usually by calling one or more assert methods. Further we discuss some issues that may occur during this pipeline following. Setup environment The @Before and @After annotated methods are executed right before/after the execution of each one of your @Test methods, and regardless of the fact whether the test failed or not. This helps you to 3 http://cobertura.sourceforge.net/ 5

extract all of your common logic, like instantiating your domain objects and setting them up in some known state. You can have as many of these methods, as you want, but beware because in case you have more than one of the @Before/@After methods no one knows what is the order of their execution. JUnit also provides the @BeforeClass and @AfterClass annotations to annotate your methods in that class. The methods that you annotate will get executed, only once before/after all of your @Test methods. Again, as with the @Before and @After annotations you can have as many of these methods as you want, and again nothing is specified about the order of the execution. You need to remember that both the @Before/@After and @BeforeClass/@AfterClass annotated methods must be public by signature. The @BeforeClass/@AfterClass annotated methods must be public and also be static by signature. Each test method must be as clear and focused as possible. This is why JUnit provides you with the @Before and @BeforeClassmethods: so you can share fixtures between tests without combining test methods. However, in preliminaries we postulated that each test should run independently, i.e. in a clean environment. Sometimes there are other considerations to take into account. Performance is a typical one. Test suites may take a long time to execute are a handicap; you will be tempted not to execute them often, which negates the regression feature of unit testing. You must be aware of this tradeoff. Depending on the situation, you may choose to have longer-running tests that execute in a clean environment, or instead tune the tests for performance by reusing some parts of the environment. Choose meaningful test method names You can see that a method is a test-method by the @Test annotation. But you also must be able to understand what a method is testing by reading the name. Although JUnit does not imply any special rules for naming your test methods, a good rule is to start with the testxxxnaming scheme, where XXXis the name of the method to test. As you add other tests against the same method, move to the testxxxyyyscheme, where YYY describes how the tests differ. Don t be afraid that the names of your tests are getting long. It is sometimes not so obvious what a method is testing simply by looking at the assert methods in it so the only chance we have is naming your test-methods in a descriptive fashion and putting comments where necessary. Explain the failure reason in assert calls Whenever you use any of the JUnit assert* methods, make sure you use the signature that takes a String as the first parameter. This parameter lets you provide a meaningful textual description that is displayed in the JUnit test runner if the assert fails. Not using this parameter makes it difficult to understand the reason for a failure when it happens. One unit test equals one @Test method Do not try to cram several tests into one method. The result will be more complex test methods, which will become increasingly difficult to read and understand. Worse, the more logic you write in your test methods, the more risk there is that it will not work and will need debugging. Unit tests give you confidence in a program by alerting you when something that had worked now fails. If you put more than one unit test in a method, it makes it more difficult to zoom in on exactly 6

what went wrong. When tests share the same method, a failing test may leave the fixture in an unpredictable state. Other tests embedded in the method may not run, or may not run properly. Your picture of the test results will often be incomplete or even misleading. Because all the test methods in a TestClass share the same fixture, and JUnit can now generate an automatic test suite, it s really just as easy to place each unit test in its own method. If you need to use the same block of code in more than one test, extract it into a utility method that each test method can call. Better yet, if all methods can share the code, put it into the fixture. Testing exceptions In most cases you should do this by specifying the expected parameter of the @Test annotation, e.g.: @Test(expected = IllegalArgumentException.class) public void getnamewithnullvalue() { MyObj obj = new MyObj(); myobj.setname(null); } Normally the expected parameter in the @Test annotation tells the developers very clearly that an exception of that type should be raised. But you can go even further. Besides naming your test-methods in an obvious fashion to understand that this method is testing an exceptional condition, you can also put some comments to highlight the line of the code which produces the expected exception. However, if your method could throw exceptions of the same type with different error codes or messages, you can use more sophisticated tool - JUnit @Rule annotation with class ExpectedException: @Rule public ExpectedException thrown = ExpectedException.none(); @Test public void getnamewithnullvalue() { thrown.expect(illegalargumentexception.class); thrown.expectmessage('name must not be null'); } MyObj obj = new MyObj(); obj.setname(null); Examples are taken from Nikos Maravitsas' post 4, where also more details could be found. Custom matchers Sometimes you need to compare objects in a non-trivial way, e.g. test string equality ignoring differences in runs of whitespace. Of course, you can write asserttrue() with special boolean function, but you can also use the Hamcrest 5 library, which contains a lot of pre-defined matchers and suggests a convenient way for implementing your own matchers. Such matchers may be required for exception 4 http://www.javacodegeeks.com/2012/10/testing-custom-exceptions-with-junits.html 5 http://code.google.com/p/hamcrest/wiki/tutorial 7

testing: if you want to compare error code, you should implement the corresponding matcher, see an example in Nikos Maravitsas' post mentioned above. Let the test improve the code Writing unit tests often helps you write better code. The reason is simple: A test case is a user of your code. And, it is only when using code that you find its shortcomings. Thus, do not hesitate to listen to your tests and refactor your code so that it is easier to use. For example, an easy way to identify exceptional paths is to examine the different branches in the code you re testing. When you start following these branches, sometimes you may find that testing each alternative is painful. If code is difficult to test, it is usually just as difficult to use. When testing indicates a poor design, you should stop and refactor the domain code. In the case of too many branches, the solution is usually to split a larger method into several smaller methods. Or, you may need to modify the class hierarchy to better represent the problem domain. 8

Automatic running To automate running of tests and reports you can use Apache Ant. Examples of Ant tasks are given below. <target name="compile.unittests" depends="compile.classes" description="compile the unit tests" >  <javac compiler="modern" srcdir="${unit.test}" destdir="${build}" debug="on" encoding="utf-8"> <classpath refid="classpath" /> <compilerarg value="-xlint:unchecked" /> </javac> </target> <target name="instrument" depends="init, compile.classes">  <delete file="cobertura.ser"/> <delete dir="${instrumented.dir}" />  <cobertura-instrument todir="${instrumented.dir}">  <ignore regex="org.apache.log4j.*" /> <fileset dir="${build}">  <include name="**/*.class" /> <exclude name="**/*test.class" /> </fileset> </cobertura-instrument> </target> <target name="unit.test" depends="compile.unittests, instrument" description="test by junit" > <junit fork="on" forkmode="once" printsummary="yes" haltonfailure="yes" showoutput="yes">  <sysproperty key="net.sourceforge.cobertura.datafile" file="${basedir}/cobertura.ser" /> <classpath location="${instrumented.dir}" /> <classpath location="${build}" /> <classpath> <path refid="classpath"/> <path refid="cobertura.classpath" /> </classpath> <formatter type="plain" usefile="false"/> <batchtest todir="${test.report}"> <fileset dir="${unit.test}"> <include name="**/*test*.java"/> 9

</junit> </target> </fileset> </batchtest>  <target name="coverage-report"> <cobertura-report format="html" destdir="${coveragereport.dir}"> <fileset dir="${src}"> <include name="**/*.java" /> </fileset> <fileset dir="${quality.test}"> <include name="**/*.java" /> </fileset> </cobertura-report> </target> 10