Open Source Testing Tools to support Agile Development/Testing

Transcription

1 Open Source Testing Tools to support Agile Development/Testing Dr Paul King ASERT, Australia June 2007 Agile

2 Overview Topics Open Source/Freeware Tools Preview Unit Testing and Coverage Mocking Metrics Acceptance Testing Web Applications and Web Services Further Information Agile

3 The Need for Change in IT? Half of all projects will be late! One third of them will be over budget! 1979 US Government Information Systems 60% schedule overruns 50% cost overruns 45% not useable 29% never delivered 2% could be used as delivered Source: U.S. Gov Accounting Office Various Industries (210 Projects) 40% late by average of 67% 30% over budget by average of 127% 47% couldn t find original numbers Source: 2000 Group CHAOS Report Source: QSM Associates Agile

4 What can Testers do? Change Management? Work Harder? Work Smarter? Find bugs early! Use risk analysis Use better techniques Use better tools Use (useful) metrics What percentage of testers time is spent fixing up obvious (developer) errors? The later an error is found, the costlier. Air Force: $75/instruction during development; $4000/instruction during maintenance. (Boehm, 1976) Cost of finding & fixing errors (testing + maintenance): 40-80% of total development cost. (Kamer, 1993) Agile

5 What can Developers do? extreme Programming (XP) / Agile methods Values Communication Simplicity Feedback Courage Programmer Practices Test-first development Pair programming Refactoring Collective ownership Continuous integration Simple design No testers were harmed in the creation of the Agile Manifesto; where they even consulted? Customer Practices Story telling Release planning Acceptance testing Frequent releases Management Practices Accepted responsibility Support Regular review Mentor Sustainable pace Agile

6 XP Tester Activities Help negotiate quality with the customer Help Clarify stories Help with estimates during planning Advocate the customer s rights Guard the programmer s rights Make sure acceptance tests verify the quality specified by the customer Help the team automate maintainable acceptance tests, using light-weight tools and test designs Make sure test results are reported in a timely manner (continuous feedback) Make sure acceptance testing keeps pace with development Help programmers write more testable code Keep the team honest (if cutting corners is required it must be reported to customer) Source: Testing Extreme Programming, Crispin & House Agile

7 Types of Testing Testing Spectrum Functional/System Testing Simple Functional Testing Task Feature Testing Boundary Testing Forced Error Condition Testing Unit Testing Is each piece behaving? Integration/Module Testing Can I put the pieces together? User Acceptance/Usability Testing Stress/Performance/Load Testing Regression Testing Has anything broken? Compatibility Testing Will it work with various hardware & software configurations? Installation Testing Will the software install/uninstall? Security Testing Certification Testing Is the product Microsoft, Java, Linux, Y2K compatible? Approaches Automated vs Manual Static vs Dynamic (running) Exploratory vs Scripted White-Box vs Black-Box Other QA Techniques Defensive Programming Walkthroughs, Inspections & Code Reviews Coding Standards Code Metrics Iterative Programming Pair Programming/Testing Multiple staged environments Effective Requirements Capture Effective Change Management Defect Tracking Tools Risk Management via Prioritisation Testing Methodologies & Standards Agile

8 Testing Techniques... Boundary Value Analysis Errors tend to happen at boundary values (negative, zero, very large, very small, minimum, maximum) Equivalence partitioning Try to determine sets of equivalent input values and have just one test case for each set Coverage testing Types: branch, basic path, condition Design tests to traverse all/most (important) logical code paths Exploration of informal truth tables can eliminate redundant testing of more complex structures Comparison Testing For mission critical situations, keep 2 or 3 copies of an application running and compare their output Others Coding for Testability Test-Driven Development (TDD) Exploratory Testing Agile

9 Testing Techniques Orthogonal Array Testing Sampling technique to limit the explosion of test cases by identifying important classes of test cases providing maximum coverage with minimum testing Instead of all combinations, systematically define pair-wise combinations of interactions between objects because most faults result from interactions due to two-way interactions Pareto analysis Identify high-frequency causes of problems 20% of causes make up 80% of frequencies so concentrate efforts on 20% Software metrics Fashionable, but sometimes counterproductive Agile

10 Traditional Testing & Software Lifecycle Testing and development activities can be merged within a single process or grouped into separate parallel but inter-linked waterfall-like processes Requirements Use Cases Review Analysis Analysis Models Guided Inspection Architectural Design Architecture Description Guided Inspection Detailed Design Design Models Integration Testing Coding Implementation Unit Testing Development Process Testing Process Agile

11 Testing Techniques (Metrics) Fault Density Software purity level Defect density Estimated number of faults remaining (by seeding) Cumulative failure profile Requirements compliance Fault days number Test coverage Functional or modular test coverage Data or information flow complexity Cause and effect graphing Reliability growth function Requirements traceability Residual fault count Defect indices Failure analysis elapsed time Error distribution (s) Software maturity index Mean time to failure Testing sufficiently Person-hours per major defect detected Failure rate Number of conflicting requirements Software documentation and source listing Software science measures Software release readiness Graph-theoretic complexity Completeness Cyclomatic Complexity Test accuracy Minimal unit test case determination System performance reliability Run reliability Independent process reliability Design structure Combined hardware and software (system) availability Mean time to discover the next K-faults Agile

12 Maturing Ideas on Testing Past You re the new grad, you can be the tester If you can't develop, try testing instead My code doesn't have any bugs, I don't need to test Our manager doesn't do much, get him/her to do the testing We have a head count issue, hire the developer, the tester will have to wait Us vs Them (Testers just slow down the developers) We don't need testers, let the customers debug the code We need to have a web presence NOW - we can fix it and make it faster later Now (Sometimes!) Testing is given a more prominent place within the development lifecycle and is a viable career option extreme Programming (XP) and Agile computing encourage test first, code second A testing framework is considered essential to support refactoring, iterative development and defensive programming Prevention considered better than cure Agile

13 Topics Overview Open Source/Freeware Tools Preview Ant JUnit JUnitPerf Cactus, JUnitEE HttpUnit, Canoo WebTest, MaxQ JMeter, Grinder Metrics & Style Tools Coverage Tools Unit Testing and Coverage Mocking Metrics Acceptance Testing Web Applications and Web Services Further Information Agile

14 Unit Tests/Developers Testing Tools Acceptance Tests/Testers Build Management Tools: Ant/Nant Continuous Integration Mock Objects Coverage Tools Metrics Tools JUnitPerf JUnit HttpUnit Abbot Canoo WebTest JUnitEE Cactus MaxQ Grinder Reporting Tools Features Attributes Code Centric JMeter User Centric Agile

15 Ant Build Tool Originally acronym for Another Neat Tool Analogy with actual ant insect preferred meaning now Features Cross-platform, Extensible Easy to use, Scalable Declarative XML-based build file Alternative to make and others Jam (like make) Amber (original Ant architect) Cons (Perl) Integrates well with JUnit, EJBs, JAR packaging, Databases, FTP/Telnet for deployment,.net users consider Nant <?xml version="1.0"?> <project name="makefile" default="all"> <property name="results.reports value="${results.dir/reports"/> <target name="all"> <javac srcdir="."/> <jar jarfile="project.jar" includes="*.class" /> </target>... <target name="httpunit" depends="deploy_ear,build_httpunit"> <antcall target="db_reset" /> <junit printsummary="false" errorproperty="test.failed" fork="true" failureproperty="test.failed"> <classpath refid="test.webtier.cp"/> <formatter type="brief" usefile="false"/> <formatter type="xml"/> <test todir="${results.data" name="asert.ageapp.webtier.functionaltest"/> </junit> <fail message="one or more test(s) failed. Check log/reports." if="test.failed"/> </target> <target name="clean"> <delete file="project.jar"/> <delete dir="." includes="*.class" /> </target> </project> Agile

16 What is JUnit? Simple framework for writing and running automated tests Includes useful abstractions for running tests including test cases, test fixtures and test suites Test Cases Component Under Test JUnit & Runner Local Environment JUnit package junit.samples; import junit.framework.*; import java.util.vector; import junit.extensions.*;.net users consider NUnit /** * Sample test for <code>java.util.vector</code> */ public class VectorTest extends TestCase { protected Vector fempty; protected Vector ffull; public VectorTest(String name) { super(name); public static void main (String[] args) { junit.textui.testrunner.run (suite()); protected void setup() { fempty= new Vector(); ffull= new Vector(); ffull.addelement(new Integer(1)); ffull.addelement(new Integer(2)); ffull.addelement(new Integer(3)); public static Test suite() { return new TestSuite(VectorTest.class); public void testcapacity() { int size= ffull.size(); for (int i= 0; i < 100; i++) ffull.addelement(new Integer(i)); asserttrue(ffull.size() == 100+size); public void testclone() { Vector clone= (Vector)fFull.clone(); asserttrue(clone.size() == ffull.size()); asserttrue(clone.contains(new Integer(1))); Agile

17 JUnitPerf What is JUnitPerf? collection of JUnit test decorators used to measure the performance and scalability of functionality contained within existing JUnit tests (so non-performance tests are not affected) Used where quantitative performance and/or scalability requirements are critical Supported JUnit test decorators: TimedTest runs a test and measures the elapsed time of the test has a specified maximum elapsed time can stop after maximum elapsed time expires or wait until completion and fail if time was exceeded LoadTest runs a test with a simulated number of concurrent users and iterations load can be ramped by registering a pluggable Timer instance that prescribes a delay between the addition of each concurrent user ThreadedTest A test decorator that runs a test in a separate thread. Unit Testing of Performance Requirements Agile

18 JUnitEE Simple Unit JUnitEE ( Testing in Extension to JUnit another JVM Allows tests to be run inside application server Hence tests run in the production environment Not as comprehensive as Cactus but simpler Browser JUnitEE Run Manager Test Cases Server Component Under Test JUnit Server Environment Agile

19 Cactus Complex Unit Testing in another JVM Cactus Simple test framework for unit testing server-side java code E.g. Servlets, EJBs, Tag Libraries, Filters,... Aims to lower the cost of writing tests for server-side code HttpUnit tests too tedious Not discriminating enough Supports code logic unit testing often used with Mock Objects integration unit testing sometimes called in-container testing functional unit testing when used with HttpUnit Redirector Proxy Cactus Test Case Cactus Test Case JUnit & Cactus Server Component Under Test JUnit & Cactus Client Environment Server Environment Agile

20 EMMA Code Coverage Tools emma.sourceforge.net Clover Commercial, free to Open Source Projects NoUnit Fast, crude.net users consider NCover Agile

21 Metrics Tools JavaNCSS, JCSC, CheckStyle, PMD, FindBugs, JDepend All provide various metrics Give you a guide to the health of your application Agile

22 Programmatic Testing of Web Content HttpUnit Open source Java API for accessing web sites without a browser Emulates the relevant portions of browser behaviour, including form submission, basic http authentication, cookies and automatic page redirection, and allows Java test code to examine returned pages as text, an XML DOM, or containers of forms, tables, and links Ideally suited for automated unit testing of web sites when combined with a Java unit test framework such as JUnit HtmlUnit Similar to HttpUnit but different implementation Slightly better support for JavaScript /** * Navigate to & enter an Enquiry **/ public void testenquiryentry() throws Exception { WebConversation linkcon =new WebConversation(); linkcon.setauthorization(cso1_id,cso1_pass); WebResponse menupage = this.getpage(linkcon, ELF_MENU_PAGE); assertmenupagecso (menupage, CSO1_id); WebLink enqlink = menupage.getlinkwith("add Enquiry"); WebRequest enqlinkrequest = enqlink.getrequest(); WebResponse enqpage = linkcon.getresponse(enqlinkrequest); assertenqpage (enqpage); enquirya = makeenquirysamplea(); assertthisisatypesamplea (enquirya); WebResponse confirmpage = enterenquiry (linkcon, enqpage, enquirya); String recordid = assertenqconfpage (confirmpage); menupage = clickbutton(linkcon, confirmpage, "ElfHomeForm", "ElfHomeLink"); assertmenupagecso (menupage, CSO1_id); WebResponse logoutresult = clickbutton(linkcon, menupage, "LogoutForm", "LogoutLink"); assertlogoutpage (logoutresult); Agile

23 Programmatic Testing of Web Content JWebUnit Slightly improved interface to HttpUnit Tigris MaxQ An open source web functional testing tool; includes: Includes an HTTP proxy that records your test script (automatically stores variables posted to forms) Includes a command line utility that can be used to playback tests Built on a number of existing free software packages: Jython, JUnit, HTTPClient PushToTest TestMaker Similar to MaxQ Supports HTTP, HTTPS, SOAP, XML-RPC, SMTP, POP3, IMAP protocols 1: # imports 2: from com.bitmechanic.maxq import HttpTestCase 3: from junit.textui import TestRunner 4: from java.utilimport HashMap 5: 6: # defintitionof test class 7: class MaxQTest(HttpTestCase): 8: def init (self): 9: HttpTestCase. init (self, "") 10: 11: def runtest(self): 12: self.get(" 13: self.assertequals(200, self.getresponse().getstatuscode()) 14: 15: self.get(" 16: self.assertequals(304, self.getresponse().getstatuscode()) 17: 18: 19: map = HashMap() 20: map.put("hl", "en") 21: map.put("q", "testing+tools") 22: self.get(" map) 23: self.assertequals(200, self.getresponse().getstatuscode()) 24: 25: ########################################## 26: 27: # Code to load and run the test 28: test = MaxQTest() 29: test.runtest() Jython Example (MaxQ) Agile

24 Declarative Testing of Web Content Canoo WebTest Open source tool for automated testing of web applications Declarative approach Has Click-O-Mat robot Test result viewer <target name="login" > <testspec name="normal" > &config; <steps> <invoke stepid="get Login Page" url="login.jsp" /> <verifytitle stepid="we should see the login title" text="login Page" /> <setinputfield stepid="set user name" name="username" value="scott" /> <setinputfield stepid="set password" name="password" value="tiger" /> <clickbutton stepid="click the submit button" label="let me in" /> <verifytitle stepid="home Page follows if login ok" text="home Page" /> </steps> </testspec> </target> Agile

25 TagUnit Other Web-Related Unit Testing Tools Checking Basics taglib uri="/tagunitcore" prefix="tagunit" %> <tagunit:testtaglibrary name="jsp Standard Tag Library (core)" uri="/jstl-core"> <tagunit:taglibrarydescriptor jar="standard.jar" name="c.tld"/> </tagunit:testtaglibrary> Checking Definitions <tagunit:assertbodycontent name="jsp"/> <tagunit:assertattribute name="value" required="false" rtexprvalue="false"/> <tagunit:assertattribute name="var" required="false" rtexprvalue="false"/> <tagunit:assertattribute name="scope" required="false" rtexprvalue="false"/> <tagunit:assertattribute name="target" required="false" rtexprvalue="false"/> <tagunit:assertattribute name="property" required="false" rtexprvalue="false"/> Checking Content Generation <tagunit:assertcontent name="simple loop"> <tagunit:actualresult><c:foreach var="i" begin="1" end="3">x</c:foreach></tagunit:actualresult> <tagunit:expectedresult>xxx</tagunit:expectedresult> </tagunit:assertcontent> Checking Attribute Setting <c:set var="myvar" scope="request" value="hello World!"/> <tagunit:assertpagecontextattribute name="myvar" type="java.lang.string" scope="request" value="hello World!"/> StrutsTestCase Supports mock and in-container tests Browser Framework JSPs Taglib Server Environment TagUnit Agile

26 GUI Acceptance Testing Tools Fat Client Testing Abbot (A Better BOT) ( Automated GUI testing framework/harness supporting Robot (which generates user events) Testers (provide component-specific user actions & tests) Scripts (basic units of test execution) Script Editor (Costello - allows manual creation & maintenance of scripts) Recorders (allows scripts to capture high-level semantic information instead of basic mouse and key events) Mozilla.org Jemmy Java library that is used to create automated tests for Java GUI applications Agile

27 JMeter What is Apache JMeter? Performance Testing Java desktop application designed to load test functional behavior and measure performance Originally designed for testing Web Applications but has since expanded to other test functions Agile

28 Grinder Asimple but useful load testing tool G3 uses Jython-based scripting language Basic functionality requires no programming Includes Console, Agents, Proxy Recorder Agile

29 Other Performance Tools Microsoft Web Application Stress testing tool (WAS) Download and install instructions can be found at: Useful for Java Web Apps too! Supports a Macro recording mode which lets expert-user tests be recorded and re-run Agile

30 Other Performance Tools HPjmeter ( Not to be confused with Jakarta Apache Jmeter Performance analysis tool which measures: method call counts, method execution times, method call graph, number of allocated objects, allocation sites, object reference graph, lock contention Agile

31 Other Performance Tools Mike's Java Profiler (mjp.sourceforge.net) MJP is a performance profiler for Java programs. It collects statistics about accumulated CPU time used by each stack frame in the profiled program and presents the collected data in a GUI. The data can be used to locate CPU bottlenecks within the profiled program. Extensible Java Profiler (ejp.sourceforge.net) EJP is a Java profiling tool that enables developers to test and improve the performance of their programs. EJP measures and reports the time spent in different parts of a program allowing you to identify any code making heavy use of the CPU. Agile

32 JBuilder X Enterprise IDE Support File New gives these options GEL ( Agile

33 IDE Support JDeveloper 9i/10G JUnit Separate Download CodeCoach provides advice Execution Profiler, Memory Profiler, Event Profiler Eclipse/WSAD Plugin for JUnit Built in Test Environment Studio Profiling Agile

34 Continuous Integration Other Topics Daily ( wimps ) or more frequent builds to ensure that no-one breaks the build Cruise Control, Anthill, Tableaux Mock Objects For unit testing, replace complex objects with simple shells EasyMock, MockMaker Fancy Tools Jester: makes arbitrary mods to your code and re-runs tests JML, Spec#, Nice: allow you to specify pre & post conditions Getting data XmlTestSuite, jtestcase Testing Databases DbUnit, SqlUnit, P6Spy Watching Brief Eclipse open source Hyades project evolving to include much Rational functionality <test name="testing putting setup calls in prepare"> <prepare> <sql><stmt>delete from department where deptno=1</stmt></sql> </prepare> <sql>... </sql> <result>... </result> </test> class ArrayList { void Insert(intindex, object value) requires 0 <= index && index <= Count otherwise ArgumentOutOfRangeException; requires!isreadonly &&!IsFixedSize otherwise NotSupportedException; ensures Count == old(count) + 1; ensures value == this[index ]; ensures Forall{inti in 0 : index ; old(this[i]) == this[i]; ensures Forall{inti in index : old(count); old(this[i]) == this[i + 1]; { Agile

35 Overview Topics Open Source/Freeware Tools Preview Unit Testing and Coverage Mocking Metrics Acceptance Testing Web Applications and Web Services Further Information Agile

36 What is Unit Testing? Automated developer testing Capture intent Give confidence that code works (all the time) Support incremental changes (run all tests) Support refactoring (rerun all tests) White box tests (state or interaction based) When do I use it? As part of Test-Driven Development (TDD) you write tests before you write code You only write code to make a failing test pass For legacy code or if developing code first Every public method call should have a unit test Write enough unit tests written to achieve coverage goals Test everything that could possibly break Use test generation tools Agile

37 What is JUnit? A regression testing framework Allows coding of unit tests in Java Various test runners automate running test suites Written by Erich Gamma and Kent Beck Open Source Software released under the IBM Public License and hosted on SourceForge Why JUnit? Already exists, so you don t have to reinvent the wheel Open source, free for all uses Lots of examples and the framework is heavily used Separates testing code from product code Easily integrated into build process Ant, a popular build tool, has a built-in target for JUnit Lots of extensions to the base JUnit framework JUnitPerf, Cactus, JUnitEE, HttpUnit, etc. Agile

38 JUnit Example Class Under Test Test Class Naming conventions used package asert; public class Calculator { public int twice(int value) { return value * 2; Eclipse Runner package asert; import junit.framework.testcase; public class CalculatorTest extends TestCase { /* * Test method for 'asert.calculator.twice(int)' */ public void testtwice() { Calculator mycalc = new Calculator(); assertequals(8, mycalc.twice(4)); Agile

39 Failure: Expected JUnit Terms Test failed so program (or test) needs fixing Error: Unexpected Program bombed out with exception (normally means that the test should be re-written to catch exception and fail) TestCase: Collection of method tests Test Fixture: Object Reuse TestSuite: Collection of Test Cases TestRunner: Interface AWT, Swing, text Agile

40 Java Testing Frameworks JUnit, JTiger, TestNG, Artima TestRunner Extensions: Surrogate, JUnitX, JTR, TESTARE, Jameleon, Cactus, JUnitEE.Net Testing Frameworks NUnit, MbUnit, VSTS IDEs Java IDEs, VSTS Testing Frameworks... Agile

41 ...Testing Frameworks Requirements to Test Mapping JRequire (limited) Test Generation TestGen4J Early days JTest, Agitar Object Repository Agile

42 Configuring JUnit Add the junit.jar file to the CLASSPATH for the JVM hosting the test environment Might be automatic in many IDEs Might be complex in some multi-tier scenarios Agile

43 Development Steps (JUnit 3) 1. Write TestCases Class should extend TestCase Create an appropriate constructor if using older versions of JUnit Write setup() method if required Initialise any shared variables Write Test Methods containing Assert methods Write teardown() method if required Dispose of any resources 2. Place TestCases in TestSuite 3. Invoke run() method of runner textual TestRunner fast to launch and can be used when you don't need a red/green success indication graphical TestRunners provide graphical feedback and progress indication Agile

44 Development Steps (JUnit 4) 1. Write TestCases Class doesn t extend TestCase annotations instead of setup() and teardown() naming conventions annotation instead of methods named testxxx() for exceptions for timeouts Can have parameterised tests 2. Place TestCases in TestSuite 3. Invoke run() method of runner Agile

45 Extending TestCase import junit.framework.*; import junit.extensions.*; public class SomeTestCase extends TestCase { public SomeTestCase(String testname) { super(testname); // constructor required in JUnit 3.7 and earlier public void testdothisfirst() {... public void testdothissecond() {... Agile

46 setup() LifeCycle methods The place to create common fixtures used by multiple tests provides alternative to repeating test initialization code is called before each test teardown() Called after each test Use it to free resources Probably do not need it Agile

47 Example: setup() package asert; import junit.framework.testcase; public class Calculator2Test extends TestCase { package asert; private Calculator2 fcalc; public class Calculator2 { public int twice(int value) { return value * 2; public int half(int value) { return value / 2; protected void setup() throws Exception { super.setup(); fcalc = new Calculator2(); /** Test method for 'asert.calculator2.twice(int)' */ public void testtwice() { assertequals(8, fcalc.twice(4)); assertequals(10, fcalc.twice(5)); /** Test method for 'asert.calculator2.half(int)' */ public void testhalf() { assertequals(8, fcalc.half(16)); assertequals(10, fcalc.half(20)); Agile

48 Add Test Cases public void testadd() { Complex result = c1.add(new Complex(5,3)); assertequals(result.get_r(),c2.get_r()); assertequals(result.get_i(),c2.get_i()); public void testequal(){ asserttrue(!c2.equal(c1)); asserttrue(c1.equal(new Complex(7,3))); public void testmoneyequals() { asserttrue(!f12chf.equals(null)); assertequals(f12chf, f12chf); assertequals(f12chf, new Money(12, "CHF")); asserttrue(!f12chf.equals(f14chf)); Agile

49 Assertions Provided by junit.framework.testcase Which extends junit.framework.assert Available methods: assert(bool) deprecated/deleted form of asserttrue() asserttrue(bool) test passes if bool expression is true assertfalse(bool) reverse of above assertequals(exp, act) checks expected object equals actual (by calling objects' equals method) assertnull(ob) test passes if ob is null assertnotnull(ob) reverse of above assertsame(exp, act) compares actual object references assertnotsame(exp, act) reverse of above fail() test fails (commonly used with exceptions) Variations assertequals has versions for all primitive types, Objects and Strings; float/double versions have a delta (rounding allowance) All assertions have an optional initial message argument, e.g. assertequals("should be start of week", "Monday", today); Agile

50 Running Multiple Tests --TestSuite Create a suite manually Useful if you want to run only a subset of your test cases suite.addtest(new MoneyTest("testMoneyEquals")); Let TestRunner extract a suite Much easier. Runs all the test automatically Searches for methods that start with test Can add new test with out having to update suite TestSuite suite = new TestSuite(MoneyTest.class); Agile

51 Test Case for a Thrown Exception Catch the exception and if it isn't thrown call the fail method. Fail signals the failure of a test case. public void testindexoutofboundsexception() { Vector v = new Vector(10); try { Object o = v.elementat(v.size()); fail("should raise an ArrayIndexOutOfBoundsException"); catch (ArrayIndexOutOfBoundsException e) { Agile

52 JUnit tips and tricks Keep test cases with the source code it exercises Production builds will avoid compiling and packaging the tests with the production classes Apache Jakarta Ant can easily filter the test case classes out if a standard naming convention is used Avoid writing tests that have side effects Tests may affect data that other tests rely on Cannot repeat the tests without manual intervention Especially prevalent in testing objects which interact with a database for their state (i.e. EJBs) Extra functionality and planning needs to included when building repeatable, robust EJB test cases Agile

53 JUnit tips and tricks Consider some of these Document test cases in javadoc Keep tests small and fast Test one and only one thing per test Test, TestCase, and TestSuite are an implementation of the Composite design pattern, so it s very easy to build up a hierarchy of test cases and suites, including a root test suite which will cause all contained test cases to be run Agile

54 JUnit tips and tricks When the domain model prohibits unit testing, use the Mock Objects testing pattern to emulate the domain model A Mock Object is a substitute implementation to emulate or instrument other domain code Helpful to use an interface to allow a plugin point for different implementations (mock vs. real) Particularly useful for testing the edges of the system (i.e. integration to a mainframe or other datasource for which no testing can be done) Agile

55 Handling Test Data Approaches to consider Hard-coded in tests Java System Properties Environment variables Cumbersome for large amounts of data In a (properties, XML, Message bundle, CSV or Excel) file Consider packaging the test data with the test and load it from the CLASSPATH -don t load it from the hard-coded file system location. Use JUnitPP Uses properties-based test data behind more robust facade Use JXUnit, jtestcase, Jameleon Builds suites of test data marked up in XML Use TestGen4J Use a database Ant <sql> task, SqlUnit, DbUnit Remember to clean up all test data once finished. Build this functionality into a base class for all database test cases to use. Agile