Test-Driven Development Revisited

Transcription

1 1 Test-Driven Development Revisited Ricardo Moutinho, Tiago Boldt Sousa FEUP, Universidade do Porto Rua Dr.Roberto Frias s/n Porto, Portugal (+351) Fax (+351) Abstract Test-Driven Development allows agile teams to reduce cost and effort on software development. This paper reviews the TDD process, how and why to implement it, highlighting common pitfalls and advantages of using it. Index Terms Agile Manifesto, TTD, Testing, Refactoring, Kent Beck I. INTRODUCTION Test-Driven Development, commonly abbreviated to TDD, is a software development process, consisting in small development iterations intended to avoid software errors and keep the developers focused in their work. TDD aims at reducing programming errors by specifying each feature to be implemented with executable tests, while reducing testing costs and improving code quality and specification by having the programmers to write the tests to what will be implemented later. II. HISTORY Test Driven Development was first introduced with the test-first programming concept of extreme programming in 1999[17]. It was (re) discovered by Kent Beck who popularized it in his book EXTREME PROGRAMING[4]. Later, he wrote a second book on TDD, named TEST DRIVEN DEVELOPMENT: BY EXAMPLE, where he set two main rules for it: 1) Code should only be written when a test fails; 2) Any (code) duplication should be eliminated. Beck explains that these two rules imply complex individual and group behavior, quoting: You design organically, with the running code providing feedback between decisions. You write your own tests because you can t wait 20 times per day for someone else to write them for you. Your development environment must provide rapid response to small changes (e.g you need a fast compiler and regression test suite). Your designs must consist of highly cohesive, loosely coupled components (e.g. your design is highly normalized) to make testing easier (this also makes evolution and maintenance of your system easier too). For developers, the implication is that they need to learn how to write effective unit tests. Beck s experience is that good unit tests: Run fast (they have short setups, run times, and break downs). Run in isolation (you should be able to reorder them). Use data that makes them easy to read and to understand. Use real data (e.g. copies of production data) when they need to. Represent one step towards your overall goal. III. USING TDD Incorrectly, software testing in only executed after development with an intuitive attempt to find bugs. Many times this is done by a different person from the one programming the feature. This testing approach is costly and faulty. By adopting TDD, the programmer also becomes the tester. In order to do so, he is fully responsible for understanding the feature to be implemented. He then must write tests, as atomic as possible, to each component to be developed. By doing that, he s becoming aware of the system requirements, getting a better overview of how the system should behave and getting himself better prepared to implement the component. The next section will further detail this process. A. Gathering the Requirements Before starting an iteration, there must be an overview of the project requirements. There are many approaches to requirements specification. One agile approach is to have user stories describing the system. An user story is a description of a feature in a non-technical language. The best option is to have the customer writing them himself. Each user story should be written in a small paper card, making them small and easy to understand and implement. Bigger requirements should be split into several user stories. In short, user stories are very slim and high-level requirements artifacts[3]. Each one should be tagged with a priority and cost estimation value. There are many available software tools for user story management. Still, one of the most adopted method is to have a wall were all stories are posted and iteratively removed once implemented.

2 2 Having user stories allows the developers to know exactly what the client expects from the product and to keep themselves focused on their work. Once implemented, each user story can be archived, giving a visual feedback to the team of what s already been implemented. Using user stories is not needed in TDD but common in agile teams[6]. B. Iterations Each feature to be implemented is an iteration for the developers involved. The Test-Driven Development process is composed by five steps, as described bellow.[5] TDD does not impose how the team organizes itself, allowing other agile methods like pair-programming to be used, which can also increase efficiency.[18] 1) Writing the Test: Every implementation starts by writing a test. To do so, the developer must be fully aware of the new feature s requirements. This is the main difference from other methods, the developer analyses all requirements needed by the implementation before starting it, because that s needed to write a good unit test, which already defines the implementation s structure. At this point, external documentation, like use cases or user stories, are used to extract the specification for what s expected from the new feature needed to write the tests. 2) Validating Tests: Once all tests are written, there is the need to verify if they fail. At this point all the tests should fails since there s no code implementing them. Writing a valid test at this point means one of two things: TDD is not being properly implemented since code that implements the new feature already exists; the test is faulty, since it is always valid. Once the feature s specification is fully covered by tests the programmer can continue to the next step. 3) Write Code: In this step, the developer analyses the test and writes code in order to make it pass. The code does not need to be perfect at this step, instead, it should be a quick, but valid, implementation that makes the system behave as expected, even if there s room for later improvement on it. The developer should always have the tests in mind in order to keep himself focused. There s no need to have the implementation doing more than what the test expects since that would be leaving the scope of the feature. 4) Test the Implementation : Now that the implementation is complete, it s time to validate it. The test fits this purpose and should now be executed. Many tools can be used in this step (see section 8) but using a framework for automating test execution is essential. When a test fails to validate, the implementation should be reviewed and fixed. This last step should be repeated until all tests are successfully accepted by the application. 5) Refactoring: At this point we have a valid implementation of the requirements according to the written tests. However, as previously mentioned, it s not relevant to achieve the best implementation at this point, only a functional and working one, making the developer work faster in order to achieve results. To improve the code, programmers can now refactor it, making it aesthetically cleaner, removing duplicates, speed up implementation or any other improvement available, keeping in mind that the implemented behavior should not be altered in any way. Once refactored, to guarantee that no errors were introduced, the tests should be executed once more. If they validate, the feature can be marked as completed. C. Next Iteration Once an iteration is completed, the developer can start working on a new feature. At this point he should get back to the first step, choose what to implement next and repeat the whole process once again. Figure 1. A. Why TDD Test-Driven Development Cycle IV. ADOPTING TDD TDD can be implemented on almost any project, as long as there s a way to write tests for it. Implementing it in greenfield projects will probably achieve better results, since there are no constrains imposed by previous work. A significant advantage of TDD is that it enables you to take small steps when writing software. This is a practice that I have promoted for years because it is far more productive than attempting to code in large steps. For example, assume you add some new functional code, compile, and test it. Chances are pretty good that your tests will be broken by defects that exist in the new code. It is much easier to find, and then fix, those defects if you ve written two new lines of code than two thousand. The implication is that the faster your compiler and regression test suite, the more attractive it is to proceed in smaller and smaller steps. [4] B. Advantages TDD prevents many common mistakes in software development. It allows teams to reduce costs and development time while increasing software quality. A study[16] states that: 87.5% of developers reported better requirements understanding.

3 3 95.8% of developers reported reduced debugging efforts. 78% of developers reported TDD improved overall productivity. 50% of developers found that it decreased overall development time. 92% of developers felt that TDD yielded high-quality code. 79% of developers believed TDD promoted simpler design. These very positive results highlight that the better understanding of the requirements by developers using TDD allows them to produce higher quality code, reducing the overall required development time. The early and frequent nature of the testing helps to catch defects early in the development cycle, preventing them from becoming endemic and expensive problems. Eliminating defects early in the process usually avoids lengthy and tedious debugging later in the project. [5] C. Disadvantages Test-Driven, as any other technique, is not perfect. The methodology works great with application logic. On the other hand, on more specific cases where there are graphical interfaces, databases or very specific network configurations, TDD might not be easy to use. For those cases, the logic code should be inside a testable library, using mocks (see 5.5) to represent the outside of the logical scope. There might still be a small portion that will not be unit tested but the most of it can be verified this way. Maintaining the tests is an overhead to the traditional software development process. It is justified by reducing the need for documentation and for increasing code quality and development speed but the extra effort might not be welcomed by some teams. In those cases, management influence is essential to motivate and get the team to understand and believe how TDD is helpful for them. Another TDD problem is the possible false sense of security to the developers when all tests validate. A hole or bad design in the tests could end up in a working, but still defective, product. Quality assurance validation should always be enforced.[5], [4] D. How is TDD Agile In order to understand why TDD fits the Agile way of creating software, first we must have in mind that to be Agile is more than just skipping documentation. The Agile Manifesto[1] set four golden rules to improve software development: 1) Individuals and interactions over processes and tools 2) Working software over comprehensive documentation 3) Customer collaboration over contract negotiation 4) Responding to change over following a plan TDD, as a software development process, fits into the second and forth statements from the manifesto. TDD provides working software over comprehensive documentation since its purpose is to actually deprecate superfluous documentation that most of the times it isn t even read. Writing small and comprehensive tests is a much cleaner way of describing what the system should do exactly. TDD also fits the responding to change statement, since it provides an easy process to insert modifications in the system. Once the requirements change, the first step would be to rewrite the correspondent test, making it compliant with the new requirements. By doing so, the test will fail and the programmer will then fix the code so that, once again, the tests validate, making the application compliant with the new requirements. A. Writing good tests V. WORKING WITH TESTS Writing good tests is as important as writing good code. A good test will be easy to understand and will correctly validate the tested portion of the software. Bad tests usually do not cover the whole specification, are ambiguous or hard to understand, increasing the complexity for later code implementations. In order to keep the tests maintainable and clean, they should be developed with the same care and quality as the rest of the code. B. Writing Tests One can start writing tests for a project either top-down or bottom-up[7]. The terminology describes the top as being the user interface or output of the application and the bottom as its inner core with business logic and algorithmic implementations. In TDD, choosing the testing method implies the development process. 1) Bottom-up : Testing bottom-up will guarantee that base components are developed first. Since these have no dependencies, working software will be available sooner. On the other hand, when writing tests for core components it s hard to know exactly how will the higher level components access these. This approach usually forces the programmer to keep coming back to previous tests and alter them, and the corresponding implementation, to comply to higher level components requirements. 2) Top-down: Starting by writing tests to the higher level of the application is a good choice since it allows the developer to focus exactly on the feature to implement and write a test specifying exactly what s expected from the component, without worrying how it will be implemented. The disadvantage of this approach is that there are no lower level components and to run the tests on produced code the user will have to create mock components to simulate each lower layer. This can be time consuming. On the other hand, unlike with bottom up, the programmer will focus first on what the system should do and only later worry on how it is done. This method can be known as Behavior Driven Development. It might take more time until all the layers are functional but it guarantees that what s being developed complies with the specifications since the top level, what the end user will interface with has the exact expected behavior and is not influenced by any previous work.

4 4 C. Testing at different levels In order to implement TDD, programmers must know how to test the entire scope of the application. There are many test types, each specific to each need: 1) Unit Tests : In computer programming, unit testing is a method by which individual units of source code are tested to determine if they are fit for use. A unit is the smallest testable part of an application. In procedural programming a unit may be an individual function or procedure[13]. This is the most common test and the base of the TDD process. 2) Integration Tests : Integration testing is the phase in software testing in which individual software modules are combined and tested as a group. It occurs after unit testing and before system testing. Integration testing takes as its input modules that have been unit tested, groups them in larger aggregates, applies tests defined in an integration test plan to those aggregates, and delivers as its output the integrated system ready for system testing. [14] 3) System Tests: System testing is performed on the entire system in the context of a Functional and System Requirement Specifications. System testing is an investigatory testing phase, where the focus is to have almost a destructive attitude and tests not only the design, but also the behavior and even the believed expectations of the customer. It is also intended to test up to and beyond the bounds defined in the software requirements specification in an attempt to find existing bugs. [15] 4) Acceptance Tests : Also known as functional tests, acceptance tests are the final tests to verify if a feature complies to its requirements. These are higher level tests and can be even executed by the client. If all acceptance tests clear, the product is ready for release. D. Code visibility Tests should always cover the entire code produced, which can be analyzed with code coverage tools[9]. Although, in Object Oriented programming, private code is very common within a class. Unit tests are unable to directly cover those functions. There are workarounds to it, like creating a testing function inside the class itself with code to test the private functions. Such a class would be removed on production code. There is some debate among practitioners of TDD, documented in their blogs and other writings, as to whether it is wise to test private and protected methods and data anyway. Some argue that it should be sufficient to test any class through its public interface as the private members are a mere implementation detail that may change, and should be allowed to do so without breaking numbers of tests. Others say that crucial aspects of functionality may be implemented in private methods, and that developing this while testing it indirectly via the public interface only obscures the issue: unit testing is about testing the smallest unit of functionality possible[10], [11]. E. Using Mocks Unit tests should be written to validate a small unit of code and not entire systems. When writing a test that covers a large chunk of code, it can be hard to understand its objective and to detect why it is failing when something goes wrong. Still, programmers tend to write test with wider scopes because it can be hard to test code with external dependencies. In object-oriented programming, mock objects can be used to simulate the behavior of real unimplemented objects. A computer programmer typically creates a mock object to test the behavior of some other object, in much the same way that a car designer uses a crash test dummy to simulate the dynamic behavior of a human in vehicle impacts[12]. Mock objects can log the interaction with the application so that the user can understand exactly how are the components communicating. Since the mocked object is a temporary representation of a future feature, it is usually written with no logic in it so it is usually tied to a specific test case. F. Continuous Integration To reduce the time spent on executing each batch of tests, continuous integration systems allow the team to have the code built and all the tests executed against it on a scheduled basis. Such a system can be of use to a team using TDD since it can monitor regressions in the code. Whenever something breaks a test will fail and a report generated. Also, continuous integration systems could be used to run integration tests. These are tests written only to validate the communication between components in a software application. The TDD approach would validate each individual component having integration tests validating how they work together, all monitored by the continuous integration system. VI. DOCUMENTATION It s common for a programmer when embracing a new job to acknowledge the system by reading its code, instead of the existing documentation. To understand how a certain module works and integrates the system, it s also normal to see how its functions and classes are invoked and what s the expected output from them. This approach normally works on helping them to understand the system s structure. TDD embraces this behavioral pattern and states that a well-written unit test can be used in this exact way. A call to a function or instantiation of a class, within a small scope, can help the programmer to understand how a feature should behave and how to use it. Using the TDD process, every feature will then be documented even previous to its implementation, resulting in a win-win situation for both developers and stakeholders. System requirements and documentation for each individual and atomic feature are automatically documented both on what the code should be doing, but also on how it should be interacted with. Although important and reliable, these tests are most likely not enough documentation for a big project, but a complement to the existing documentation as a relevant description of what the system does and how it does it.

5 5 VII. TOOLS Software testing goes way beyond TDD. It is a necessary task that must be done continuously alongside code development to guarantee that implementations are according to what s expected in any project. Teams must be aware that any minor code patch might break any other feature. Using automated tests reduce the need for human effort in testing, which implies a reduction on the overall verification and validation cost. These tests can be written as a different application that interacts with the main project but this solution would be too complex to maintain. To simplify testing execution, automatic testing frameworks were developed. These are an essential part of any TDD project. The testing framework selection should be part o the technology selection phase on TDD projects. [12] Collaborative authorship. Mock Object. Wikipedia, the Free Encyclopedia. Retrived November 2, 2010 from [13] Collaborative authorship. Unit Testing. Wikipedia, the Free Encyclopedia. Retrived November 5, 2010 from [14] Collaborative authorship. Integration Testing. Wikipedia, the Free Encyclopedia. Retrived November 5, 2010 from [15] Collaborative authorship. System Testing. Wikipedia, the Free Encyclopedia. Retrived November 5, 2010 from [16] Hawley, Matt. TDD Research Findings. April 15, Blog Post. Retrived November 5, 2010 from [17] Copeland, Lee. "Extreme Programming", Computerworld. December Blog Post. Retrived November 5, 2010 from taxonomyid=063 [18] Xu, Shaochun;Rajlich, Vaclav. Empirical Validation of Test-Driven Pair Programming in Game Development. IEEE Computer Society VIII. CONCLUSION Test-Driven Development allows software development teams to write better code while reducing the development time. Despite the small initial investment and training, the process is rather simple to adopt and will guarantee that developers understand better the system requirements, making them better prepared to develop the application s code. TDD can be combined with other techniques and might even continue to evolve itself into new ones but it s essential idea of test-first that started with extreme programming in 1999 is a proven theory and should be considered in any software project. REFERENCES [1] Beck, Kent; et al.. "Manifesto for Agile Software Development". Agile Alliance Retrieved November 3, 2010 from [2] Scott W. Ambler. How Agile Are You? 2010 Survey Results. Ambysoft Inc Retrieved November 3, 2010 from [3] Scott W. Ambler. Introduction to User Stories. Ambysoft Inc Retrieved November 3, 2010 from [4] Scott W. Ambler. Introduction to Test Driven Design (TDD). Ambysoft Inc Retrieved November 3, 2010 from [5] Collaborative authorship. Test-Driven Development. Wikipedia, the Free Encyclopedia. Retrived November 2, 2010 from [6] Exceedhl TDD from starting from user stories a top-down style. Blog Post. August 2, Retrieved November 3, 2010 from [7] Ryan Kinderman. Testing on High: Bottom-up versus Topdown Test-driven Development. Blog Post. November 19, [8] Collaborative authorship. Behavior Driven Development. Wikipedia, the Free Encyclopedia. Retrived November 2, 2010 from [9] Collaborative authorship. Code Coverage. Wikipedia, the Free Encyclopedia. Retrived November 2, 2010 from [10] Burton, Ross. "Subverting Java Access Protection for Unit Testing". O Reilly Media, Inc.. November 12, Retrieved from [11] Newkirk, James. "Testing Private Methods/Member Variables - Should you or shouldn t you". Microsoft Corporation. June 7, Retrieved from