Introduction to Parasoft Tracer. A Demonstration of How to Use Tracer Technology While Extending an Existing Application

Transcription

1 Introduction to Parasoft Tracer A Demonstration of How to Use Tracer Technology While Extending an Existing Application

2 1 What is Parasoft Tracer? 1.1 Overview This paper explains how a development team responsible for extending an existing application can use Parasoft Tracer technology to verify whether the new functionality works correctly now and in the future, as the application continues to evolve. Tracer technology is the first and only technology that provides software development and QA organizations an effective and practical means to deploy unit testing as a standard practice in their development lifecycle, extending their ability to ensure secure and reliable applications. Although the value of unit testing has long been respected, its practice has not been widely embraced by development organizations. Why? Because the time, skill, and resources to develop unit test cases are often in short supply, given the increasingly aggressive development schedules that software development organizations face. Significant adoption requires automation. Parasoft Jtest has been providing Java developers with automated unit testing and code analysis for years. With Tracer technology, Jtest now extends automated unit testing to a wider audience and addresses the complexity of generating functional unit test cases that verify whether code satisfies the required application behavior. Parasoft Tracer automatically generates unit test cases that capture application behavior by monitoring a running application. Simply exercise the application functionality you want to test, and Parasoft Tracer automatically designs JUnit test cases with real data that represents the paths taken through the application. No coding or scripting is required. The result is a library of test cases against which new code can be tested to ensure it meets specifications and does not break existing functionality. Parasoft Tracer benefits include: Fast and easy generation of an intelligent low-maintenance functional unit test suite. Anyone in QA or development can automatically create realistic JUnit test cases by simply exercising the application like the end-user would. Because the test suite operates at the code level rather than the UI level, it is sensitive enough to detect even the subtle internal problems that UI testing overlooks (for instance, failure to send purchase-related data to a database). With minimal overhead, development and/or QA can now readily use unit testing to identify, isolate, and resolve software errors early in the development process. Automated creation of a thorough regression test suite against which to validate and assess the impact of new code changes. Developers can verify new code's functionality as soon as it's completed (when it's fresh in their minds), then the team can run the generated test cases nightly to determine whether new or modified code impacted previously verified functionality. If problems are discovered, they are traced to a specific class/method and reported immediately. The team can then quickly correct each problem as soon as it's introduced before the problem becomes more deeply embedded in the application and grows exponentially more difficult and costly to fix. This, in turn, helps the team meet increasingly aggressive release schedules without compromising quality and reliability. Ability to test a full range of application functionality on a single machine independent of a complex testing environment. For applications that contain database interactions or calls to other external data sources, processes, services, and so on, Parasoft Tracer generates test cases that represent that behavior, developing test case stubs as needed. These test cases can be used to test complex application behavior from a single machine (which may be a developer or QA desktop) without requiring live data connections or a fully staged test environment. Development can easily test critical 1

3 application behavior without extensive overhead and extend the range of testing that can be done during development. 1.2 How Parasoft Tracer Addresses Common Business Problems Maintenance Projects Most software development projects involve maintaining an existing application, rather than writing a new one from scratch. While new projects are commonly supported with large budgets and additional human resources, insufficient resources are usually allocated for maintenance projects. Nevertheless, these projects are expected to satisfy the same high quality standards as new development projects. Thorough testing must be performed on areas that are directly affected by system modifications and extensions, and even areas that were not explicitly changed need to be tested to verify that the modifications did not cause unexpected changes or side effects (generally called regressions). If the team builds an effective regression test suite when a new system is developed, testing can be performed thoroughly at the maintenance project phase. If not, performing regression testing at the maintenance project phase is challenging. Parasoft Tracer is very powerful in situations where a code base does not have a well-planned regression test suite, and that code base needs to be modified or extended. Iterative New Development Projects Another common challenge is how to implement regression test suites in an iterative development process, such as Agile Development or Extreme Programming. The unit test suites created using Jtest or JUnit can be used for regression tests. By also using Parasoft Tracer from the earliest phases of development, it is possible to create a test case that uses the actual data obtained from functional-level tests or load tests, or even more realistic data obtained from a simulated production environment. Various Development Projects Most developers have experienced a problem that occurs only in the production environment, only when the database or another system is connected, or only when the program is loaded into the application server. Such problems are difficult to debug; since a similar environment usually cannot be setup on the developer desktop, the same problem typically cannot be reproduced and analyzed from the developer desktop. Parasoft Tracer can significantly reduce debugging costs by allowing developers to reproduce realistic program behavior on the developer desktop. The desired operations can be performed in the production environment, then Parasoft Tracer can then capture these operations in JUnit tests, which can be run on the developer desktop because dependencies on the production environment are automatically stubbed out. The purely class-level unit test cases that Jtest generates without Tracer typically test how the code behaves under unexpected conditions. On the other hand, the functional unit test cases generated by Parasoft Tracer check the code s functionality under expected, realistic use case scenarios. Together, these different types of test cases create a robust regression test suite that can be run regularly to immediately expose any code modifications/additions that break existing functionality or introduce unexpected side effects. This is especially critical for supporting short release cycles and ensuring the continued functionality of constantly-evolving and difficult-to-test applications. Using Jtest s Test Case Parameterization feature, all of these test cases can be extended with a wide variety of test input values (runtime-generated, user-defined, or from data sources) to create test scenarios that reflect a broad range of realistic usage. This streamlines the development and 2

4 execution of complex test case scenarios that ensure broad, thorough test coverage, and that validate the code's response to a wide range of expected and unexpected user actions. 2 Sample Project Scenario 2.1 Sample Application This paper explains how Parasoft Tracer can be used in a development project that involves extending an existing application. The application that we will extend is the JPetStore open source application. The original J2EE-based JPetStore 1.0 was developed by Clinton Begin of ibatis in Since then, it has been completely rewritten, and a number of different versions are currently available. This paper uses the version bundled as a Jtest 8.0 sample project. JPetStore is a sample application for an online pet shop. It implements the complete functionality of a major online pet shop, such as searching for a product, saving the selected product in the shopping cart, and checking out. Figure 5. shows the main pages and the application flow. A demonstration version of JPetStore is available on the internet at Welcome page Main page Product directory (Fish page) Product list (Angelfish page) 3

5 Product detail Order input Checkout [ Fig. 5] Login for payment 2.2 Specifications for the Project Extension This project is expected to extend the JPetStore application as follows: (1) Security Policy New Specification Requirements The quantity of any product ordered in a single purchase must not exceed 10. Any quantity over 10 should be reduced to 10. (2) Discount Policy A 5% discount should be applied to all orders totaling over $100. If the discounted price is less than $100, it should be adjusted to $100. 4

6 The security policy requirement is implemented by preventing users from ordering more than 10 of each item in a single purchase. Fake orders designed to prevent sales activities are a common problem on internet shopping sites. This requirement attempts to reduce the damage caused by fake orders by limiting how many of each item can be ordered. In an actual e-commerce application, a more complex solution would be needed. However, this simple solution is sufficient for our purpose, which is demonstrating how to use Parasoft Tracer in a realistic project that involves extending an existing application. The discount policy requirement implements a newly-defined business rule: A 5% discount should be applied to all orders totaling over $100. As with the previous requirement, the implementation would need to be more complex in an actual application. 2.3 Functional Test Cases (Use Cases) At least three functional test cases are needed to test these two new specification requirements: TEST 1 (For the Security Policy) Scenario: Order 11 or more a single product. Original result (before the application modification): 11 of the same product can be ordered. New result (after the application modification): The per-product item quantity is limited to 10. TEST 2 (For the Discount Policy) Scenario: The order total is more than $ (which will be $100 or more, even after a 5% discount). Original result (before the application modification): The total price is the product s list price quantity. New result (after the application modification): The final total price is the product s list price quantity, minus 5% of that total. TEST 3 (For the Discount Policy) Scenario: The order total is between $100 and $ (which will be less than $100 when a 5% discount is applied). Original result (before the application modification): The total price is the product s list price quantity. New result (after the application modification): The total price is $ Source Code to be Extended The source of the JPetStore application is available in the packages shown in Figure 6. 5

7 [ Fig. 6] The application logic is controlled by the source code in the com.ibatis.jpetstore.domain package (See Figure 7). [ Fig. 7] Implementation of the Security Policy The security policy will be implemented in the com.ibatis.jpetstore.domain package s CartItem class (in the setquantity(int) method, as shown in Figure 8). This is where the quantity of the ordered product is set. 6

8 [ Fig. 8] [ Fig. 9] Implementation of the Discount Policy The discount policy is implemented in the com.ibatis.jpetstore.domain package s CartItem class (in the SetQuantity(int) method, as shown in Figure 9). 3 Sample Usage of Parasoft Tracer 3.1 Overall Project Flow Before we discuss the details of how Parasoft Tracer can be used to verify our extensions to this project, it is important to understand the overall project flow. The overall project flow will involve: 1) Preparing the Regression Test Suite Prepare the regression test suite that will be run after the source code modifications are completed. This regression test suite will include functional and unit-level regression test cases. a) Functional Regression Test Cases Use Parasoft WebKing to create functional regression test cases. The scenario that WebKing will record includes the 3 test cases which are described in section 2.3. b) Unit-Level Regression Test Cases Prepare unit-level regression test cases by generating and executing test cases from Parasoft Tracer s monitoring, then review their results 7

9 Generating and executing test cases from Parasoft Tracer s monitoring Parasoft Tracer can directly monitor and generate test cases for an application that is already running on localhost or a remote machine, or that can be launched from the Eclipse/Jtest environment. This is the case with the JPetstore project. Alternatively, Parasoft Tracer can record a monitoring log file as you exercise an application, such as a Web application deployed on a staging server or production server. This log file can later be used for unit test generation within the Jtest environment. For example, a QA tester can record a monitoring log file as he or she exercises an application running on a Web server; the log file can then be passed on to a developer, who can use that file to generate realistic unit tests. In this example, we will use the test cases created with WebKing to exercise the application as Parasoft Tracer monitors it Reviewing and verifying the test execution results After the generated test cases are executed, review the results, verify the unverified outcomes, and save them for use as a regression test suite. 2) Extending the Source Code Update the SetQuantity(int) method of the CartItem class and the getsubtotal () method of the Cart class (both from the com.ibatis.jpetstore.domain package) to implement the new requirements, then compile the code. If this were a real project, we would perform class-level unit testing with Jtest or JUnit. For the sake of simplicity, we will skip unit testing on this sample project. 3) Executing and Updating the Regression Tests a) Unit-Level Regression Tests Using the unit-level regression testing suite that was prepared before extending the source code, check to see if the modified code causes test case failures (this is the expected result, since the code functionality changed), and verify whether it had any unexpected side effects in the rest of the application. Also, update the test cases to reflect the change in expected functionality. b) Functional Regression Tests Using the functional regression testing suite that was prepared before extending the source code, check to see if the modified code causes test case failures (this is the expected result, since the code functionality changed), and verify whether it had any unexpected side effects in the rest of the application. Also, update the test cases to reflect the change in expected functionality. 3.2 Preparing the Functional Regression Test Cases To prepare the functional regression test suite, we will use WebKing to create the three test cases described in section

10 Creating the Functional Regression Test Suite Start WebKing, and create the project. WebKing spiders through the project, starting at the specified URL, and adds the scanned items the project. Implementing TEST 1 (For the Security Policy) Scenario: Order 11 or more of a single product. Original result (before the application modification): 11 items of the same product can be ordered. Start creating the test case in record using browser mode. 9

11 From the Welcome page, browse to the Main page, then to the Fish page. Click the Goldfish link in the Fish page. The Goldfish Stock page will open. 10

12 Add one adult male goldfish to the shopping cart. The total cost of items shopping cart is $5.50. Change the quantity of adult male goldfish to 11, then click the Update Cart button. The total cost will be recalculated. The recalculated total cost is $ Implementing TEST 2 (For the Discount Policy) Scenario: The order total is more than $ (which will be $100 or more, even after a 5% discount). Original result (before the application modification): The total price is the product s list price quantity. 11

13 Create a second test case path and add a large angelfish to the shopping cart. Use the same general procedure that you used in Test 1. The total cost of items in the shopping cart is $ Change the quantity of large angelfish to 10, then click the Update Cart button. The total cost will be recalculated. The total cost is recalculated; the new cost is $

14 Implementing TEST 3 (For the Discount Policy) The order total is between $100 and $ (which will be less than $100 when a 5% discount is applied). Original result (before the application modification): The total price is the product s list price quantity. Create a third test case path and add an adult female goldfish, plus a small angelfish, to the shopping cart. Use the same general procedure that you used in Test 1 and Test 2. The total cost of items in the shopping cart is $ Change the quantity of adult female goldfish to 4. Change the quantity of small angelfish to 5. Click the Update Cart button The total cost will be recalculated. The total price is recalculated and it becomes $

15 Verifying the Test Cases Execute these three test cases, then check if any problems are reported. 3.3 Preparing the Unit-Level Regression Test Cases Preparing the unit regression test cases involves generating and executing test cases from Parasoft Tracer s monitoring, then reviewing their results. Generating and Executing Test Cases from Parasoft Tracer s Monitoring To generate and execute JUnit test cases that capture the operations represented in our three tests, run Jtest s Generate and Run from Monitoring built-in Test Configuration. This will launch the application so you can exercise it, then monitor the application as it is exercised, generate test cases that capture the monitored operations, then execute the generated tests. We will exercise the application by running the three functional test cases that were prepared in WebKing. 14

16 In the Package Explorer for Jtest/Eclipse, select the item that you want to monitor, Here, we will select the entire project. Monitoring the entire project is helpful because the modifications that we will make might have side effects that affect other parts of the application. Run Jtest s builtin Generate and Run from Monitoring Test Configuration. 15

17 Parasoft Tracer starts to monitor the application as soon as it is launched. Start WebKing, execute the three functional test cases that were previously created, then verify whether tests are executed without any problems. Stop the application server. 16

18 When the application server stops, Parasoft Tracer will stop monitoring the application and start generating test cases. After the test cases have been generated, they will be executed. 17

19 Reviewing and Verifying Test Results The next step is to review the results, verify all unverified outcomes and save them for use as a regression test suite. A summary is displayed after test case execution is completed. This summary shows that the three WebKing test cases covered 57% of the code in the project. Confirm the coverage of the methods that we need to modify in order to implement the new requirements: the setquantity method in the CartItem class, and the getsubtotal method in the Cart class. 18

20 Note that 100% coverage was achieved for both methods. Review the unverified outcomes. In the generated JUnit file, the assertion for each unverified outcome is marked with a "// jtest_unverified" comment. 19

21 The Jtest Test Class Outline offers a convenient way to review the test cases. Verify each outcome that is correct. 20

22 Alternatively, multiple unverified outcomes can be verified with a single-click. To do this, select the node or nodes that indicate which outcomes you want to verify, then choose Verify All. In the generated JUnit file, the assertion for each verified outcome is marked with a "// Verified by [user name]" comment. 3.4 Extending the Source Code Next, we will modify and compile the SetQuantity method of the CartItem class and the getsubtotal method of the Cart class (both from the com.ibatis.jpetstore.domain package). Extending the SetQuantity(int) Method in the CartItem Class The SetQuantity method sets the quantity of a single product. This method needs to be modified to implement the new security policy. Code before the update public void setquantity(int quantity) { } this.quantity = quantity; calculatetotal(); 21

23 Code after the update public void setquantity(int quantity) { ////////////////////////// // implementation of security policy //. Limited quantity of one product can be ordered in one order process is up to 10. } // //..{ if(quantity > 10).. if the number of orders is 10 or more } quantity = 10; // Limit the number of orders to 10. ////////////////////////// this.quantity = quantity; calculatetotal(); Extending the getsubtotal () Method in the Cart Class The getsubtotal method calculates the total cost of items saved in the shopping cart. This method needs to be modified to implement the new discount policy. Code before the update public BigDecimal getsubtotal() { BigDecimal subtotal = new BigDecimal("0"); Iterator items = getallcartitems(); while (items.hasnext()) { CartItem cartitem = (CartItem) items.next(); Item item = cartitem.getitem(); BigDecimal listprice = item.getlistprice(); BigDecimal quantity = new BigDecimal(String.valueOf(cartItem.getQuantity())); subtotal = subtotal.add(listprice.multiply(quantity)); return subtotal; } 22

24 Code after the update public BigDecimal getsubtotal() { BigDecimal subtotal = new BigDecimal(0); BigDecimal discounttotal = new BigDecimal(100); Iterator items = getallcartitems(); while (items.hasnext()) { CartItem cartitem = (CartItem) items.next(); Item item = cartitem.getitem(); BigDecimal listprice = item.getlistprice(); BigDecimal quantity = new BigDecimal(String.valueOf(cartItem.getQuantity())); subtotal = subtotal.add(listprice.multiply(quantity)); } //start added code //Implementation of the discount policy //5% discount will be applied to the total price if an order is $100 or more //if the price after the 5% discount becomes less than $100, the total price will be $100 if (subtotal.compareto(discounttotal) > 0) { //if the total is $100 or more let's apply 5% discount BigDecimal sb = new BigDecimal(0.95); subtotal = subtotal.multiply(new BigDecimal(0.95)); //if the total after discount is less than $100 adjust to $100 if(subtotal.compareto(discounttotal) < 0) { subtotal = discounttotal; // Round it up to $100. } } //end added code return subtotal; } 3.5 Executing and Updating the Unit-Level Regression Test Suite Using the unit-level regression testing suite we prepared before extending the source code, we will check to see if the modified code causes test case failures (this is the expected result, since the code functionality changed), and verify whether it had any unexpected side effects in the rest of the application. Also, we will update the test cases to reflect the change in expected functionality. 23

25 Use Jtest s built-in Run Unit Tests Test Configuration to execute the unit-level regression test suite for com.ibatis.jpetstore.domain. Review the test case failures reported. Test Case Failures from the Cart Class Test Case Failure 1 This test case expected that when the getsubtotal calculation was performed, the total cost would be $ (the total cost of all items in the cart). However, since the new discount policy specification requirement applies a 5% discount to orders totaling over $105.26, the expected result has changed from 165 to The test case s expected value needs to be updated 24

26 accordingly. This can be accomplished by right-clicking the error message, choosing Change expected value from the shortcut menu, then clicking OK in the dialog that opens. Test Case Failure 2 This test case expected that when the getsubtotal calculation was performed, the total cost would be $ (the total cost of all items in the cart). However, since the new discount policy specification requirement discounts all purchases between $100 and $ to $100, the expected result has changed from to 100. The test case s expected value needs to be updated accordingly. This can be accomplished by right-clicking the error message, choosing Change expected value from the shortcut menu, then clicking OK in the dialog that opens. 25

27 Test Case Failure 3 This test case expected that when the setquantity method was used to update the item quantity to 11, the number of items would be set to 11, and the total cost would be $60.50 (11 items at the cost of $5.50 per item). However, since the new security policy specification requirement prohibits a purchase with more than 10 of the same item, only 10 can be purchased, which causes the expected result to change from 60.5 to 50.5 (10 items at the cost of $5.50 per item). The test case s expected value needs to be updated accordingly. This can be accomplished by rightclicking the error message, choosing Change expected value from the shortcut menu, then clicking OK in the dialog that opens. Test Case Failures from the CartItem Class This test case expected that when the setquantity method was used to update the item quantity to 11, the number of items would be set to 11. However, since the new security policy specification requirement prohibits a purchase with more than 10 of the same item, the expected result must change from 11 to 10. The test case s expected value needs to be updated accordingly. This can be accomplished by right-clicking the error message, choosing Change expected value from the shortcut menu, then clicking OK in the dialog that opens. Updating Expected Results When a test case s expected value changes (for instance, because the code s functionality has been intentionally changed to implement new specification requirements), it can be updated as follows. Right-click the message for the value that needs to be modified, then choose Change expected value from the shortcut menu. 26

28 Enter the correct expected value, then click OK. When this procedure is repeated for each test failure caused by the intentional functionality changes, the test suite will be configured to check for the new expected functionality. These tests can then be run every 24 hours to verify whether additional changes impact these recentlyimplemented functionality requirements (the security policy and the discount policy). If any problems are introduced, these test cases will fail in order to alert the team to the problem. 3.6 Executing and Updating the Functional Regression Test Suite Using the functional regression testing suite we prepared before extending the source code, we will check to see if the modified code causes test failures (this is the expected result, since the code functionality changed), and verify whether it had any unexpected side effects in the rest of the application. Also, we will update the test cases to reflect the change in expected functionality. Executing the Test Suite First, we will execute the functional test suite that we previously created. 27

29 Reviewing Errors Next, we review the reported errors (indicated with red bubbles). To explore a reported error, we right-click it, then choose Regression Results from the shortcut menu. This launches a comparison tool that will display the expected results (the results obtained when the test was created) on the left side, and the actual results (the results obtained from the current test run) on the right side. 28

30 The first error occurred because original test case recorded a quantity of 11, but this result could not be achieved after the new security policy was implemented. This change is expected, so the test case s expected value needs to be updated accordingly. The other reported errors are also expected because the code s functionality has been intentionally changed to implement new specification requirements. These values also need to be updated. 29

31 Updating Expected Results When a test case s expected value changes (for instance, because the code s functionality has been intentionally changed to implement new specification requirements), it can be updated by clicking Replace Current Control. When this procedure is repeated for each test failure caused by the intentional functionality changes, the test suite will be configured to check for the new expected functionality. These tests can then be run every 24 hours to verify whether additional changes impact these recentlyimplemented functionality requirements (the security policy and the discount policy). If any problems are introduced, these test cases will fail in order to alert the team to the problem. 30

32 4 About Parasoft For 20 years, Parasoft has investigated how and why software errors are introduced into applications. Our solutions leverage this research to deliver quality as a continuous process throughout the SDLC. This promotes strong code foundations, solid functional components, and robust business processes. Whether you are delivering Service-Oriented Architectures (SOA), evolving legacy systems, or improving quality processes draw on our expertise and awardwinning products to increase productivity and the quality of your business applications. For more information visit: 5 Contacting Parasoft USA 101 E. Huntington Drive, 2nd Floor Monrovia, CA Toll Free: (888) Tel: (626) Fax: (626) info@parasoft.com URL: Europe France: Tel: +33 (1) UK: Tel: +44 (0) Germany: Tel: info-europe@parasoft.com Asia Tel: info-psa@parasoft.com Other Locations See Parasoft Corporation All rights reserved. Parasoft and all Parasoft products and services listed within are trademarks or registered trademarks of Parasoft Corporation. All other products, services, and companies are trademarks, registered trademarks, or servicemarks of their respective holders in the US and/or other countries. 31