Secrets of Successful Java Testing

Transcription

1 Presentation Paper Bio Return to Main Menu P R E S E N T A T I O N W7 Wednesday, Nov 10, 1999 Secrets of Successful Java Testing Gary Guillon International Conference On Software Testing, Analysis & Review NOV 8-12, 1999 BARCELONA, SPAIN

2 Secrets of successful Java testing Gary Guillon Sun Microsystems

3 Refresher : What is Java? The same identical Java application can run on any OS, without modification! The Java Virtual Machine levels the playing field... JAVA JAVA JAVA JAVA An APP application written APP for one OS, will not APP run on another... APP APP JVM JVM JVM JVM OS 1 OS 2! OS 3 OS 4 Different operating systems have different facilities for running programs It s a language, and an environment! Slide 2

4 Cost of Software Defects Defect Cost Test Earlier and More Often Design Implementation Release Deployment The cost of software defects increases rapidly. Slide 3

5 The Testing Process Design Implement Release Deploy Traditional Approach Design Static Analysis Implement Unit testing Release Regression Deploy Load Recommended Approach Slide 4 Test Earlier and More Often

6 When and what to check Static Check Dynamic Check Unit testing Code coverage analysis Regression Load testing Design Implement Release Deployment Slide 5

7 Why is it different to test? Java Apps JVM Native OS Native Apps Java Apps JVM Java OS Java Apps Java Chip Slide 6

8 Platform Specific Problems Slide 7 Java Apps JVM Native Apps Platform-Specific Test Tool Is your OS supported? Unix Test tool Win 32 tool Java Apps Mac Test tool OS/2 Test tool Win3.x Test tool JVM AS/400 test tool Java OS Unix test script Win 32 script Java Apps Mac test script OS/2 test script Win3.x test script AS/400 test script Java Chip JavaOS tool??? Java Chip tool????? Platform-specific tools require one test for each platform Platform-specific test tools are blind to most Java events

9 Java Test Tools Run Anywhere Java Test tool Java Test script Java Apps Java Apps Java Apps Java Tool Native Apps Java Tool Java Tool JVM Native OS JVM Java OS Java Chip Pure Java testing tools work on all Java platforms The Java test tool runs in the same process as the application 100% portable tests, write tests once, run tests anywhere Slide 8

10 Can you spot the difference? A GUItesting toolthatcannot resolve objects and is forced to resolve bitmap images.. WillFA IL a testforthe wrong reason. There is no difference,these GUIs are functionally identical! A testpa SS! Slide 9

11 The Advantages Java Apps Java Tool JVM PlatformTool Native OS! Data,method! getbackground ()! getlabel()! getlocation ()! isv isible ()! isenabled () Java-based Return Value Platform- Specific Color.Grey Replace All 100,120 Boolean.TRUE Boolean.TRUE (Bitmap image) Peerless components are fully visible to Java-based tools Peerless components are invisible to platform specific tools Slide 10

12 Testing Applets Applets can be tested using Suns appletviewer technology, or in the browser itself. (This may require the installation of the java plug-in) Slide 11

13 Write tests once, run tests anywhere! Application Test Script Application is installed on target configurations Script is reviewed once for thoroughness and coverage Solaris Win NT/95 NC Result File Execution on Solaris Result File Execution on NT/95 Result File Execution on NC C O M P A R E Slide 12

14 What is the Highest Priority? What gets effected, when you shift priorities? 10 Total Available Marks = 4-3 = Quality of Product? 6 10 Time to Market? 3 Cost Saving = 1!! Slide 13

15 Are you dealing with the real problem? Actual radio conversation released by Chief of Naval Operations Ship #1: Please divert your course 15 degrees to the North to avoid a collision. Unknown contact#2: Recommend you divert your course 15 degrees to the south to avoid a collision. Ship #1: This is the Captain of a US Navy ship. I say again, divert YOUR course. Contact #2: This is the watch ensign. I say again, you divert YOUR course. Ship #1: THIS IS THE AIRCRAFT CARRIER ENTERPRISE! WE ARE A LARGE WARSHIP OF THE US NAVY! DIVERT YOUR COURSE NOW! Contact #2: This is a lighthouse. It's your call!!!! Slide 14

16 Load Performance Time (sec) Whatis the response time at100 users? Time by Load testing! Time by Finger in Air method System Crash! In the past Clients have estimated performance based on an existing system or available hardware User Count Slide 15

17 Asynchronous Load testing. Ouch. Traditional Client/Server Synchronous arrangement. Client always waits for server. New Era ORB based or distributed application provides asynchronous communication for greater efficiency! Query and response are always in the same order. GUI client waits whilst server is busy. Multiple request can be sent, data can arrive in any order! GUI can do useful work whilst server busy. Sam e System,sam e end result,butdifferentinternaltransaction organisation! Slide 16

18 Asynchronous Load testing.cont. Input 1 Response 1 Input 2 Response 2 Input3 Response3 Script captured from Scenario 1 Tries to play against Scenario 2 but the system under test responds differently... Input 1 Waiting for response 1 waiting waiting waiting... R ecord,capture and encapsulate load testing technologies are all synchronous! Scriptdoesn tsend input2 untilresponse 1 is received,forcing system to behave differently, INVALIDATING THE TEST! FORCES SYSTEM TO BE SYNCHRONOUS! NOT A TRUE REPRESENTATION OF THE SYSTEM!!! Slide 17

19 Network Java Load Testing Injector Network Mainframe / Legacy App Server Internet GUI vu 500 Net 4 Telemetry Distributed 3-tier SAP / Baan / Oracle Central Console Slide 18

20 The Testing Infrastructure JavaStar CM Systems Scripts & Results GUI Testing GUI Coverage Tools JavaScope? Test Management! Manual System Software Under Test API Defect Tracking! TestM anagem entis essential!!! Unit Testing JavaSpec Slide 19

21 Need to know more? Web : /suntest Em sun.com Phone : (UK) Phone : (International) Slide 20

22 That was Secrets of successful Java testing By... Gary Guillon Sun Microsystems

23 Gary Guillon European Manager - Java Testing Tools for Sun Microsystems Gary was a Engineer for GEC Avionics working on Hardware Data Conditioning for the Tornado Aircraft and moved into the software group to test the Nimrod - Early Warning Submarine Detection equipment, which is still being used today. He has a number of humorous stories of 'Testing Disasters' that appear during his testing presentation. Java is rapidly becoming the language/environment of choice in the Times Top 1000 companies world-wide. With it comes a whole host of Testing concerns. Asynchronous Testing being one of the frequently overlooked elements of testing software running on parallel processors. This talk addresses the uniqueness of Java and how Asynchronous Testing can be done effectively with technology that was invented by Sun Microsystems, the author of Java itself.