Presentation Paper Bio Return to Main Menu P R E S E N T A T I O N W7 Wednesday, Nov 10, 1999 Secrets of Successful Java Testing Gary Guillon International Conference On Software Testing, Analysis & Review NOV 8-12, 1999 BARCELONA, SPAIN
Secrets of successful Java testing Gary Guillon Sun Microsystems
Refresher : What is Java? The same identical Java application can run on any OS, without modification! The Java Virtual Machine levels the playing field... JAVA JAVA JAVA JAVA An APP application written APP for one OS, will not APP run on another... APP APP JVM JVM JVM JVM OS 1 OS 2! OS 3 OS 4 Different operating systems have different facilities for running programs It s a language, and an environment! Slide 2
Cost of Software Defects Defect Cost Test Earlier and More Often Design Implementation Release Deployment The cost of software defects increases rapidly. Slide 3
The Testing Process Design Implement Release Deploy Traditional Approach Design Static Analysis Implement Unit testing Release Regression Deploy Load Recommended Approach Slide 4 Test Earlier and More Often
When and what to check Static Check Dynamic Check Unit testing Code coverage analysis Regression Load testing Design Implement Release Deployment Slide 5
Why is it different to test? Java Apps JVM Native OS Native Apps Java Apps JVM Java OS Java Apps Java Chip Slide 6
Platform Specific Problems Slide 7 Java Apps JVM Native Apps Platform-Specific Test Tool Is your OS supported? Unix Test tool Win 32 tool Java Apps Mac Test tool OS/2 Test tool Win3.x Test tool JVM AS/400 test tool Java OS Unix test script Win 32 script Java Apps Mac test script OS/2 test script Win3.x test script AS/400 test script Java Chip JavaOS tool??? Java Chip tool????? Platform-specific tools require one test for each platform Platform-specific test tools are blind to most Java events
Java Test Tools Run Anywhere Java Test tool Java Test script Java Apps Java Apps Java Apps Java Tool Native Apps Java Tool Java Tool JVM Native OS JVM Java OS Java Chip Pure Java testing tools work on all Java platforms The Java test tool runs in the same process as the application 100% portable tests, write tests once, run tests anywhere Slide 8
Can you spot the difference? A GUItesting toolthatcannot resolve objects and is forced to resolve bitmap images.. WillFA IL a testforthe wrong reason. There is no difference,these GUIs are functionally identical! A testpa SS! Slide 9
The Advantages Java Apps Java Tool JVM PlatformTool Native OS! Data,method! getbackground ()! getlabel()! getlocation ()! isv isible ()! isenabled () Java-based Return Value Platform- Specific Color.Grey Replace All 100,120 Boolean.TRUE Boolean.TRUE (Bitmap image) Peerless components are fully visible to Java-based tools Peerless components are invisible to platform specific tools Slide 10
Testing Applets Applets can be tested using Suns appletviewer technology, or in the browser itself. (This may require the installation of the java plug-in) Slide 11
Write tests once, run tests anywhere! Application Test Script Application is installed on target configurations Script is reviewed once for thoroughness and coverage Solaris Win NT/95 NC Result File Execution on Solaris Result File Execution on NT/95 Result File Execution on NC C O M P A R E Slide 12
What is the Highest Priority? What gets effected, when you shift priorities? 10 Total Available Marks - 10-6 = 4-3 = 1 10 10 Quality of Product? 6 10 Time to Market? 3 Cost Saving = 1!! Slide 13
Are you dealing with the real problem? Actual radio conversation released by Chief of Naval Operations Ship #1: Please divert your course 15 degrees to the North to avoid a collision. Unknown contact#2: Recommend you divert your course 15 degrees to the south to avoid a collision. Ship #1: This is the Captain of a US Navy ship. I say again, divert YOUR course. Contact #2: This is the watch ensign. I say again, you divert YOUR course. Ship #1: THIS IS THE AIRCRAFT CARRIER ENTERPRISE! WE ARE A LARGE WARSHIP OF THE US NAVY! DIVERT YOUR COURSE NOW! Contact #2: This is a lighthouse. It's your call!!!! Slide 14
Load Performance Time (sec) 100 90 80 70 60 50 40 30 20 10 0 Whatis the response time at100 users? Time by Load testing! Time by Finger in Air method System Crash! In the past Clients have estimated performance based on an existing system or available hardware 0 20 40 60 80 100 120 User Count Slide 15
Asynchronous Load testing. Ouch. Traditional Client/Server Synchronous arrangement. Client always waits for server. New Era ORB based or distributed application provides asynchronous communication for greater efficiency! Query and response are always in the same order. GUI client waits whilst server is busy. Multiple request can be sent, data can arrive in any order! GUI can do useful work whilst server busy. Sam e System,sam e end result,butdifferentinternaltransaction organisation! Slide 16
Asynchronous Load testing.cont. Input 1 Response 1 Input 2 Response 2 Input3 Response3 Script captured from Scenario 1 Tries to play against Scenario 2 but the system under test responds differently... Input 1 Waiting for response 1 waiting waiting waiting... R ecord,capture and encapsulate load testing technologies are all synchronous! Scriptdoesn tsend input2 untilresponse 1 is received,forcing system to behave differently, INVALIDATING THE TEST! FORCES SYSTEM TO BE SYNCHRONOUS! NOT A TRUE REPRESENTATION OF THE SYSTEM!!! Slide 17
Network Java Load Testing Injector Network Mainframe / Legacy App Server Internet... 20 GUI vu 500 Net 4 Telemetry Distributed 3-tier SAP / Baan / Oracle Central Console Slide 18
The Testing Infrastructure JavaStar CM Systems Scripts & Results GUI Testing GUI Coverage Tools JavaScope? Test Management! Manual System Software Under Test API Defect Tracking! TestM anagem entis essential!!! Unit Testing JavaSpec Slide 19
Need to know more? Web :http://www.sun.com /suntest Em ail:suntest_saleseurope@ sun.com Phone :0800 962761 (UK) Phone :+353 1 819 9000 (International) Slide 20
That was Secrets of successful Java testing By... Gary Guillon Sun Microsystems suntest_saleseurope@sun.com
Gary Guillon European Manager - Java Testing Tools for Sun Microsystems Gary was a Engineer for GEC Avionics working on Hardware Data Conditioning for the Tornado Aircraft and moved into the software group to test the Nimrod - Early Warning Submarine Detection equipment, which is still being used today. He has a number of humorous stories of 'Testing Disasters' that appear during his testing presentation. Java is rapidly becoming the language/environment of choice in the Times Top 1000 companies world-wide. With it comes a whole host of Testing concerns. Asynchronous Testing being one of the frequently overlooked elements of testing software running on parallel processors. This talk addresses the uniqueness of Java and how Asynchronous Testing can be done effectively with technology that was invented by Sun Microsystems, the author of Java itself.