RemoteLaunch Automated Testing Integration & User Guide Inflectra Corporation

Transcription

1 RemteLaunch Autmated Testing Integratin & User Guide Inflectra Crpratin Date: January 8th, 2015

2 Cntents Intrductin RemteLaunch Guide QuickTestPr Engine SmarteScript Engine TestCmplete Engine Selenium Engine Squish Engine Cmmand-Line Engine LadRunner 11 Engine SapUI Engine FitNesse Engine NeLad Engine TestPartner Engine BadBy Engine JMeter Engine Intrductin SpiraTest prvides an integrated, hlistic Quality Assurance (QA) management slutin that manages requirements, tests and incidents in ne envirnment, with cmplete traceability frm inceptin t cmpletin. SpiraTeam is an integrated Applicatin Lifecycle Management (ALM) system that manages yur prject's requirements, releases, test cases, issues and tasks in ne unified envirnment. SpiraTeam cntains all f the features prvided by SpiraTest - ur highly acclaimed quality assurance system and SpiraPlan - ur agile-enabled prject management slutin. This guide utlines hw t use the RemteLaunch add-n fr SpiraTest r SpiraTeam t remtely schedule and launch autmated tests using a variety f cmmercial and pen-surce test autmatin engines. Fr infrmatin regarding hw t use SpiraTest itself, please refer t the SpiraTest User Manual. The first sectin utlines hw t use the RemteLaunch applicatin, and the subsequent sectins describe each f the plugins that supprt a different test autmatin tl. 15. Ranrex Engine Ratinal Functinal Tester TestingAnywhere Page 1 f 133

3 1. RemteLaunch Guide There are actually tw separate versins f RemteLaunch that are available frm Inflectra: 1. The Micrsft Windws cmpatible Spira RemteLaunch applicatin that prvides a graphic user interface applicatin fr executing autmated tests n remte cmputers using varius plugins fr different testing technlgies and have the results be sent t the cnfigured SpiraTest/SpiraTeam server. 2. The crss-platfrm Spira RemteLaunchX Java applicatin that prvides a lightweight cnsle applicatin that can execute simple cmmand line scripts n the target cmputer and send the results back t the cnfigured SpiraTest/SpiraTeam server. This applicatin can be used in Micrsft Windws, Linux r Apple MacOS X cmputers prvided that they have the Java 1.7 (r later) runtime installed. The first part f this sectin will describe hw t use the Windws-nly RemteLaunch GUI applicatin and the secnd part will describe hw t use the crss-platfrm RemteLaunchX cnsle applicatin Installing RemteLaunch It is required that yu install the prgram befre cpying r installing any test extensins fr the prgram. Testing applicatins, like Selenium and QuickTest Pr can be installed with n regards t the client applicatin if they are nt installed by the time a test requiring them needs t be executed, the test extensin will simply reprt an errr r blck fr the specified test set. There are n ptins t the installer except fr installatin path. If yu d nt use the default installatin path (typically C:\Prgram Files\Inflectra\Spira RemteLaunch\), then make a nte f where the installatin path is, because it will be needed t install test extensins later Installing a Test Extensin A test extensin is a single r a set f DLLs that the prgram will read upn startup and prvides a link in which testing applicatins (like TestCmplete and Squish) t reprt test infrmatin and status back t SpiraTeam. When yu dwnlad a test extensin, the ZIP file shuld cntain at least ne DLL file. Unless therwise specified by a readme.txt file included in the cmpressed file, cpy the DLL file t the \extensin directry lcated within Spira RemteLaunch installatin directry. (If n such flder exists, yu must create it.) If an extensin is remved r added, the prgram must be restarted fr the any changes t take effect. The prgram will nly lad up t the first number f extensins that the license allws. Additinal extensins will nt be laded r used during testing Registratin Spira RemteLaunch has its wn License key needed fr using the prgram. Yu cannt use yur existing SpiraTest/Plan/Team key in Spira RemteLaunch. Upn the first launch f the prgram, yu will be asked t update yur license infrmatin. Enter in yur rganizatin name and license key in the that was sent when yu purchased the license, r as listed n yur custmer infrmatin page at Page 2 f 133

4 Trial licenses are gd until the 28 th day f the listed mnth. The next time the prgram is run after the 28 th f the mnth, yu will be prmpted t re-enter a new permanent license key, r the prgram will be unusable. The license key can be updated at any time by ging t the Tray Menu and select Help -> Abut. Once the Abut screen pens up, click the Update buttn in the license details sectin t update r change license infrmatin Using RemteLaunch Basic Unattended Operatin When run, the prgram will start minimized t the system tray and will start its plling f the server. Plling will ccur every x minutes (60 by default) fr any autmated test sets that are scheduled t be run. When time cmes fr a test t be launched, it will start the test extensin. The installed test extensin will then perfrm the test and reprt results back t SpiraTeam. At the end f the test, the prgram will g back and resume scanning fr tests that need t be executed. N user input is ever needed frm the applicatin itself. Hwever, testing applicatins may pp up dialgs needing user input. Fr existing Inflectra testing extensins, effrt was put in t avid as much userinteractin as pssible, but in sme cases it is unavidable Client Cnfiguratin By right clicking n the system tray icn and selecting Cnfiguratin, the applicatin s windw will pen t the cnfiguratin panel. The panel has the fllwing ptins: SpiraTeam Server Cnfiguratin: Server URL: This is the URL f the SpiraTeam installatin. Be sure t nt put /Lgin.aspx r any ther page in the string, this shuld be just the rt URL f the applicatin s install. Lgin Username: This is the SpiraTeam lgin id f the user that yu want the tests reprted as. Nte that while the applicatin is plling and updating test results, if the user is lgged int a web brwser sessin, they will get kicked ut. Lgin Passwrd: The passwrd t the Username abve. Page 3 f 133

5 Test: Clicking this will test the lgin t make sure the applicatin can cnnect t the server prperly. Server Plling: Autmatin Hst Tken: This field is required, and uniquely identifies the lcal testing machine. Any scheduled tests assigned t the Autmatin Hst n SpiraTeam will get plled fr this machine. Except in special circumstances, this ID shuld be unique amng all testing machines. Imprtant: This field must match the string that is entered int the Autmatin Hst Details screen in the Tken: field, r scheduled tests will nt be recgnized. Autmatically Run Overdue Tests: When this is checked, any tests that are pulled frm the SpiraTest server that has a scheduled date in the past will be marked as Overdue. Nrmally, verdue tests will nt be executed. With this check, they will be executed as sn as the pll is finished. Cntinue plling server after a cnnect errr: When this is checked, if RemteLaunch receives an errr cnnecting t the SpiraTest server, it will cntinue plling in the future. If this is unchecked, RemteLaunch will switch t the errr status upn encuntering a cnnectin errr. It is imprtant t check this ptin if yur SpiraTest server will be peridically unavailable fr server maintenance. Plling Frequency: Hw ften in minutes the applicatin will pll the SpiraTeam server fr updates t the autmatin hst s schedule. The default is 60 (1 hur), and shuld be fine fr mst installatins. Nte that tests will still be executed n their scheduled time, this is simply hw ften the prgram will talk t the SpiraTeam server t detect schedule changes. Updating the plling frequency will reset the currently running timers. Plling Read Ahead: Hw far ahead in minutes the prgram shuld read the schedule fr the Autmatin hst. Tests that are scheduled farther in advance will nt shw up as a pending test n the status screen Extensin Cnfiguratin If an extensin has custm cnfiguratin ptins, they will appear as separate tabs lcated after the Client Setup tab. The cntents f each tab will vary depending n the extensin. View the extensin s dcumentatin fr ptins given in thse extensins Status Screen The status screen is usually hidden, but can be brught up fr display by duble-clicking n the system tray icn. The tp f the screen shws the current status, whether it s running a test r waiting t pll the server fr an update. It will als shw any errrs present n the applicatin, like a registratin errr r cnfiguratin issue. Under the status bar is a list f any pending r executing tests that are scheduled fr this testing machine. The list will get cleared at every pll, s tests that have executed since the previus pll will still be n the list, and will shw their executin status: Page 4 f 133

6 Green Arrw: A green arrw indicates that the test is still running, r RemteLaunch is waiting fr a reply frm the testing engine / test applicatin. Blue Checkbx: A blue checkbx indicates that the test is cmpleted, regardless f status f the individual test steps in the scheduled test set. Red Errr: A red errr indicatr indicates that the test extensin r the testing applicatin ran int an issue (utside f test results). In this case, any further tests that require the extensin will be marked as blcked, as the issue needs t be crrected within the extensin settings r testing applicatin. N Indicatin: N indicatin means that the test is currently awaiting fr its scheduled date t start. Nte that nly ne test will be launched at a time, s that if tw tests are scheduled at the same time, the ne with the lwer TestSet ID will be executed first, then as sn as it s finished, the secnd scheduled test will be run. By highlighting a test that has nt been executed yet, yu can click the Frce Execute buttn. This will cause the selected test t have its scheduled date t the current time, causing it t be immediately executed (r, if anther test is already running, next in line fr executin). At any time the Frce Pll buttn can be clicked, causing RemteLaunch t initiate an immediate pll f the SpiraTeam server t check fr pending runs. The timers fr the next server pll will be reset when the buttn is clicked Tray Icn Menu Instead f perating frm the applicatin windw, all functins exist n the tray icn menu as well, as well as sme additinal cmmands: Page 5 f 133

7 Pause / Resume: The Pause/Resume ptin pauses r resumes the timers fr plling and executing tests. If a test r server pll is already in prgress, it will nt cancel these. Hwever, after they are finished, n further plls r tests will be run. Pll Nw: This will frce a server pll fr upcming tests, and reset the pll timer. Cnfiguratin: Opens the main windw t the Cnfiguratin page. Help -> Abut: Opens the Abut windw, which displays the current license infrmatin and any laded extensins. Help -> View Help: Opens this PDF file in a brwser. Exit: Will cmpletely exit the prgram. Ding this will cancel any tests currently running and shut dwn the prgram. Any tests that were waiting t be executed will nt execute until the prgram is restarted and the plling is resumed. Yu can duble-click the try icn t bring up the main windw n the Status page. 1.3 Test Executin and Reprting All test handling is perfrmed by the extensin that the autmated tests are cnfigured fr. Test Sets that have multiple Test Cases, the Test Cases will all be executed in rder, sequentially. (N parallel executing.) At the start f executin fr a Test Set, the test set will be updated in SpiraTeam as In Prgress. As tests are perfrmed, the Test Cases will be updated with their status. The Test Set n the status screen will be marked with the executing icn. Once the Test Set is cmpleted, the status f the Test Set will be changed t Cmpleted, and will be marked n the status screen with a cmpleted icn. In case f an uncaught exceptin that is thrwn by the testing extensin, the Test Set will be marked Blcked, and the Test Case will be recrded as Blcked. All ther fllwing tests will nt be run and remain as Nt Run. The Test Set must be reset t be executed again, and it is recmmended t lk int the cause f the errr (recrded in the Blcked Test Case results) and crrect it befre rescheduling the test. This Test Set will be marked with and errr icn. The same results are applied in the case where a Test Set cntains a Test Case that references a testing extensin that is nt installed. Install the extensin and re-run the Test Set. Executing, Cmpleted, and Errr Test Sets are marked with the icns next t their scheduled date in the Status screen. They will stay in the list until the next scheduled server pll. Yu cannt manually re-run them Running RemteLaunch frm a Build Script Nrmally yu schedule tests in SpiraTeam using the Planned Date field f the test sets and let the varius instances f RemteLaunch pll SpiraTeam fr upcming tests. In additin (as described in the SpiraTeam User Manual) yu can execute a test set n the lcal machine immediately by clicking the Execute buttn within SpiraTeam. Hwever there are situatins where yu want t be able t launch an autmated test script using ne f the supprted engines frm an external batch file r build script (e.g. as part f a cntinuus integratin envirnment) and have thse tests reprt their results back int SpiraTeam. Yu can achieve this by using the special cmmand-line argument testset which is passed t RemteLaunch. Fr mre details n this parameter see the next sectin. Page 6 f 133

8 1.5 Cmmand line arguments Fr debugging and additinal ptins when running the prgram, the fllwing cmmand-line arguments are available: -status -paused -pll -trace -lgfile -testset:[test Set ID] <filename> Shws the Status screen upn startup. (Nrmal actin is t run minimized t the system tray. Starts the applicatin with timers Paused instead f active. Frces the prgram t d an initial pll upn startup. (Nrmal actin is t wait the pending time befre ding the initial pll.) Enables tracelgging t the EventLg fr debugging and watching tests execute. Frces events t be written t a text file instead f the Applicatin EventLg. This ptin enables trace as well. Files are lcated in the Lcal Applicatin Data flder. (C:\Users\<user>\AppData\Lcal n Vista/Win7). Allws yu t tell RemteLaunch t execute a specific test set n the remte cmputer (e.g. -testset:45 runs test set TX00045) Must be the last item n the cmmand line. This is a TST file dwnladed frm SpiraTeam t start immediate executin n. Page 7 f 133

9 1.6. Using RemteLaunchX When yu need t run autmated tests n a variety f different platfrms (Windws, MacOS X, Linux, Unix, etc.) the RemteLaunchX crss-platfrm autmated testing agent is a better chice than the standard RemteLaunch GUI applicatin. T start using RemteLaunchX, please g t the Custmer Area f the Inflectra website and dwnlad the latest versin f the RemteLaunchX applicatin. It will be packaged as a simple.zip cmpressed flder that yu can extract nt the target cmputer: The fllwing fur files are included: RemteLaunchX.jar this is the main applicatin, packaged as a Java JAR file. This versin f RemteLaunch requires Java 1.7 SE r later t be installed. cnfig.prperties this cntains all the settings used by RemteLaunchX. Yu will need t edit this file in a text editr t cnfigure RemteLaunchX fr use. RemteLaunchX.bat this is a sample Windws batch file that can be used t simplify running RemteLaunchX n Windws systems. RemteLaunchX.sh this is a sample UNIX/Linux/MacOS X shell script that can be used t run RemteLaunchX n UNIX, Linux r Mac OS X Cnfiguring RemteLaunchX Once yu have extracted the files listed abve, pen up the cnfig.prperties file in a text editr: #This file cntains the cnfiguratin data used by the RemteLaunch-X applicatin #Spira cnnectin infrmatin server-url = server-lgin = fredblggs server-passwrd = PleaseChange #The autmatin hst tken hst-tken = MyHst1 #The license key license-rganizatin: TBD license-key: TBD #The regular expressins fr each f the pssible executin statuses pass-regex =.* fail-regex =.*(Errr Fail Fatal).* cautin-regex =.*(Warning Cautin).* blcked-regex =.*(Blcked).* The fllwing changes need t be made t this cnfiguratin file: Page 8 f 133

10 server-url This is the URL f the SpiraTest r SpiraTeam installatin (hereafter referred t as just SpiraTest). Be sure t nt put /Lgin.aspx r any ther page in the string, this shuld be just the rt URL f the applicatin s install. server-lgin This is the SpiraTest lgin id f the user that yu want the tests reprted as. Nte that while the applicatin is plling and updating test results, if the user is lgged int a web brwser sessin, they will get kicked ut. server-passwrd The passwrd f the SpiraTest lgin listed abve. hst-tken This field is required, and uniquely identifies the lcal testing machine. Any scheduled tests assigned t the Autmatin Hst n SpiraTest will get plled fr this machine. Except in special circumstances, this ID shuld be unique amng all testing machines. Imprtant: This field must match the string that is entered int the Autmatin Hst Details screen in the Tken: field, r scheduled tests will nt be recgnized. license-rganizatin The name f the Organizatin that yur RemteLaunch license key was issued t. This is listed in the Custmer Area f the Inflectra website alngside the license key. Nte: RemteLaunch and RemteLaunchX use the same license keys, s yu dn t need t have a separate RemteLaunchX ne. license-key The RemteLaunch license key that is listed in the secure Custmer Area f the Inflectra website Yu shuld leave the fur regex settings alne fr nw, they can be changed when yu start executing tests and need t fine-tune hw RemteLaunchX interprets the results. Nw that yu have cnfigured the plugin, yu can execute the RemteLaunchX cnsle applicatin by either running the prvided batch / shell cmmand r just executing the JAR file directly: Java jar RemteLaunchX.jar When yu run the applicatin, the fllwing shuld be utput t the cnsle: Starting RemteLaunch... ======================== Server URL: Server Lgin: fredblggs Autmatin Hst: MyHst1 Checking License Key fr: Inflectra Crpratin Prductin License Key in Use. Testing cnnectin t Spira... Successfully cnnected t Spira. WARNING: Unable t retrieve test runs fr SpiraTest prject PR2, s skipping this prject - Autmatin Hst with tken 'MyHst1' desn't exist in prject PR2. WARNING: Unable t retrieve test runs fr SpiraTest prject PR3, s skipping this prject - Autmatin Hst with tken 'MyHst1' desn't exist in prject PR3. Retrieved 0 test run(s) frm SpiraTest. Exiting RemteLaunch... ======================== The system will reprt back zer Test Runs at this pint because nthing has been scheduled in SpiraTest. In the next sectin we shall setup an autmated test set that cntains an autmated test case. Page 9 f 133

11 Setting up Autmated Tests in SpiraTest This sectin assumes that yu already have a wrking installatin f SpiraTest r SpiraTeam and have installed RemteLaunchX n the varius test autmatin hsts fllwing the instructins abve. Once thse prerequisites are in place, please fllw these steps: Lg in t SpiraTeam as a system administratr and g int SpiraTeam main Administratin page and click n the Test Autmatin link under Integratin. Click the Add buttn t enter the new test autmatin engine details page. The fields required are as fllws: Name: This is the shrt display name f the autmatin engine. It can be anything that is meaningful t yur users. Descriptin: This is the lng descriptin f the autmatin engine. It can be anything that is meaningful t yur users. (Optinal) Active: If checked, the engine is active and able t be used fr any prject. Tken: This needs t be the assigned unique tken fr the autmatin engine and is used t tell RemteLaunch which engine t actually use fr a given test case. Fr Cmmand-Line this shuld be simply CmmandLine. Once yu have finished, click the Insert & Clse buttn and yu will be taken back t the Test Autmatin list page, with Cmmand-Line listed as an available autmatin engine. Next yu need t display the list f test cases in SpiraTeam (by clicking Testing > Test Cases) and then add a new test case. Once yu have added the new test case, click n it and select the Autmatin tab: Page 10 f 133

12 Yu need t enter the fllwing fields: Autmatin Engine - Chse the Cmmand-Line Autmatin Engine that yu created in the previus sectin frm the drp-dwn list. Script Type This can be set t Attached r Linked (see belw fr the difference). Filename This needs t cnsist f the fllwing three sectins separated by a pipe ( ) character: 1. The full path t the cmmand-line tl being executed. 2. Any arguments fr the cmmand-line tl. In additin, yu can use the fllwing additinal tkens fr sme f the special RemteLaunchX values: [TestCaseId] the ID f the test case [TestSetId] the ID f the test set [ReleaseId] the ID f the release (if specified) [Filename] - This special tken will be replaced by the actual filename f the test script when RemteLaunchX dwnlads it frm SpiraTeam. 3. The mask fr cnverting any parameter values frm SpiraTeam int valid cmmand line arguments. If parameters are nt accepted by the cmmand-line tl, yu can leave this sectin ut. The mask can include any symbls tgether with name t refer t the parameter name and value t refer t the parameter value. Example 1: If yu want parameters t be prvided in the frm: -param1=value1 param2=value2 yu wuld use the fllwing mask: -name=value Example 2: If yu want parameters t be prvided in the frm: /param1:value1 /param2:value2 yu wuld use the fllwing mask: /name:value Sme example filenames wuld be: C:\Temp\TestApp.exe -arg1 -arg2 -name=value C:\Temp\TestApp.exe -arg1 -arg2 "-arg3=[filename]" where the first ne is fr a Linked test and the secnd ne is fr an Attached test. Dcument Type Yu can chse which dcument type the autmated test script will be categrized under. Dcument Flder Yu can chse which dcument flder the autmated test script will be stred in. Versin The versin f the test script (1.0 is used if n value specified) Test Script Fr Attached test scripts, this needs t cntain the cmplete test script in whatever language and syntax is being expected by the cmmand-line applicatin. Fr Linked test scripts, yu shuld leave this blank. If yu wuld like t have SpiraTeam pass any parameter values t this test script yu can specify them by using the syntax ${parametername} inside the test script. Page 11 f 133

13 here is an advanced feature f SpiraTest/Team and RemteLaunch that lets yu pass parameters frm SpiraTeam t yur cmmand-line autmated testing tl. This is very useful if yu want t have a data-driven test script that be executed multiple times with different parameter values. T setup the autmated test case fr parameters, click n the Edit Parameters hyperlink abve the Test Script bx: The name f the parameter ${lgin} needs t match the name f a parameter accepted by the cmmand-line tl. Once yu are happy with the values, click [Save] t update the test case. Nw yu are ready t schedule the autmated test case fr executin. G t Testing > Autmatin Hsts in SpiraTeam t display the list f autmatin hsts: Make sure that yu have created an Autmatin Hst fr each cmputer that is ging t run an autmated test case. The name and descriptin can be set t anything meaningful, but the Tken field must be set t the same tken that is specified in the RemteLaunchX applicatin n that specific machine. Once yu have at least ne Autmatin Hst cnfigured, g t Testing > Test Sets t create the test sets that will cntain the autmated test case. Nte: Unlike manual test cases, autmated test cases must be executed within a test set they cannt be executed directly frm the test case. Create a new Test Set t hld the Cmmand-Line autmated test cases and click n its hyperlink t display the test set details page: Page 12 f 133

14 Yu need t add at least ne autmated test case t the test set and then cnfigure the fllwing fields: Autmatin Hst This needs t be set t the name f the autmatin hst that will be running the autmated test set. Planned Date The date and time that yu want the scenari t begin. (Nte that multiple test sets scheduled at the exact same time will be scheduled by Test Set ID rder.) Status This needs t be set t Nt Started fr RemteLaunch t pick up the scheduled test set. When yu change the Planned Date, the status autmatically switches back t Nt Started Type This needs t be set t Autmated fr autmated testing If yu have parameterized test cases inside the autmated test set yu need t set their values by rightclicking n the test case and chsing Edit Parameters : Enter the parameter values and click Update t cmmit the change. This allws yu t have the same test case in the test set multiple times with different data fr each instance. Page 13 f 133

15 Running RemteLaunchX Once yu have set the varius test set fields (as described abve), yu are nw ready t execute RemteLaunchX. Yu can execute the RemteLaunchX cnsle applicatin by either running the prvided batch / shell cmmand r just executing the JAR file directly: Java jar RemteLaunchX.jar When yu run the applicatin, the fllwing shuld be utput t the cnsle: Starting RemteLaunch... ======================== Server URL: Server Lgin: fredblggs Autmatin Hst: MyHst1 Checking License Key fr: Inflectra Crpratin Prductin License Key in Use. Testing cnnectin t Spira... Successfully cnnected t Spira. WARNING: Unable t retrieve test runs fr SpiraTest prject PR2, s skipping this prject - Autmatin Hst with tken 'MyHst1' desn't exist in prjec PR2. WARNING: Unable t retrieve test runs fr SpiraTest prject PR3, s skipping this prject - Autmatin Hst with tken 'MyHst1' desn't exist in prjec PR3. Retrieved 1 test run(s) frm SpiraTest. Executing test case TC18 with filename 'C:\Windws\System32\ipcnfig.exe /all' This is a Linked test script Executing cmmand 'C:\Windws\System32\ipcnfig.exe' with arguments '/all' Executin Status = Passed Exiting RemteLaunch... ======================== The cnsle utput will indicate which test sets are being executed and what the final result was. Inside SpiraTest, nce executin begins the status f the test set will change frm Nt Started t In Prgress, and nce test executin is dne, the status f the test set will change t either Cmpleted the autmatin engine culd be launched and the test has cmpleted (passed r failed) r Blcked RemteLaunchX was nt able t execute the test. In additin, the individual test cases in the set will display a status based n the results f the cmmandline test that was executed: Passed The autmated test ran successfully and matched the PASS regular expressin. Failed The autmated test ran successfully, and matched the FAIL regular expressin. Cautin The autmated test ran successfully, and matched the CAUTION regular expressin. Blcked The autmated test did nt run successfully r it matched the BLOCKED regular expressin. If yu receive the Blcked status fr either the test set r the test cases yu shuld pen up the Test Run that was recrded and the Cnsle utput sectin will cntain the underlying errr message(s). Once the tests have cmpleted, yu can lg back int SpiraTest and see the executin status f yur test cases. If yu click n a Test Run that was generated by the cmmand-line tl, yu will see the fllwing infrmatin: Page 14 f 133

16 This screen indicates the status f the test run that was reprted back frm cmmand-line tl tgether with any messages r ther infrmatin. The executin status will be set accrding t the rules described abve, the Message field will cntain the first line f cnsle utput and the large details bx will cntain the full cnsle utput frm the cmmand-line tl. Cngratulatins Yu are nw able t run a custm cmmand-line test and have the results be recrded within SpiraTest / SpiraTeam Scheduling RemteLaunchX Unlike the main RemteLaunch applicatin, RemteLaunchX des nt have a built-in timer and s when executed it will run nce, check fr pending test sets and then exit. If yu want t have it run n a peridic basis, yu will need t schedule it externally. If yu are using Micrsft Windws yu wuld use the Windws Task Scheduler and in ther perating systems yu wuld setup a CRON jb. We recmmend scheduling RemteLaunchX t run every 5 minutes Custmizing the Reprting By default, RemteLaunchX will use the fllwing rules t determine if a test has passed, failed, blcked r passed with warnings (cautin): Passed The test cmpleted and the cnsle utput didn t cntain any f the errr phrases listed in the ther rules (belw). Failed The test cmpleted and the cnsle utput cntained the phrases Errr, Fail r Fatal. Cautin The test cmpleted and the cnsle utput cntained the phrases Warning, r Cautin. Blcked The autmated test did nt run successfully r the cnsle utput cntained the phrase Blcked. Page 15 f 133

17 Yu can custmize the reprting by changing the Regular Expressins (Regex) stred in the cnfig.prperties files: #The regular expressins fr each f the pssible executin statuses pass-regex =.* fail-regex =.*(Errr Fail Fatal).* cautin-regex =.*(Warning Cautin).* blcked-regex =.*(Blcked).* Page 16 f 133

18 2. QuickTestPr Engine HP QuickTest Prfessinal (hereafter QTP) is an autmated functinal test autmatin system that lets yu recrd applicatin peratins and generate VBA test autmatin scripts that can be used t playback the test script against the test applicatin. HP Unified Functinal Testing (hereafter UFT) is an updated versin f QTP that als includes functinality fr web service testing. This sectin describes hw yu can use SpiraTest / SpiraTeam (hereafter SpiraTeam) tgether with RemteLaunch t schedule and remtely launch instances f QTP and UFT n different cmputers and have the testing results be transmitted back t SpiraTeam. This allws yu t extend yur SpiraTeam s test management capabilities t include autmated QTP and UFT tests. Nte: This integratin requires at least versin 3.0 f SpiraTest/Team and versin 9.0 f Quick Test Prfessinal. Fr accessing UFT, yu d need at least versin 4.0 f SpiraTest/Team and versin 11.0 f UFT Installing the QTP/UFT Engine This sectin assumes that yu already have a wrking installatin f SpiraTest r SpiraTeam and have installed RemteLaunch n the varius test autmatin hsts fllwing the instructins in Sectin 1 (abve). Once thse prerequisites are in place, please fllw these steps: Dwnlad and extract the QuickTestPrAutmatinEngine.zip file frm the Inflectra website and lcate the apprpriate QuickTestPrX.dll r UftX.dll fr the versin f QTP r UFT that yu are using. If yu dn t see the versin listed, just use the nearest versin that is lwer than yur current versin. Cpy the file QuickTestPrX.dll r UftX.dll (where X is the apprpriate versin) int the extensins sub-flder f the RemteLaunch installatin. Lg in t SpiraTeam as a system administratr and g int SpiraTeam main Administratin page and click n the Test Autmatin link under Integratin. Click the Add buttn t enter the new test autmatin engine details page. The fields required are as fllws: Name: This is the shrt display name f the autmatin engine. It can be anything that is meaningful t yur users. Page 17 f 133

19 Descriptin: This is the lng descriptin f the autmatin engine. It can be anything that is meaningful t yur users. (Optinal) Active: If checked, the engine is active and able t be used fr any prject. Tken: This needs t be the assigned unique tken fr the autmatin engine and is used t tell RemteLaunch which engine t actually use fr a given test case. Fr QTP this shuld be QuickTestPrX where X is the versin number f the DLL file that yu are using. Fr UFT this shuld be UftX where X is the versin number f the DLL file that yu are using. Once yu have finished, click the Insert & Clse buttn and yu will be taken back t the Test Autmatin list page, with QTP listed as an available autmatin engine Setting up the Autmated Test Cases This sectin describes the prcess fr setting up a test case in SpiraTeam fr autmatin and linking it t an autmated QTP r UFT test script. First yu need t display the list f test cases in SpiraTeam (by clicking Testing > Test Cases) and then add a new test case. Once yu have added the new test case, click n it and select the Autmatin tab: Yu need t enter the fllwing fields: Autmatin Engine - Chse the QTP/UFT Autmatin Engine that yu created in the previus sectin frm the drp-dwn list. Script Type This shuld be set t Linked as the integratin with QTP/UFT nly supprts referencing QTP/UFT test script flder paths and nt physically uplading the test scripts int SpiraTeam. Filename This needs t be the full path t the QTP/UFT test script flder (i.e. the flder that yu pen in QTP/UFT t run the test). T make this easier acrss different machines, yu can use several cnstants fr standard Windws lcatins (see example in screensht): [MyDcuments] The user s My Dcuments flder. The user indicated is the user that ran RemteLaunch. [CmmnDcuments] The Public Dcument s flder. [DesktpDirectry] The user s Desktp flder. The user indicated is the user that ran RemteLaunch. Page 18 f 133

20 [PrgramFiles] Translated t the Prgram Files directry. Fr 64-bit machines, it s the 64-bit directry. [PrgramFilesX86] Translated t the 32-bit Prgram Files directry. Dcument Type If using SpiraTeam (nt SpiraTest) yu can chse which dcument type the autmated test script will be categrized under. Dcument Flder If using SpiraTeam (nt SpiraTest) yu can chse which dcument flder the autmated test script will be stred in. Versin The versin f the test script (1.0 is used if n value specified) Test Script This is nt used with the QTP/UFT Engine since it nly supprts linked test scripts. Once yu are happy with the values, click [Save] t update the test case. Nw yu are ready t schedule the autmated test case fr executin Using Parameterized Test Cases There is an advanced feature f SpiraTest/Team and RemteLaunch that lets yu pass parameters frm SpiraTeam t yur QTP/UFT autmated test script. This is very useful if yu have a data-driven QTP/UFT test script that accepts input parameters frm an external data surce. T setup the autmated test case fr parameters, click n the Test Steps tab and click n Edit Parameters : The name f the parameter ${lgin} needs t match the name f the input parameter defined within the QTP/UFT script in its input parameters cnfiguratin Executing the QTP/UFT Test Sets frm SpiraTeam There are tw ways t execute autmated test cases in SpiraTeam: 1. Schedule the test cases t be executed n a specific cmputer (lcal r remte) at a date/time in the future 2. Execute the test cases right nw n the lcal cmputer. We shall utline bth f these tw scenaris in this sectin. Hwever first we need t setup the apprpriate autmatin hsts and test sets in SpiraTeam: Cnfiguring the Autmatin Hsts and Test Sets G t Testing > Autmatin Hsts in SpiraTeam t display the list f autmatin hsts: Page 19 f 133

21 Make sure that yu have created an Autmatin Hst fr each cmputer that is ging t run an autmated test case. The name and descriptin can be set t anything meaningful, but the Tken field must be set t the same tken that is specified in the RemteLaunch applicatin n that specific machine. Once yu have at least ne Autmatin Hst cnfigured, g t Testing > Test Sets t create the test sets that will cntain the autmated test case: Nte: Unlike manual test cases, autmated test cases must be executed within a test set they cannt be executed directly frm the test case. Create a new Test Set t hld the QTP/UFT autmated test cases and click n its hyperlink t display the test set details page: Yu need t add at least ne autmated test case t the test set and then cnfigure the fllwing fields: Autmatin Hst This needs t be set t the name f the autmatin hst that will be running the autmated test set. Planned Date The date and time that yu want the scenari t begin. (Nte that multiple test sets scheduled at the exact same time will be scheduled by Test Set ID rder.) Status This needs t be set t Nt Started fr RemteLaunch t pick up the scheduled test set. When yu change the Planned Date, the status autmatically switches back t Nt Started Type This needs t be set t Autmated fr autmated testing Page 20 f 133

22 If yu have parameterized test cases inside the autmated test set yu need t set their values by rightclicking n the test case and chsing Edit Parameters : Enter the parameter values and click Update t cmmit the change. This allws yu t have the same test case in the test set multiple times with different data fr each instance Executing the Test Sets Once yu have set the varius test set fields (as described abve), the Remte Launch instances will peridically pll SpiraTeam fr new test sets. Once they retrieve the new test set, they will add it t their list f test sets t be execute. Once executin begins they will change the status f the test set t In Prgress, and nce test executin is dne, the status f the test set will change t either Cmpleted the autmatin engine culd be launched and the test has cmpleted r Blcked RemteLaunch was nt able t start the autmatin engine. If yu want t immediately execute the test case n yur lcal cmputer, instead f setting the Autmatin Hst, Status and Planned Date fields, yu can instead click the [Execute] icn n the test set itself. This will cause RemteLaunch n the lcal cmputer t immediately start executing the current test set. In either case, nce all the test cases in the test set have been cmpleted, the status f the test set will switch t Cmpleted and the individual test cases in the set will display a status based n the results f the QTP/UFT test: Passed The QTP/UFT autmated test ran successfully and all the test cnditins in the test script passed Failed The QTP/UFT autmated test ran successfully, but at least ne test cnditin in the test script failed. Blcked The QTP/UFT autmated test did nt run successfully If yu receive the Blcked status fr either the test set r the test cases yu shuld pen up the Windws Applicatin Event Lg n the cmputer running RemteLaunch and lk in the event lg fr errr messages. Nte: While the tests are executing yu may see brwser r applicatin windws launch as QuickTest Pr (QTP) r Unified Functinal Testing (UFT) executes the apprpriate tests. Once the tests have cmpleted, yu can lg back int SpiraTeam and see the executin status f yur test cases. If yu click n a Test Run that was generated by QTP/UFT, yu will see the fllwing infrmatin: Page 21 f 133

23 This screen indicates the status f the test run that was reprted back frm QTP/UFT tgether with any messages r ther infrmatin. The Test Name indicates the name f the test inside QTP/UFT, and the executin status crrespnds the matching status inside QTP/UFT as illustrated belw: QTP/UFT Status Passed Failed Warning Stpped Nt Applicable (Any ther status) SpiraTeam Status Passed Failed Cautin Blcked N/A Nt Run In additin, the detailed test reprt frm QTP/UFT is available in the large text-bx belw. It will cntain messages such as: QuickTest Prfessinal Test: Test1 Results Name: Res21 Run Started: 10/22/ :49:06 Run Ended: 10/22/ :49:10 Summary Results ======= ======= Passed: 0 Failed: 0 Warnings: 0 Detailed Results ======= ======= Iteratin: 1 ============= Actin: Lg In T Flight ============= Page 22 f 133

24 Step: Lgin: Dialg Step: Agent Name:.SetText: "Bbba Fett" Step: Agent Name:.Type: "&lt MicTab&gt" Step: Passwrd:.SetSecureText: "4cc08e b35bb8a7dab8442c69b8441f3e" Step: OK.Click: Step: Flight Reservatins: Dialg Step: OK.Click: Cngratulatins Yu are nw able t run QTP/UFT autmated functinal tests and have the results be recrded within SpiraTest / SpiraTeam. Page 23 f 133

25 3. SmarteScript Engine SmarteSft SmarteScript (hereafter SmarteScript) is a Graphic User Interface (GUI) script-free functinal test autmatin system that lets yu recrd applicatin peratins by capturing the varius testable bjects f the applicatin and then playback the peratins t autmatically test the applicatin. This sectin describes hw yu can use SpiraTest / SpiraTeam (hereafter SpiraTeam) tgether with RemteLaunch t schedule and remtely launch instances f SmarteScript n different cmputers and have the testing results be transmitted back t SpiraTeam. This allws yu t extend yur SpiraTeam s test management capabilities t include autmated SmarteScript tests. Nte: This integratin requires at least versin 3.0 f SpiraTest/Team and versin 5.0 f SmarteScript Installing the SmarteScript Engine This sectin assumes that yu already have a wrking installatin f SpiraTest r SpiraTeam and have installed RemteLaunch n the varius test autmatin hsts fllwing the instructins in Sectin 1 (abve). Once thse prerequisites are in place, please fllw these steps: Dwnlad and extract the SmarteScriptAutmatinEngine.zip file frm the Inflectra website and lcate the apprpriate SmarteScriptX.dll fr the versin f SmarteScript that yu are using. If yu dn t see the versin listed, just use the nearest versin that is lwer than yur current versin. Cpy the file SmarteScriptX.dll (where X is the apprpriate versin) int the extensins subflder f the RemteLaunch installatin. Lg in t SpiraTeam as a system administratr and g int SpiraTeam main Administratin page and click n the Test Autmatin link under Integratin. Click the Add buttn t enter the new test autmatin engine details page. The fields required are as fllws: Name: This is the shrt display name f the autmatin engine. It can be anything that is meaningful t yur users. Descriptin: This is the lng descriptin f the autmatin engine. It can be anything that is meaningful t yur users. (Optinal) Active: If checked, the engine is active and able t be used fr any prject. Tken: This needs t be the assigned unique tken fr the autmatin engine and is used t tell RemteLaunch which engine t actually use fr a given test case. Fr Page 24 f 133

26 SmarteScript this shuld be SmarteScriptX where X is the versin number f the DLL file that yu are using. Once yu have finished, click the Insert & Clse buttn and yu will be taken back t the Test Autmatin list page, with SmarteScript listed as an available autmatin engine Setting up the Autmated Test Cases This sectin describes the prcess fr setting up a test case in SpiraTeam fr autmatin and linking it t an autmated SmarteScript test script. First yu need t display the list f test cases in SpiraTeam (by clicking Testing > Test Cases) and then add a new test case. Once yu have added the new test case, click n it and select the Autmatin tab: Yu need t enter the fllwing fields: Autmatin Engine - Chse the SmarteScript Autmatin Engine that yu created in the previus sectin frm the drp-dwn list. Script Type This shuld be set t Linked as the integratin with SmarteScript nly supprts referencing SmarteScript test script file (.ses) and nt physically uplading the test scripts int SpiraTeam. Filename This needs t be the full path t the SmarteScript test script (i.e. the.ses file that yu pen in SmarteScript t run the test). T make this easier acrss different machines, yu can use several cnstants fr standard Windws lcatins (see example in screensht): [MyDcuments] The user s My Dcuments flder. The user indicated is the user that ran RemteLaunch. [CmmnDcuments] The Public Dcument s flder. [DesktpDirectry] The user s Desktp flder. The user indicated is the user that ran RemteLaunch. [PrgramFiles] Translated t the Prgram Files directry. Fr 64-bit machines, it s the 64-bit directry. [PrgramFilesX86] Translated t the 32-bit Prgram Files directry. Dcument Type If using SpiraTeam (nt SpiraTest) yu can chse which dcument type the autmated test script will be categrized under. Dcument Flder If using SpiraTeam (nt SpiraTest) yu can chse which dcument flder the autmated test script will be stred in. Versin The versin f the test script (1.0 is used if n value specified) Page 25 f 133

27 Test Script This is nt used with the SmarteScript Engine since it nly supprts linked test scripts. Once yu are happy with the values, click [Save] t update the test case. Nw yu are ready t schedule the autmated test case fr executin Using Parameterized Test Cases SmarteScript des nt supprt the passing f input test parameters s the SmarteScript autmatin engine des nt supprt this feature f SpiraTeam r RemteLaunch Executing the SmarteScript Test Sets frm SpiraTeam There are tw ways t execute autmated test cases in SpiraTeam: 1. Schedule the test cases t be executed n a specific cmputer (lcal r remte) at a date/time in the future 2. Execute the test cases right nw n the lcal cmputer. We shall utline bth f these tw scenaris in this sectin. Hwever first we need t setup the apprpriate autmatin hsts and test sets in SpiraTeam: Cnfiguring the Autmatin Hsts and Test Sets G t Testing > Autmatin Hsts in SpiraTeam t display the list f autmatin hsts: Make sure that yu have created an Autmatin Hst fr each cmputer that is ging t run an autmated test case. The name and descriptin can be set t anything meaningful, but the Tken field must be set t the same tken that is specified in the RemteLaunch applicatin n that specific machine. Once yu have at least ne Autmatin Hst cnfigured, g t Testing > Test Sets t create the test sets that will cntain the autmated test case: Nte: Unlike manual test cases, autmated test cases must be executed within a test set they cannt be executed directly frm the test case. Create a new Test Set t hld the SmarteScript autmated test cases and click n its hyperlink t display the test set details page: Page 26 f 133

28 Yu need t add at least ne autmated test case t the test set and then cnfigure the fllwing fields: Autmatin Hst This needs t be set t the name f the autmatin hst that will be running the autmated test set. Planned Date The date and time that yu want the scenari t begin. (Nte that multiple test sets scheduled at the exact same time will be scheduled by Test Set ID rder.) Status This needs t be set t Nt Started fr RemteLaunch t pick up the scheduled test set. When yu change the Planned Date, the status autmatically switches back t Nt Started Type This needs t be set t Autmated fr autmated testing Executing the Test Sets Once yu have set the varius test set fields (as described abve), the Remte Launch instances will peridically pll SpiraTeam fr new test sets. Once they retrieve the new test set, they will add it t their list f test sets t be execute. Once executin begins they will change the status f the test set t In Prgress, and nce test executin is dne, the status f the test set will change t either Cmpleted the autmatin engine culd be launched and the test has cmpleted r Blcked RemteLaunch was nt able t start the autmatin engine. If yu want t immediately execute the test case n yur lcal cmputer, instead f setting the Autmatin Hst, Status and Planned Date fields, yu can instead click the [Execute] icn n the test set itself. This will cause RemteLaunch n the lcal cmputer t immediately start executing the current test set. In either case, nce all the test cases in the test set have been cmpleted, the status f the test set will switch t Cmpleted and the individual test cases in the set will display a status based n the results f the SmarteScript test: Passed The SmarteScript autmated test ran successfully and all the test cnditins in the test script passed Failed The SmarteScript autmated test ran successfully, but at least ne test cnditin in the test script failed. Blcked The SmarteScript autmated test did nt run successfully Page 27 f 133

29 If yu receive the Blcked status fr either the test set r the test cases yu shuld pen up the Windws Applicatin Event Lg n the cmputer running RemteLaunch and lk in the event lg fr errr messages. Nte: While the tests are executing yu may see brwser r applicatin windws launch as SmarteScript executes the apprpriate tests. Once the tests have cmpleted, yu can lg back int SpiraTeam and see the executin status f yur test cases. If yu click n a Test Run that was generated by SmarteScript, yu will see the fllwing infrmatin: This screen indicates the status f the test run that was reprted back frm SmarteScript tgether with any messages r ther infrmatin. The Test Name indicates the name f the test inside SmarteScript, and the executin status crrespnds the matching status inside SmarteScript. Cngratulatins Yu are nw able t run SmarteScript autmated functinal tests and have the results be recrded within SpiraTest / SpiraTeam. Page 28 f 133

30 4. TestCmplete Engine SmarteBear TestCmplete (hereafter TestCmplete) is an autmated functinal test autmatin system that lets yu recrd applicatin peratins and generate test autmatin scripts in a variety f languages (JavaScript, C#, VBScript). These test scripts can then be used t playback the test script against the test applicatin and verify that it wrks crrectly. This sectin describes hw yu can use SpiraTest / SpiraTeam (hereafter SpiraTeam) tgether with RemteLaunch t schedule and remtely launch instances f TestCmplete n different cmputers and have the testing results be transmitted back t SpiraTeam. This allws yu t extend yur SpiraTeam s test management capabilities t include autmated TestCmplete tests. Nte: This integratin requires at least versin 3.0 f SpiraTest/Team and versin 8.0 f TestCmplete Installing the TestCmplete Engine This sectin assumes that yu already have a wrking installatin f SpiraTest r SpiraTeam and have installed RemteLaunch n the varius test autmatin hsts fllwing the instructins in Sectin 1 (abve). Once thse prerequisites are in place, please fllw these steps: Dwnlad and extract the TestCmpleteAutmatinEngine.zip file frm the Inflectra website and lcate the apprpriate TestCmpleteX.dll r TestExecuteX.dll fr the versin f TestCmplete r TestExecute that yu are using. If yu dn t see the versin listed, just use the nearest versin that is lwer than yur current versin. Cpy the file TestCmpleteX.dll r TestExecuteX.dll (where X is the apprpriate versin) int the extensins sub-flder f the RemteLaunch installatin. Lg in t SpiraTeam as a system administratr and g int SpiraTeam main Administratin page and click n the Test Autmatin link under Integratin. Click the Add buttn t enter the new test autmatin engine details page. The fields required are as fllws: Name: This is the shrt display name f the autmatin engine. It can be anything that is meaningful t yur users. Descriptin: This is the lng descriptin f the autmatin engine. It can be anything that is meaningful t yur users. (Optinal) Active: If checked, the engine is active and able t be used fr any prject. Page 29 f 133