TuchTest TM Appceleratr Jenkins CI Tutrial
TuchTest TM Appceleratr Jenkins CI Tutrial 2015, SOASTA, Inc. All rights reserved. The names f actual cmpanies and prducts mentined herein may be the trademarks f their respective cmpanies. This dcument is fr infrmatinal purpses nly. SOASTA makes n warranties, express r implied, as t the infrmatin cntained within this dcument.
Table f Cntents Prerequisites... 1 Third Party Prerequisites... 1 ios-only Prerequisites... 1 Abut Shell Steps and the ios Signing/Prvisining Prerequisite... 2 Andrid-Only Prerequisites... 3 CludTest Utilities and Plugins... 3 Test Cmpsitin Prerequisites... 4 CludTest Cntinuus Integratin Supprt... 5 Installing the SOASTA CludTest Jenkins/Hudsn Plugin... 5 Installing the GitHub Plugin... 7 Static vs. Dynamic Instrumentatin... 9 Jenkins Wrkflws fr TuchTest... 9 Dynamic Instrumentatin f an APP @ile... 10 Dynamic Instrumentatin f an IPA @ile... 10 Static Instrumentatin f an Xcde Prject... 10 Dynamic Instrumentatin f an APK @ile... 10 Static Instrumentatin f an Andrid Prject... 11 Creating a Jenkins Jb... 12 Using the CludTest Jenkins Plugin, MakeAppTuchTestable... 15 Abut the Titanium SDK Path... 15 Using MATT n an ios APP Bundle (Dynamic Instrumentatin)... 16 Using MATT n an IPA (Dynamic Instrumentatin fr Device)... 16 Using MATT n a Prject (Static Instrumentatin fr Simulatrs/Devices)... 18 Using MATT n an APK (Dynamic)... 19 Using MATT n a Prject (Static)... 22 Build the Titanium Prject... 24 Preparing the App fr Simulatrs and Devices... 26 Build the APP fr a Simulatr... 26 Deplying the APP File n a Simulatr... 27 Installing the IPA Archive n iphnes and ipads... 28 Using SCmmand t Play One r Mre Cmpsitins... 29 i
Adding the Publish Junit Test Result Reprts Step... 31 Building the Prject... 31 Inspecting Test Results in Jenkins... 33 Appendix: Inspecting the Mbile App in CludTest... 1 Appendix: Cmpleted Execute Shell Script... 1 ii
Prerequisites This tutrial guides the user thrugh the prcess f using the Jenkins cntinuus integratin tl cmbined with the CludTest Jenkins/Hudsn Plugin in tandem with an example Titanium prject and precnfigured CludTest test cmpsitins. This tutrial prvides guidance fr tw audiences: Users wh wuld like t add Titanium ios r Andrid Testing t a pre-existing Jenkins setup Users wh are either Titanium ios Develpers r Andrid Develpers starting ut with TuchTest wh wuld als like t add TuchTest t a cntinuus integratin setup TIP: If yur rganizatin is nt already using Jenkins refer t the dcumentatin n the Use Jenkins page t get started. Additinal Jenkins installatin references are included at the end f this guide. A Jenkins jb will be defined that uses git t retrieve the sample prject, makes that prject tuchtestable using the MakeAppTuchTestable utility, and which then deplys the cmpiled app t multiple devices using a deplyment script; after which several preexisting test cmpsitins are called t run silently in CludTest and n the specified devices. Finally, CludTest results are inspected inline in Jenkins, als via the CludTest Jenkins/Hudsn Plugin. Nte: Third Party Prerequisites T get started, yu'll need: The Appceleratr sample mbile app, KitchenSink, which is available frm GitHub, is used as the example prject in this guide. Fr ios, a Macintsh cmputer with sufficient USB prts t run the desired number f devices, pssibly using a USB hub Fr Andrid, a cmputer with sufficient USB prts t run the desired number f devices, pssibly using a USB hub Jenkins cntinuus integratin sftware (this can be n the same Mac that is running Titanium r n a different Mac nde) This tutrial assumes a minimal familiarity with Appceleratr Titanium Studi and the fllwing prerequisites: Titanium Studi is installed with Titanium SDK versin 2.1.3 r abve. ios-only Prerequisites The remaining ios prerequisites are cmmn t any Appceleratr prject that utilizes the ios Develper Prgram t install an ios app n a device: The develper is enrlled in the ios Develper Prgram (s that the Appceleratr app can be pushed t the ios Device via itunes). 1
The Unique Device Identifier (UDID) f the ios Device that will be used t test must be registered at the Apple Prvisining Prtal. itunes 10.x r greater must be installed n the desktp client that is running Titanium Studi. One f the key steps during an ios autmated build is deplying the app t yur test device, withut requiring any human interactin. Typical slutins (e.g. ver-the-air distributin) require that the user accept a prmpt. SOASTA TuchTest includes a tl (built using AppleScript) that silently deplys an app. It des this by autmating the prcess f pening the prject in Titanium and running it, which causes Xcde t deply. T use the deply script, yu will need the fllwing: 1. A dedicated machine running Mac OS X, with Xcde. If yu are using Jenkins r Hudsn, this can be either the master nde r a slave. 2. One r mre tethered devices. If yu have mre devices than USB inputs, yu can use a USB hub. Nte als that sufficient pwer t prevent the device frm running dwn unexpectedly shuld be available via that USB input. A nte n tethering: SOASTA TuchTest des nt require tethering fr recrding r playback. Hwever, yu d need t tether the device fr silent deplyment f yur app. 3. The device(s) shuld have the ios Aut-Lck setting set t Never. Abut Shell Steps and the ios Signing/Prvisining Prerequisite In general, the best practice fr all shell steps presented in this tutrial is t first execute all f them frm the cmmand line. With respect t ios signing and prvisining, yu must perfrm the fllwing ne-time prcedure frm Terminal t sign each prfile/applicatin cmbinatin that is in its first use. Prvisining will nt be silent until this is dne. TIP: If signing is nt dne, then a "User interactin nt allwed" errr will ccur in the build. This is because the Operating System requires a user t sign an applicatin using a specific Prvisining prfile fr the first time. In that case, the Operating System needs a user t authrize the signing prcess. Use the fllwing steps: 1. In Terminal, run the xcrun cmmand frm yur Jenkins jb manually. Nte: If yu are using the Jenkins $WORKSPACE variable, yu'll need t change it t a Mac OS X path t wrk here. When yu d that, OS will ppup a dialg bx asking permissin fr the signing prcess. 2. Chse "Always allw" and this errr will nt shw up again. 2
Andrid-Only Prerequisites This tutrial uses Apache Ant with Andrid SDK. Learn mre abut the Andrid SDK here, and Apache Ant here. The Minimum Andrid Versin supprted fr use with TuchTest is 2.3.3 (Gingerbread). The Minimum Apache Ant versin required fr using the SOASTA CludTest Jenkins Plugin is 1.8.0 r later. The Jenkins GitHub Plugin is used t retrieve the prject used in this tutrial (instructins fr installing it are presented belw). Refer t the git site here. Each Andrid device must have the TuchTest Agent app installed and Cnnected at runtime (if mbile web testing will ccur then the TuchTest Web app is als required). Each Andrid device shuld have the fllwing Settings: In the device settings, tap Develper Optins and check the USB Debugging bx. In the device settings, tap "Security" and then tap t check the Unknwn surces bx. CludTest Utilities and Plugins Befre prceeding, dwnlad the fllwing CludTest plugin and utility sftware frm the CludTest Welcme page, Dwnlads sectin. CludTest Jenkins/Hudsn Plugin (this Jenkins/Hudsn plugin will be installed using the Jenkins plugin interface). Jenkins CludTest Plugin versin 2.9 r later and TuchTest 51.07 r later are required fr dynamic instrumentatin. CludTest MakeAppTuchTestable Utility (this utility will be called at the apprpriate time via a Jenkins jb using an Execute Shell build step) 3
Nte: The CludTest user specified t run the MakeAppTuchTestable utility must be a user with Mbile Device Administratr rights. CludTest Cmmand Line Client (als knwn as scmmand, this cmmand line interface utility will be called at the apprpriate time via a Jenkins jb using the CludTest Jenkins Plugin's MakeAppTuchTestable, Play Cmpsitin(s) build step). It is nt necessary t dwnlad the cmmand-line utility since the Plugin als handles this task. Fr ios nly, the CludTest ios App Installer Utility is required (this utility cntains tw executables; the is_app_installer, which is used t install IPA files t ios physical devices; and the is_sim_launcher, which is used t install cmpiled APP bundle files). This archive cntains tw executables: Fr deplyment t Simulatrs, use the is_sim_launcher executable fund in the ios App Installer Utility at the apprpriate time(s) via a Jenkins jb using an Execute Shell build step. Fr deplyment t iphne and ipad devices, use the ios App Installer Utility t deply.ipa archives t the physical device(s). This executable can be called at the apprpriate time(s) via a Jenkins jb using an Execute Shell build step. The apprpriate executable will be called at the apprpriate time(s) via a Jenkins jb using an Execute Shell build step. Test Cmpsitin Prerequisites The test cmpsitins yu will use must already exist n the CludTest instance that yu specify and yu must use the crrect SOASTA Repsitry path t invke them. The test cmpsitin specifies the device(s) that it will run n. If yu re a new TuchTest user, refer t the fllwing dcumentatin befre prceeding with this tutrial. 4
CludTest Cntinuus Integratin Supprt SOASTA CludTest includes first-class supprt fr including test utput in build reprts fr Jenkins and Hudsn via the SCmmand utility. Additinally, the Jenkins/Hudsn Plugin prvides visual integratin with CludTest dashbards within Jenkins itself. When yu run test cmpsitins via SCmmand, CludTest autmatically utputs JUnitXML-cmpatible test utput. Since Jenkins prvides ut-f-the-bx JUnit supprt these result details frm a given test cmpsitin run in Jenkins using SCmmand will appear n the crrespnding Test Results page in Jenkins. By placing all its XML int a directry that we als prvide t Jenkins as a part f defining a given jb, we can easily display these JUnit-friendly test results inside Jenkins. Using this cnfiguratin, a Jenkins Test Result detail page will display details frm CludTest. While it is nt necessary t install the CludTest Jenkins/Hudsn Plugin t utilize CludTest s JUnitXML-frmatted utput in Jenkins in this manner, plugin installatin adds the capability t als jump t specific test cmpsitin errrs in the CludTest Result Details dashbard frm within Jenkins. In the screensht abve, the errr heading and detail text are an utput f SCmmand fr the test shwn. Clicking the link, and prviding CludTest credentials, will then display the precise failure inline in Jenkins fr the given CludTest result. Installing the SOASTA CludTest Jenkins/Hudsn Plugin Use the fllwing steps t install the CludTest Jenkins Plugin (versin 2.9 r later are required fr dynamic instrumentatin as well as t autmatically update the CludTest Jenkins Plugin's MakeAppTuchTestable) frm within yur Jenkins instance. The MATT mdule is aut-updated, s nce it's installed the mst current versin is assured. 1. In Jenkins, click Manage Jenkins, and then click Manage Plugins. 5
2. Click the Plugin Manager, Advanced tab. 3. Lcate the SOASTA CludTest Plugin in the list (using Cmd+F and "Sasta" is a quick way t lcate it). 4. Click the Install withut Restart buttn at the bttm f the page. The Installing Plugins/Upgrades page appears and indicates success nce the install cmpletes. 6
Befre using the CludTest Jenkins Plugin, yu will need t prvide the CludTest server URL and user credentials via the Manage Jenkins > Cnfigure System page, CludTest sectin. We recmmend creating a dedicated CludTest accunt fr Jenkins t use. Installing the GitHub Plugin Use the fllwing steps t install the GitHub Plugin frm within yur Jenkins instance. This will allw us t use GitHub as a Surce Cde Repsitry t retrieve the example prject, KitchenSink. TIP: Git is a distributed versin cntrl system used fr sftware develpment. It is nt necessary t signup r lgin t GitHub in rder t checkut the cde using the fllwing cmmand. In the Surce Cde Management sectin, click Git. If yu are using yur wn app, yu can skip this requirement and then substitute the SCM tl and repsitry t use in the place f Git. 1. In Jenkins, click Manage Jenkins, and then click Manage Plugins. 2. Click the Plugin Manager, Advanced tab 3. Lcate the GitHub Plugin and check it as well. Installing the GitHub Plugin will als install the GitHub Plugin. 7
Yu can verify plugin installatin n the Manage Plugins, Installed tab: Nte: We will specify a Git repsitry t clne in a later step in the Jenkins jb. 8
Static vs. Dynamic Instrumentatin The CludTest Jenkins Plugin's MATT mdule supprts tw instrumentatin methds: static and dynamic. The MATT mdule plays a rle in making either the ios r Andrid prject r its cmpiled APP bundle, IPA, r APK TuchTestable. Dynamic instrumentatin ccurs when MATT instruments a cmpiled file (i.e. an APP bundle flder, an IPA file, r APK file). This methd requires that yu cmpile yur prject first t create an APP, IPA, r APK, after which it can be instrumented using SOASTA 51.07 r later (TuchTest 7040.58). Dynamic instrumentatin is available fr all supprted Andrid versins, while fr ios versin 6 r later is required. Static instrumentatin ccurs when MATT instruments an ios Andrid prject. This methd requires that yu apply MATT t the prject prir t building the APK. Static instrumentatin is available in all TuchTest releases and fr all supprted Andrid versins. 1. Determine whether t instrument the mbile app using the MATT input type prject r t instrument it using the APP, IPA, r APK ptins. The subsequent steps will differ since MATT is applied at a different time in the wrkflw. Jenkins Wrkflws fr TuchTest The CludTest Jenkins Plugin is used t make the mbile app TuchTestable either by applying it t the prject r t the APP bundle, IPA, r APK file itself (as discussed belw). OS-specific tls are used where applicable. In Andrid, Ant is used t build the APK and the cmmand used depends whether MATT has already been applied. After which, adb is used in an Execute Shell step, fllwed by a final pre-build step that uses the CludTest Jenkins Plugin, Play Cmpsitin(s) cmmand. In ios, xcdebuild and xcrun are used, and the iosappinstaller utility cmmandline cmmands are used t build and deply. Because there are different pssible Jenkins jb wrkflws, the pssible steps are presented a la carte. Fr each Jenkins jb there will be (minimally): A First step; used t retrieve the surce prject (all wrkflws) In between the first and last steps, each Jenkins jb will have:! A CludTest Plugin, MakeAppTuchTestable step;! with ne (r mre) Build step using Ant (in Execute Shell)! and ne (r mre) Install steps using adb (in Execute Shell) A Last step; t Play the Cmpsitin 9
Nte: Nte the fllwing terminlgy: Signing the app is dne using the CludTest Jenkins Plugin, MATT, Advanced Optins t enter the ptinal MATT parameters (refer t the relevant sectins). Signing can als be dne frm the cmmand line using an Execute Shell step and the tl f yur chice. Static instrumentatin applies MATT t the ios prject file r t te Andrid prject flder Dynamic instrumentatin applies MATT t the ios APP bundle file, IPA file, r t the Andrid APK file Dynamic Instrumentatin f an APP file In this wrkflw, yu'll apply MATT t the cmpiled APP file using the appbundle parameter. If yu're als deplying t physical devices, yu'll need t mix and match steps t d bth, keeping in mind t build the APP first. Build the APP with an Execute Shell step using xcdebuild Apply MATT t that cmpiled APP using ios APP Bundle Run App n ios Simulatr cmmand Dynamic Instrumentatin f an IPA file In this wrkflw, yu'll delay applying MATT until the IPA is created Build the APP with an Execute Shell step using xcdebuild Build the IPA file with an Execute Shell step using xcrun, Apply MATT t that cmpiled IPA using ipa Install App n ios Device cmmand. In the remainder f the Jb Creatin steps, mix and match the tasks that yu need t build yur Jenkins jb. Static Instrumentatin f an Xcde Prject In this wrkflw, yu'll apply MATT t the Xcde prject itself, using the prject parameter. After which, yu'll add the steps necessary t build and install t yur simulatrs and devices. Apply MATT using the prject parameter (static nly) Build the APP with an Execute Shell step using xcdebuild (ptinal) Build the IPA with an Execute Shell step using xcrun Either Run App in ios Simulatr r Install ios App n Device (using IPA) Dynamic Instrumentatin f an APK file In this wrkflw, yu'll apply MATT t the cmpiled APK file using the appbundle parameter. If yu're als deplying t physical devices, yu'll need t mix and match steps t d bth, keeping in mind t build the APK first. 10
Get the surce cde (in this scenari, using git) Build the APK with an Execute Shell step using Ant Apply MATT t that cmpiled APK using MATT's APK input type Install the APK Run the cmpsitin(s) Pst results (ptinal) In the fllwing sectins, chse nly thse steps that match yur wrkflw. Static Instrumentatin f an Andrid Prject In this wrkflw, yu'll apply MATT t the Andrid prject itself, using the prject input type. This is dne befre the APK build step. After which, add the step(s) necessary t build and install t yur simulatrs and devices. Get the surce cde (in this scenari, using git) Apply MATT using the prject input type (static nly) Build the APK with an Execute Shell step using Ant Install the APK Run the cmpsitin(s) Pst results (ptinal) 11
Creating a Jenkins Jb The Jenkins jb created belw will run n the desktp machine with Titanium and ne r mre prperly prvisined USB-attached devices. The jb will have build steps using Execute Shell that will utilize git t dwnlad a prject frm surce, run the Make AppTuchTestable utility n that prject, build and deply using the deply script, and use the Plugin's Play Cmpsitin(s) cmmand t silently play a list f cmpsitins. Additinally, a Pst-build actin is used t publish the JUnit test result reprt. 1. In tp-level Jenkins dashbard, click New Jb. The Jb name page appears. 1. Enter a jb name withut spaces and select the first ptin, "Build a free-style sftware prject". 12
The Jb Details page appears. 2. Enter the fllwing cnfiguratin details fr this jb: A descriptin. Fr example, Fr example, "Checks ut the KitchenSink surce cde, makes the prject r cmpiled file TuchTestable, deplys the app, and runs a suite f CludTest cmpsitins." Click the Add build step drp-dwn and select Execute shell. 13
Get KitchenSink using the GitHub Plugin (All Wrkflws) Next, we will add a step that will get the KitchenSink prject that will be used in the remainder f this tutrial. The fllwing instructins use the Jenkins Git Plugin (installed abve). If yu are using yur wn Surce Cde Management system simply select its type and enter its repsitry URL, as yu wuld nrmally d. With the Git Plugin installed in ur Jenkins instance, we will add the Surce Cde Management step as we wuld with any SCM tl. 1. In the Surce Cde Management sectin, check the Git radi buttn. Nte: Yu can specify yur wn SCM tl and Repsitry URL here. 2. Enter the GitHub Repsitry URL in the entry field: https://github.cm/appceleratr/kitchensink 3. Save the Jenkins jb and prceed with the OS-specific instructins belw. 14
Using the CludTest Jenkins Plugin, MakeAppTuchTestable As nted in the prerequisites abve, the CludTest Jenkins Plugin's MakeAppTuchTestable mdule is used t autmate prtins f the Jenkins jb. Nte: The CludTest user specified t run the CludTest Jenkins Plugin, MakeAppTuchTestable mdule must be a user with Mbile Device Administratr rights. CludTest Lite users have admin rights fr the given device n their wn instance. In the fllwing sectins, select nly thse steps necessary t cmplete yur Jenkins jb. The wrkflws are rganized first by mbile OS (ios r Andrid) and then by the instrumentatin type. Abut the Titanium SDK Path In sme cases where the Titanium SDK path is nt autmatically detected, yu will need t add yur Titanium SDK path t the examples included belw. In such cases, use the -titaniumsdk parameter. The Titanium SDK argument can be appended t the CludTest Jenkins Plugin's MATT step using the fllwing syntax: -titaniumsdk <Path f the Titanium SDK t use> where the value f the Titanium SDK is either /Library/Applicatin Supprt/ Titanium/ mbilesdk/sx/ r ~/Library/Applicatin Supprt/Titanium/mbilesdk/sx/ with the flavr r SDK appended at the end f the path. Fr example, /Library/Applicatin Supprt/Titanium/mbilesdk/sx/2.0.1.GA2. 15
Using MATT n an ios APP Bundle (Dynamic Instrumentatin) Use the fllwing steps t dynamically instrument an APP bundle. This is typically dne befre the APP is run n a simulatr but after the prject APP is built. Fr example, yu can use this step after an Execute Shell step using xcdebuild. 1. Add a MakeAppTuchTestable step t the jb. 2. Apply MATT t the cmpiled APP by selecting the Input Type, ios App Bundle (the MATT cmmand-line equivalent is the appbundle parameter). 1. Enter the APP name as the Input File (frm the wrkspace rt). 2. Optinally, click Advanced t display additinal MATT cnfiguratin fields. 3. Save the Jenkins jb. Using MATT n an IPA (Dynamic Instrumentatin fr Device) Use the fllwing steps t dynamically instrument an IPA file. This is typically dne as the last step befre it is installed n a device. Fr example, yu can use this step after an Execute Shell step using xcrun. 3. Add a MakeAppTuchTestable step t the jb. 16
4. Apply MATT t the cmpiled IPA by selecting the Input Type, IPA (e.g. the MATT equivalent is the ipa parameter). 4. Enter the IPA name as the Input File (frm the wrkspace rt). 5. Optinally, click Advanced t display additinal MATT cnfiguratin fields. Launch URL Same as MATT launchurl. Fr example: my-app://launch Back up mdified files Check this t keep backups in the prject). Additinal ptins - Enter any additinal MATT cmmand line parameters. Mst ntably, yu can use MATT t prvisin and cde sign the dynamically instrumented IPA file (cde signing and prvisining can, f curse, be dne using xcrun, which is discussed in the Execute Shell step, Building the IPA fr a Device). Use the fllwing MATT ptinal IPA parameters -prvisiningprfile <prfilepath> - Path f the Prvisining prfile t be used fr building IPA file. The prvisining prfile yu input MUST t be a Distributin prfile. -signingidentity <signingidentityname> - Name f the signing identity t be used fr cdesigning the applicatin. (e.g. "ios Distributin: Develper Name") -entitlementsfile <entitlementsfilepath> - path f the entitlements file t be used fr cdesigning the applicatin Fr mre abut using additinal MATT parameters, use: 17
0sh MakeAppTuchTestable/bin/MakeAppTuchTestable -help 6. Save the Jenkins jb. Using MATT n a Prject (Static Instrumentatin fr Simulatrs/ Devices) This utility is autmatically dwnladed by the SOASTA CludTest Jenkins Plugin and can be easily specified in yur Jenkins jb using the fllwing steps. 7. Click Add a Build Step, and select MakeAppTuchTestable frm the drp-dwn list. This MakeAppTuchTestable step will always be the first step in a Static Instrumentatin wrkflw, but will cme after the APP r IPA is built where thse files are in use n a simulatr(s) r device(s). The Make App TuchTestable frm appears. 8. Enter the Xcde prject name as the Input File. 18
9. Optinally, click Advanced t display additinal MATT cnfiguratin fields. Launch URL Same as MATT launchurl. Fr example: my-app://launch Target Same as MATT target. Fr example: Stckfish cpy Back up mdified files Check this t keep backups in the prject flder (where.xcdeprj resides). Additinal ptins - Enter any additinal MATT cmmand line parameters. Fr mre abut using additinal MATT parameters, use: 0sh MakeAppTuchTestable/bin/MakeAppTuchTestable -help 10.Save the Jenkins jb. Using MATT n an APK (Dynamic) Use the fllwing steps t dynamically instrument an APK file at the prper pint in yur wrkflw. If yu are using dynamic instrumentatin, yu must first d the APK step prir t this step. But, if yu'd rather fllw alng sequentially, yu can add the MATT step nw s lng as yu place an APK build step prir t it befre building the Jenkins jb. If yu are using static instrumentatin, yu must apply MATT t the prject prir t this step. 1. Add a MakeAppTuchTestable step t the jb. 19
2. Select the Input Type, APK (e.g. the MATT equivalent is the APK parameter). 3. Enter the APK name as the Input File (frm the wrkspace rt). 20
4. Optinally, click Advanced t display additinal MATT cnfiguratin fields. 5. Specify MATT flags as required. Fr example, andridsdk when installing a debug APK t a physical device and either verwriteapp r dntcreateapp t prevent Jenkins frm marking the MATT step as FAILURE (even thugh the TuchTestable app is created as expected). Launch URL Same as MATT launchurl. Fr example: my-app://launch Backup mdified files Check this t keep backups in the prject. Additinal ptins - Enter any additinal MATT cmmand line parameters. Mst ntably, yu can use MATT t add keystre, keypass, and strepass arguments t sign the dynamically instrumented APK file. Use the fllwing MATT ptinal APK parameters -keystre <keystrepath> - Path f the keystre t be used t sign the APK file.. -strepass <keystrepasswrd> - Passwrd f the keystre t be used t sign the APK file. -keypass <privatekeypasswrd> - Passwrd f the private key (if different than the keystre passwrd) t be used t sign the APK file. Fr mre abut using additinal MATT parameters, use: 0sh MakeAppTuchTestable/bin/MakeAppTuchTestable - help 21
6. Save the Jenkins jb. Using MATT n a Prject (Static) Use the fllwing steps if yu'll be instrumenting the prject using the static methd via the Input Type, prject. This step will always prceed the build APK step while using static instrumentatin. 1. Click Add a Build Step, and select Make App TuchTestable frm the drp-dwn list. The Make App TuchTestable frm appears. TIP: Click the Help icns fr any rw t get tip text. 2. Select the CludTest Server frm amng thse cnfigured. Nte: Indicating the CludTest server was dne as part f the CludTest Jenkins Plugin, Cnfigure System step. If nt entries appear here return t Manage Jenkins > Cnfigure System, and fill in the infrmatin in the CludTest Servers sectin. Be sure t save these changes. 3. Specify the current prject flder t use. "dridfishchess_andrid" t indicate the prject flder in the Jenkins wrkspace. 22
4. Optinally, specify additinal parameters by first clicking the Advanced buttn (page right). 5. Select the CludTest Server frm amng thse cnfigured. TIP: Indicating the CludTest server was dne as part f the CludTest Jenkins Plugin, Cnfigure System step. If nt entries appear here return t Manage Jenkins > Cnfigure System, and fill in the infrmatin in the CludTest Servers sectin. Be sure t save these changes. 6. Optinally, click Advanced t display additinal MATT cnfiguratin fields. Launch URL Same as MATT launchurl. Fr example: my-app://launch Back up mdified files Check this t keep backups in the prject flder (where build.xml resides). Additinal ptins - Enter any additinal MATT cmmand line parameters. Mst ntably, yu can use MATT t add keystre, keypass, and strepass arguments t sign the dynamically instrumented APK file. Use the fllwing MATT ptinal APK parameters -keystre <keystrepath> - Path f the keystre t be used t sign the APK file.. -strepass <keystrepasswrd> - Passwrd f the keystre t be used t sign the APK file. -keypass <privatekeypasswrd> - Passwrd f the private key (if different than the keystre passwrd) t be used t sign the APK file. Fr mre abut using additinal MATT parameters, use: 23
0sh MakeAppTuchTestable/bin/MakeAppTuchTestable - help 7. Click Apply and then Save. Build the Titanium Prject Nw that we ve revised ur script t make the directry, clne the KitchenSink app surce, and t make the app surce tuchtestable, the next sectin will be used t build the Titanium prject using the Pythn builder.py script that cmes with the Titanium SDK in use. Nte: The builder.py file is available as part f all Titanium SDKs. Hwever, yu must use the ne that crrespnds t the Titanium SDK yur prject uses. As nted in Prerequisites, TuchTest supprts nly Titanium SDK 2.1.3 and later. This sectin f ur Execute Shell script will generate bth an Xcde prject and stre it in the "build/iphne" sub-directry f yur Titanium prject (e.g. "~/dev/jenkins/ wrkspace/jenkinsfucntinaltests/kitchensink/build/iphne/kitchensink.xcdeprj"). Belw is a sample sectin that uses all the parameters abve (as cmmented fields) fllwed by the cmmand itself. # Build the Titanium prject. # # Parameters: # mde # Minimum ios versin ("5.1" in this example) # Path t the Titanium prject directry # Titanium app ID # Titanium app name # ios prvisining prfile identifier (see cmments belw) # ios prvisining prfile wner (see cmments belw) $ "/Library/Applicatin Supprt/Titanium/mbilesdk/sx/2.1.3.GA/iphne/ builder.py" install 5.1 "~/dev/jenkins/wrkspace/jenkinsfucntinaltests/ KitchenSink" cm.appceleratr.kitchensink KitchenSink b92b4cc0-fc88-11e1- a21f-0800200c9a66 "Jhn De" universal 1. Determine all f the fllwing values and then paste the revised cmmand int the Execute Shell script using the sample at the end f this sectin. Yu will need the fllwing infrmatin t prceed: Mde Users have 3 ptins as a mde. Chse the mde based n yur Apple Prfile (use Xcde's Organizer t review yur prfile ptins). install (requires Apple Develper Prfile) adhc (requires Apple Distributin Prfile) 24
distribute (requires Apple Distributin Prfile). Minimum ios versin Fr example, 5.1. Path t the Titanium prject directry n the Jenkins nde (will be the same as the ne used fr MakeAppTuchTestable abve). Titanium app ID - the app ID can be fund in tiapp.xml (in the <id/> element). Fr this example, the ID cm.appceleratr.titanium Titanium app Name the app Name can be fund in tiapp.xml (in the name/> element. Fr this example, KitchenSink ios prvisining prfile identifier - The prvisining prfile identifier is typically lcated in the "~/Library/MbileDevice/Prvisining Prfiles" directry. Yu'll find ne r mre files with the extensin ".mbileprvisin". The prfile ID is the name f the file, minus the extensin. ios prvisining prfile wner Team Name- the prfile "wner" is the name f the ios develpment team t which the prfile is registered. 25
Preparing the App fr Simulatrs and Devices The ios App Installer Utility cntains tw executable files is_sim_launcher and is_app_installer. Deplyment is achieved fr bth Simulatrs and ipad/iphne devices using these executables and the steps described belw. Unzip the utility at this time if yu have yet t d s and nte the cntents f the resulting iosappinstaller flder. Fr a Simulatr, the./bin/is_sim_launcher -app cmmand is used In rder t deply t a simulatr, we must first build an APP file and then use that APP file with the is_sim_launcher Fr an iphne r ipad, the./bin/is_app_installer -ipa In rder t deply t a physical device, we must first build an APP file, fllwed by building an IPA file, after which we can use the is_app_installer. Build the APP fr a Simulatr The app file is a requirement fr deplyment t all ios simulatrs and devices. In rder t build the app, the cmmand line xcdebuild cmmand is used. 1. Enter the fllwing lines in the end f the Execute Shell field (revise the paths t match that f yur wn envirnment): #Build the KitchenSink app fr Simulatr /usr/bin/xcdebuild -sdk iphnesimulatr -target "KitchenSink-universal" -prject ~/dev/jenkins/wrkspace/jenkinsfunctinaltests/kitchensink/ build/iphne/kitchensink.xcdeprj -cnfiguratin Release clean build where (as in the MakeAppTuchTestable cmmand abve): <sdk> in this case is iphnesimulatr <target> is the name f the target in the ".xcdeprj" file representing yur prject. Nte that yu need t either specify the sdk as an argument (recmmended) r create a Target specifically fr Simulatrs <prject> is the path f the ".xcdeprj" file t build <cnfiguratin> is the type f build. Refer t /usr/bin/xcdebuild help fr mre infrmatin. 26
After the APP has been successfully created, it can be deplyed t a simulatr using the steps in the next sectin. If yu are deplying t an iphne r ipad, yu must first create an IPA archive befre deplyment. Deplying the APP File n a Simulatr Once the APP file has been created, use the fllwing step t run the is_sim_launcher t deply the cmpiled APP file. Befre starting, nte the path t the unarchived ios App Installer Utility flder where is_sim_launcher resides. 1. In the Execute Shell field, enter the fllwing (be sure t use yur wn paths as well as t specify the ios SDK and family f the simulatr t use): #Deply the APP t a simulatr cd ~/Dcuments/Dem/iOSAppInstaller/./bin/is_sim_launcher --app ~/dev/jenkins/wrkspace/ JenkinsFunctinalTests/KitchenSink/build/iphne/build/Releaseiphnesimulatr/KitchenSink.app --sdk 5.1 --family ipad --agenturl "http://<cludtest URL>/cncert/tuchtest" The is_sim_launcher requires the app path: --app <app path> - The path t the cmpiled APP. Nte: The highlighted CludTest URL placehlder in the abve sht, which riginates in the appendix script at the end f this tutrial, must be replaced with yur wn CludTest URL. The is_sim_launcher takes the fllwing additinal parameters. These are used in the example abve t specify the SDK and family: --sdk <versin> - The ios SDK versin t use. Fr example, 5.1. --family <list> - The simulated device family, values are iphne (default), ipad, iphne_retina, and ipad_retina. --agenturl The agenturl parameter specifies the TuchTest Agent URL t use t launch the device agent, which is a requirement fr cntinuus integratin. The value is the CludTest URL with the /cncert/tuchtest URL string appended. 27
Installing the IPA Archive n iphnes and ipads In this sectin, we ll run the is_app_installer t deply the IPA archive created in the prir sectin. Befre starting, nte the path t the dwnladed ios App Installer Utility. In the example belw, the IPA archive is in the Jenkins wrkspace we already defined as part f this Jenkins jb. 1. In the Execute Shell field, enter the fllwing (be sure t revise this example t use yur wn paths): #Deply the IPA t all devices cd ~/Dcuments/Dem/iOSAppInstaller/./bin/is_app_installer --ipa ~/dev/jenkins/wrkspace/ JenkinsFunctinalTests/KitchenSink/build/iphne/build/Debug-iphnes/ KitchenSink.ipa The script requires the fllwing parameters: --ipa <ipapath> - The path t the IPA archive. Nte: Yur user-cnfigurable IPA archive lcatin will vary. In the abve example, the file is in the Jenkins wrkspace that we specified as ~/ dev/jenkins/wrkspace/jenkinsfunctinaltests/kitchensink/build/ iphne/build/debug-iphnes/. The ios App Installer Utility will deply t all the tethered prvisined devices by default. If yu d like t limit the deplyment t specific devices use the fllwing ptinal parameters: --udid <list> - One r mre device UDID in a cmma-separated list, if unspecified it install n all the cnnected ios devices --device <list> : device name list cmma-separated, if unspecified it install n all the cnnected ios devices 28
Using SCmmand t Play One r Mre Cmpsitins Next, we will add SCmmand lines that will silently play the specified test cmpsitins n the specific CludTest instance. Additinally, we will add arguments that will utput junitxml-cmpatible XML cde that will appear in Jenkins fr each test result. 1. In the Execute Shell field, enter the fllwing (be sure t substitute the full path fr the cmpsitin yu d like t use): # Run the first cmpsitin. # The result will be stred in the "testresults/kitchensink1.xml" file. # Jenkins will use this file t render the test reprt. ~/Dcuments/Dem/scmmand/bin/scmmand \ cmd=play \ name="/soastatutrial/titanium/cmpsitin fr KitchenSink1" \ wait \ frmat=junitxml \ url=http://ctmbile.sasta.cm/cncert \ username=soasta_doc passwrd=secret >testresults/kitchensink1.xml 2. Enter any additinal cmpsitins fr the jb: # Run the secnd cmpsitin. # The result will be stred in the "testresults/kitchensink2.xml" file. # Jenkins will use this file t render the test reprt. ~/Dcuments/Dem/scmmand/bin/scmmand \ cmd=play \ name="/soastatutrial/titanium/cmpsitin fr KitchenSink2" \ 29
wait \ frmat=junitxml \ url=http://ctmbile.sasta.cm/cncert \ username=soasta_doc \ passwrd=secret >testresults/kitchensink2.xml 30
Adding the Publish Junit Test Result Reprts Step Next, we will add a Pst-build actin that will display SCmmand utput in Jenkins and pt in t plugin display as well. 1. In the Pst-build actin sectin, check Publish JUnit test result reprt. 2. In the Test reprt XMLs field, enter a path where the JUnit XML will be created. This shuld be in the Jenkins wrkspace. The flder need nt exist prir t the first build. Fr example, the flder testresults/**/*.xml. Nte: The errr shwn belw will display the n match errr until after the first build. 3. Under the Additinal test reprt features sectin, check the 'Include links t SOASTA CludTest dashbards' bx. 4. Check the Include links t SOASTA CludTest dashbards bx t pt int the plugin fr this jb. 5. Click Save t exit the jb. Building the Prject After yu click the Save buttn, yu will be taken t the prject page fr the jb yu just created. 1. T build the prject, click the "Build Nw" link. 31
The build will start. After a shrt delay, yu shuld see a prgress bar appear n the left side f the page. Click this prgress bar t watch the build prcess "live" in the Cnsle view. Yu shuld see the fllwing happen: a. Jenkins checks ut the surce cde frm Git, and runs the MakeAppTuchTestable utility. b. Jenkins runs the DeplyPrjectTDevice script. On the build nde, yu shuld see Xcde immediately launch and deply the KitchenSink app t the tethered device. D nt interact with Xcde while this is happening. Once the app has been deplyed, Xcde will autmatically exit. c. On the tethered device, yu shuld see the KitchenSink app briefly appear. It will immediately switch ver t the SOASTA TuchTest Agent web page (in Safari), with the Status shwing "Cnnected" <insert screen sht>. d. Jenkins plays the CludTest cmpsitins using SCmmand. On the tethered device, yu shuld see the KitchenSink app launch and run thrugh the test steps. When the test finishes, KitchenSink will exit, and the SOASTA TuchTest Agent page will re-pen. 32
Inspecting Test Results in Jenkins On the build summary page, the Test Result link (added by the steps in the previus sectin) appears. 1. Click Test Result. Fr a successful test with n failures, the Test Result page merely lists the All Tests sectin with the given package (i.e. in this case the package equates t a CludTest repsitry flder). 2. Click the Package link until yu reach the cmpsitin flders. Each click pens the subsequent CludTest flder. The cmpsitin was a success in this case. 33
3. Use the Click here dashbard link t view CludTest inline. Lgin using yur credentials if yu get the Lgin page. After ding s, the dashbard fr the given cmpsitin is shwn. 34
Things get mre interesting when an errr in the test ccurs. The subsequent SCmmand utput is displayed (in text) n the Jenkins Test Result page (as discussed abve). In this case, the All Failed Tests sectin is added with the name f the test listed with a link t mre f the SCmmand details. Clicking the link under the Test Name sectin where the cmpsitin is named displays an Errr detail page (fr the given errr). In the errr abve a validatin in the Cmpsitin fr KitchenSink1 has failed. 4. T view this errr in the CludTest, Result Details dashbard, click the plugin link prvided (i.e. Click here t see the SOASTA CludTest dashbard fr this test). 35
After the plugin link is clicked, enter CludTest credentials whenever required. After credentials are entered, the dashbard tab pens, displays the test result, and jumps t the relevant errr. 36
Frm here, the Result Details dashbard can be navigated as within any CludTest dashbard. Refer t Result Details Dashbard fr a quick review f Result Details features. 37
Appendix: Inspecting the Mbile App in CludTest In the steps abve at the end f each run f the MakeAppTuchTestable.jar we were ntified that the Mbile App Object had been created in the CludTest Repsitry. TIP: This mbile app will appear in the Chse Device Agent and Mbile App bx whenever end-users start a mbile app recrding. Selecting which mbile app t launch n which test devices is a crucial enduser step. 1. Optinally, verify that the Mbile App has been added by lgging int CludTest and lking fr its entry in the Central > Mbile Apps list. Fr example, in the screensht belw the KitchenSink cpy app appears as expected. 2. Duble-click the KitchenSink Mbile App t inspect its details. The Mbile App detail frm appears. All f the fields shwn were ppulated frm the Xcde prject, with the exceptin f Supprted Device Type and Minimum OS Versin. a
The default Supprted Device Type is Universal (e.g. bth iphne and ipad). Leave this as is, since we will be specifying multiple devices later. The default Minimum OS Versin supprted in TuchTest beta is ios 5.0 b
Appendix: Cmpleted Execute Shell Script The cmpleted Execute Shell script described abve is presented in its entirety belw. Yu can use this as a template t insert yur wn paths as well as yur wn CludTest URL wherever it is required. #Cntinue playing subsequent cmpsitins if a prir cmpsitin fails set +e #Create the test result reprts flder if it desn't already exist mkdir -p testresults #Remve the versin we dwnladed as part f last build (if any) rm -rf "KitchenSink" #Dwnlad KitchenSink frm Github /usr/lcal/git/bin/git clne https://github.cm/appceleratr/kitchensink #Run MakeAppTuchTestable (be sure t enter yur CludTest URL here) sh MakeAppTuchTestable/bin/MakeAppTuchTestable -prject $WORKSPACE/ KitchenSink -target "KitchenSink" -url http://<cludtest URL>/cncert - username SOASTA_DOC -passwrd secret # Build the Titanium prject. # # Parameters: # mde (shuld be "adhc" fr CI) # Minimum ios versin ("5.1" in this example) # Path t the Titanium prject directry # Titanium app ID # Titanium app name # ios prvisining prfile identifier # ios prvisining prfile team name "/Library/Applicatin Supprt/Titanium/mbilesdk/sx/2.1.3.GA/iphne/ builder.py" install 5.1 ~/dev/jenkins/wrkspace/jenkinsfunctinaltests/ KitchenSink cm.appceleratr.kitchensink KitchenSink 20433FAB-7A9C-4996-BDD3- E660DB279261 "James GARDNER" universal I
#Build the KitchenSink app fr Simulatr /usr/bin/xcdebuild -sdk iphnesimulatr -target "KitchenSink-universal" - prject ~/dev/jenkins/wrkspace/jenkinsfunctinaltests/kitchensink/build/ iphne/kitchensink.xcdeprj -cnfiguratin Release clean build #Deply the APP t a simulatr (be sure t enter yur CludTest URL here) cd ~/Dcuments/Dem/iOSAppInstaller/./bin/is_sim_launcher --app ~/dev/jenkins/wrkspace/jenkinsfunctinaltests/ KitchenSink/build/iphne/build/Release-iphnesimulatr/KitchenSink.app --sdk 5.1 --family ipad --agenturl "http://<cludtest URL>/cncert/tuchtest" #Deply the IPA t all devices cd ~/Dcuments/Dem/iOSAppInstaller/./bin/is_app_installer --ipa ~/dev/jenkins/wrkspace/jenkinsfunctinaltests/ KitchenSink/build/iphne/build/Debug-iphnes/KitchenSink.ipa # Run the first cmpsitin. # The result will be stred in the "testresults/kitchensink1.xml" file. # Jenkins will use this file t render the test reprt. ~/Dcuments/Dem/scmmand/bin/scmmand \ cmd=play \ name="/soastatutrial/titanium/cmpsitin fr KitchenSink1" \ wait \ frmat=junitxml \ url=http://ctmbile.sasta.cm/cncert \ username=soasta_doc \ passwrd=secret >testresults/kitchensink1.xml # Run the secnd cmpsitin. # The result will be stred in the "testresults/kitchensink2.xml" file. # Jenkins will use this file t render the test reprt. ~/Dcuments/Dem/scmmand/bin/scmmand \ cmd=play \ name="/soastatutrial/titanium/cmpsitin fr KitchenSink2" \ wait \ frmat=junitxml \ url=http://ctmbile.sasta.cm/cncert \ II
username=soasta_doc \ passwrd=secret >testresults/kitchensink2.xml III
SOASTA, Inc. 444 Castr St. Muntain View, CA 94041 866.344.8766 http://www.sasta.cm