TE TER. Continuous testing and delivery. number 21. SUBSCRIBE It s FREE for testers. Including articles by: Roy de Kleijn Polteq.

Transcription

1 TE TER SUBSCRIBE It s FREE for testers Essential for software Continuous testing and delivery Including articles by: Bogdan Bereza VictO Roy de Kleijn Polteq Mark Lehky Jessica Schiffmann Prism Informatics Eric M.S.P. Veith Wilhelm Büchner Hochschule/TU Bergakademie Freiberg testers June / 5 v2.0 number 21 THIS ISSUE OF PROFESSIONAL TESTER IS SPONSORED BY

2 Flash light by Mark Lehky Genie and monster: a groovy soap opera development (or test) needs arise. If the model is adhered to strictly, test automation becomes simple: any of the tiers can be replaced with virtually any tool. If, due to design and/or development decisions, the boundaries between the tiers start to blur, testing (and future development) become more difficult. This is one of the reasons testing should get involved in the design process as early as possible. The data tier is usually, and should be, unimportant from the test point of view. It should have very little impact on the tiers above, and the choice of database engine is usually dictated by other factors such as performance, security and cost. Mark Lehky illuminates Flash for testers Its impending decline has been rumoured for years, but Flash remains one of the most popular ways to create rich Internet applications. By testers, it is often perceived as difficult to automate. In this article I will try to dispel misconceptions and explain how effective test automation of Flash applications can be achieved easily: and when and why it cannot. Technology overview Most web Flash applications can be described very well by the three-tier architecture model (see figure 1). Ideally, none of the tiers knows anything about the platform, technology, or structure of any of the others, and therefore any one can easily be swapped out as new The logic tier should be responsible for transmitting, recoding, and processing all messages. Transmission of messages is usually done over classic HTTP or Real Time Messaging Protocol (RTMP). The content of the messages is usually encoded using Action Message Format (AMF). If the application is written in Java, then the logic tier server is typically BlazeDS; if the application is written in C#.NET, the server will likely be FluorineFX. Some logic tiers are very complex and use multiple technologies (web server, message queue, etc). Here I will deal with the most common AMF messages only. The presentation tier of a Flash application is most often written in ActionScript. This has the look and feel of Java: object-oriented design, compile-once-run-anywhere to produce an SWF file playable by a large number of available player applications on all desktop and mobile platforms. This technology is completely different to HTML, but the player can be made compatible with a web browser via a plug-in: I am often (ie several times a week) asked: can I automate Flash with Selenium?. The answer is no. Sort of. PT - June professionaltester.com 21

3 Presentation tier The top-most level of the application is the user interface. The main function of the interface is to translate tasks and results to something the user can understand. >GET SALES TOTAL >GET SALES TOTAL 4 TOTAL SALES Logic tier This layer coordinates the application, processes commands, makes logical decisions and evaluations, and performs calculations. It also moves and processes data between the two surrounding layers. Data tier Here information is stored and retrieved from a database or file system. The information is then passed back to the logic tier for processing, and then eventually back to the user. GET LIST OF ALL SALES MADE LAST YEAR QUERY Database Figure 1: three-tier architecture (public domain image from File:Overview_of_a_three-tier_application_vectorVersion.svg) ADD ALL SALES TOGETHER SALE 1 SALE 2 SALE 3 SALE 4 Storage also need to run a Genie Socket Server (reminiscent of the old Selenium Server). After that, things get pretty easy. Genie browser Genie automation scripts refer to objects by their GenieID. This is similar to the HTML element id or the element locators used by Selenium, but more complex: it is formed from several attributes of the Flash application determined by its developers together with some cryptic portions assigned by Genie itself. The result can look quite cryptic, for example SP^botBa rcontainer:::fp^userbox:::se^imgbox:: PX^2::PTR^0::IX^2::ITR^0. GenieIDs are difficult to predict, even in close collaboration with developers. UI automation with Genie Selenium is not a tool, but a library available in many programming languages that allows the coder to access and manipulate objects (text, controls, images etc) in a web browser s Document Object Model (DOM). Many technologies, eg HTML5, JavaScript, AJAX and others, generate or manipulate the DOM. Flash does not. The fact that a web browser requires a plug-in to be able to display Flash applications is the first giveaway that Selenium cannot automate Flash. A Flash application is separate to the DOM, even though it can be displayed in the browser window alongside DOM objects. Selenium was never intended to automate Flash, and in all likelihood never will. Fortunately, there are other libraries to enable you to manipulate objects in a Flash application. Here I will discuss one of them, Genie ( adobe/genie/). Another example, also often recommended, is flash-selenium ( One reason, and probably the most important, why you would want to choose Genie over flash-selenium is that flashselenium is dependent on Selenium Remote Control, which was deprecated (in favour of WebDriver) in July Genie depends on nothing and can be used completely standalone, or combined with anything else you may want to use. It can be plugged into almost any existing framework or toolset (Ant, Maven, JUnit, TestNG, etc etc). Unlike Selenium, Genie is only available in Java and some of its development support tools will only work in Eclipse. This is a barrier for testers using only.net and Visual Studio. Genie is open source so a port is possible but I don t foresee it. If you need to test Flash and want to use Genie but don t know Eclipse I strongly recommend the tutorials at org/resources/. Before installing Genie you will need to install Eclipse, the Java Development Kit (JDK) at least version 1.5, and the debug version of Flash Player. The Genie download comes with full instructions. You will Genie includes a browser (figure 2) that helps you locate elements and discover their GenieIDs and other attributes. Genie recorder User actions within the Flash application are recorded as a generic Java script. Genie has no playback functionality; the script is intended to be pasted into Eclipse and run from there, somewhat similar to recording a script with Selenium IDE and then exporting it to Java. An example Genie script is shown in figure 3. It is somewhat similar to Selenium RC code, and the recorder will help to ease you into Genie programming. There is no contextual menu to help create assertions; they have to be inserted manually, in Eclipse. Once you become proficient at writing code by hand you will almost never use Genie recorder again. As can be seen from the script, Genie has its own test framework called GenieScript. This is all you need to create and execute extensive test suites, 22 PT - June professionaltester.com

4 although it does not have all the features of more mature frameworks such as JUnit or TestNG which are often used instead. Genie also features built-in custom logging which produces an XML file, with an XSLT provided to convert it to viewable HTML. The file can of course be parsed with any tool, but it does not follow any established format such as JUnit s. Furthermore, the log is limited to only Genie commands: if your test fails in any other step (for example an assert), this will not be logged automatically. Genie does have extra commands to log your own events, but this is additional overhead. If you decide to use some other framework such as JUnit or TestNG you will probably use the logging capabilities it provides so the logs can be parsed automatically by your continuous integration server, and once you have finished debugging your test will want to turn off Genie s logging. This is done with: 1. LogConfig logger = new LogConfig(); 2. logger. setnologging(true); Figure 2: Genie browser All your Selenium skills (and frustrations) still apply! Because Genie, like Selenium, is a Java library, having the framework of your choice run your tests is trivial. Figure 4 shows how to do this with JUnit; figure 5 combines it with Selenium. I have also found that all my Genie tests run with no problems extended from GroovyTestCase. This is significant because Groovy, a scripting language for the Java platform, is natively understood by SoapUI. I ll explain this significance further in the section on API automation below. The Page Object Model Think of your application as a collection of screens, each with a number of controls and actions that it can perform. If you can describe one screen s actions as a collection of methods, abstracting away all the controls, and make it independent of any other screen, then you have a PageObject. Alan Richardson described the advantages of this in Professional Tester August 2012 ( com/magazine/backissue/pt016/ ProfessionalTester-August2012- Richardson.pdf). Genie, with its obscure GenieIDs, is perfect for this approach! Further, because Flash is often embedded in a webpage via a browser plug-in, there is great incentive to combine the automation with Selenium. Using the PageObject pattern, you can effectively hide which actions require accessing Selenium and which are accessing Genie. Page loading delays As with any web technology, there are issues with page loading delays. Since the Flash application runs in a plug-in you will not be able to query the browser to see if the page is loaded it is always loaded. PT - June professionaltester.com 23

5 1. package scripts; import com.adobe.genie.geniecom.swfapp; 4. import com.adobe.genie.executor.geniescript; 5. import com.adobe.genie.executor.components.*; 6. import com.adobe.genie.executor.uievents.*; 7. import static com.adobe.genie.executor.genieassertion.*; 8. import com.adobe.genie.executor.enums.genielogenums; /** 12. * This is a sample Genie script. 13. */ 14. //Change name of the class 15. public class Unnamed extends GenieScript { public Unnamed() throws Exception { 18. super(); } public void start() throws Exception { 24. //Turn this on if you want script to exit 25. //when a step fails 26. EXIT_ON_FAILURE = false; //Turn this on if you want a screenshot 29. //to be captured on a step failure 30. CAPTURE_SCREENSHOT_ON_FAILURE = false; SWFApp app1=connecttoapp( [object pamain] ); 33. (new GenieDisplayObject( SP^stageContainer:::FP^loginContainer:::SE^userN ametxt::px^6::ptr^0::ix^1::itr^0,app1)).click(56,17,436,174,1090,821,3,f alse); 34. (new GenieTextInput( SP^stageContainer:::FP^loginContainer:::SE^userNameT xt::px^6::ptr^0::ix^1::itr^0,app1)).selecttext(0,0); 35. (new GenieTextInput( SP^stageContainer:::FP^loginContainer:::SE^userNameT xt::px^6::ptr^0::ix^1::itr^0,app1)).input( ); 36. (new GenieDisplayObject( SP^stageContainer:::FP^loginContainer:::SE^passw ordtxt::px^6::ptr^0::ix^2::itr^0,app1)).click(63,12,443,246,1090,821,3,f alse); 37. (new GenieTextInput( SP^stageContainer:::FP^loginContainer:::SE^passwordT xt::px^6::ptr^0::ix^2::itr^0,app1)).selecttext(0,0); 38. (new GenieTextInput( SP^stageContainer:::FP^loginContainer:::SE^passwordT xt::px^6::ptr^0::ix^2::itr^0,app1)).input( 1234 ); 39. (new GenieDisplayObject( SP^stageContainer:::FP^loginContainer:::SE^login Btn::PX^6::PTR^0::IX^4::ITR^0,app1)).click(42,44,1011,218,1090,821,3,fal se); 40. } 41. } Figure 3: Genie script 24 PT - June professionaltester.com

6 1. import junit.framework.testcase; import com.adobe.genie.executor.genie; 4. import com.adobe.genie.executor.logconfig; 5. import com.adobe.genie.executor.components.geniedisplayobject; 6. import com.adobe.genie.executor.components.genietextinput; 7. import com.adobe.genie.geniecom.swfapp; public class MyTestSuite extends TestCase { Genie jean; 12. SWFApp app1; protected void setup() throws Exception { 15. super.setup(); LogConfig logger = new LogConfig(); 18. logger.setlogfolder( log ); 19. jean = Genie.init(logger); 20. app1 = jean.connecttoapp( [object pamain] ); 21. jean.exit_on_failure = true; 22. jean.exit_on_timeout = true; 23. jean.capture_screenshot_on_failure = false; 24. } protected void teardown() throws Exception { jean.stop(); super.teardown(); 31. } void testcase_login() { (new GenieDisplayObject( SP^stageContainer:::FP^loginContainer:::SE^userN ametxt::px^6::ptr^0::ix^1::itr^0, app1)).click(); 36. (new GenieTextInput( SP^stageContainer:::FP^loginContainer:::SE^userNameT xt::px^6::ptr^0::ix^1::itr^0, app1)).selecttext(0, 100); 37. (new GenieTextInput( SP^stageContainer:::FP^loginContainer:::SE^userNameT xt::px^6::ptr^0::ix^1::itr^0, app1)).input( ); 38. (new GenieDisplayObject( SP^stageContainer:::FP^loginContainer:::SE^passw ordtxt::px^6::ptr^0::ix^2::itr^0, app1)).click(); 39. (new GenieTextInput( SP^stageContainer:::FP^loginContainer:::SE^passwordT xt::px^6::ptr^0::ix^2::itr^0, app1)).selecttext(0, 100); 40. (new GenieTextInput( SP^stageContainer:::FP^loginContainer:::SE^passwordT xt::px^6::ptr^0::ix^2::itr^0, app1)).input( 1234 ); 41. (new GenieDisplayObject( SP^stageContainer:::FP^loginContainer:::SE^login Btn::PX^6::PTR^0::IX^4::ITR^0, app1)).click(); 42. } void testcase2() { } 47. } Figure 4: Genie in JUnit PT - June professionaltester.com 25

7 1. import junit.framework.testcase; import org.openqa.selenium.webdriver; 4. import org.openqa.selenium.firefox.firefoxdriver; import com.adobe.genie.executor.genie; 7. import com.adobe.genie.executor.logconfig; 8. import com.adobe.genie.executor.components.geniedisplayobject; 9. import com.adobe.genie.executor.components.genietextinput; 10. import com.adobe.genie.geniecom.swfapp; public class MyTestSuite extends TestCase { Genie jean; 15. SWFApp app1; 16. WebDriver driver; protected void setup() throws Exception { 19. super.setup(); driver = new FirefoxDriver(); 22. driver.get( ); LogConfig logger = new LogConfig(); 25. logger.setlogfolder( log ); 26. jean = Genie.init(logger); 27. app1 = jean.connecttoapp( [object pamain] ); 28. jean.exit_on_failure = true; 29. jean.exit_on_timeout = true; 30. jean.capture_screenshot_on_failure = false; 31. } protected void teardown() throws Exception { jean.stop(); 36. driver.quit(); super.teardown(); 39. } void testcase2() { 42. // tests now have access to Selenium such as 43. driver.findelement(by...) 44. } 45. } Figure 5: Genie with Selenium 26 PT - June professionaltester.com

8 Just as in Selenium, you will need to use tricks to discover if elements are present, detect busy streamers, and so on. Genie has methods for all elements, such as.isenabled(),.ispresent(), and.isvisible(). So, for example, waiting for something like a busy streamer to go away is simple (figure 6). XPath and PageFactory Neither XPath not PageFactory (a design pattern used in a framework to make discovery of elements simpler and more dynamic) is implemented in Genie. However a useful feature of the Genie browser gives Genie the ability to handle both: it can dump an XML representation of the entire application. The code to do this is shown in figure 7 and you can then search for GenieIDs using XPath as shown in figure 8. The Genie IDs are guaranteed to be consistent from one run of your application to the next. However, it is possible for the developer to create a Flash application in which controls look the same but are instantiated on the fly, making the GenieIDs effectively dynamic. This can be a problem but PageFactory solves it by finding the GenieIDs on the fly. To build your own PageFactory you will need to add the Java Reflection library to your framework. API automation with SoapUI In many Flash projects the underlying API will be available to testing long before the UI materializes (assuming it ever does). It makes sense to test the API first. The popular API testing framework SoapUI has limited support for AMF messages and no support for any flavour of the RTMP protocol. To test the APIs you 1. static void waitforstreamer(geniemovieclip streamer, int seconds) { 2. Date rightnow = new Date().getTime(); 3. while(streamer.isvisible()) { 4. if(rightnow + (seconds * 1000) < new Date(). gettime()) { 5. throw new TimeoutException( Page busy? Streamer still visible after + seconds + seconds. ); 6. } 7. } 8. } Figure 6: waiting for a busy streamer to go away 1. String appxml = SynchronizedSocket.getInstance(). getappxmlgeneric(app.name); 2. ByteArrayInputStream inputstream = new ByteArrayInputStream(appXml.getBytes()); 3. DocumentBuilder docbuilder = DocumentBuilderFactory.newInstance(). newdocumentbuilder(); 4. Element docelement = docbuilder.parse(inputstream). documentelement; 5. XPath xpath = XPathFactory.newInstance(). newxpath(); Figure 7: dumping an XML representation of the application 1. Node page = xpath.evaluate( //*[@name= + fieldname + ], pagenode, XPathConstants.NODE ); 2. return pagenodes.attributes. getnameditem( genieid ).getnodevalue(); Figure 8: searching for a GenieID using XPath will need to have HTTP enabled on the server: RTMP and HTTP can be enabled simultaneously. Although you may not be doing this in production, bear in mind that (i) you are not testing the protocol nor, probably, the server configuration; (ii) transmission via the RTMP protocol will be validated as part of your end-to-end test, which you will probably do through the UI. Right now we are concerned only with functional testing of the APIs. There is no call discovery for AMF messages in SoapUI. This is the equivalent of reading in a WSDL for SOAP endpoints, or a WADL for REST endpoints. For AMF everything has to be done by hand. However SoapUI does provide other goodies: test scripting, Groovy extensions and XML parsing of responses. We will now see how to hook up AMF with SoapUI tests. There is a useful, short tutorial on these at calling-amf-services.html. Method discovery As there is no way to define an AMF endpoint in SoapUI, one just creates an empty project (unless the project also contains SOAP and/or REST endpoints) and a new test case. When you add an AMF step to your test case you will need to provide three pieces of information: the endpoint, the AMF call, and all the inputs (figure 9). If you are lucky you will be able to get these from a detailed technical specification provided by your developers. If you are like me and you want any documentation you will have to write it yourself. PT - June professionaltester.com 27

9 Figure 9: adding an AMF step Figure 10: method calls in BlazeMonster Figure 11: two arguments of the same type: which is which? Figure 12: naming inputs AMF endpoint This is made up of several parts as follows: {protocol}://{server}/{service}/messagebroker/amf. You will have to find out the parts in curly brackets from development. Remember that for SoapUI the protocol has to be http or https: rtmp will not work. For secure messaging the last part is sometimes messagebroker/amf. Note however that this does not make the messaging secure: configured secure transmission (the s part of https) does. Again, remember that for now we are only concerned with functional testing of the APIs and everything else just gets in the way of that. AMF call Now that we have an endpoint, we need to know the method calls. The tool BlazeMonster ( wordpress.com/blazemonster/) can be used for, among other things, discovering them. For that we need only its Existing Services function. To use it, enter the web application s root URL, that is the variable parts of the endpoint, and click Load destinations. If everything is configured correctly, BlazeMonster fills in the AMF endpoint URL automatically and all the method calls are loaded up (figure 10). Note that BlazeMonster is not perfect. It shows all the method calls on the server, whether they are public (available to be called) or private (not exposed as callable methods). Later on, when you are actually making the call using SoapUI, if you get a response that such a method does not exist even though BlazeMonster says it does, check with your development whether that particular method is public or private. You can now determine the AMF call. It is {destination name}{method} as those are shown in BlazeMonster: I usually have both BlazeMonster and SoapUI open on facing screens, and just copy-paste everything between the two windows. Now you can click on the run step button in SoapUI, and should get an error response saying that you did not supply any inputs (unless that call does not require any). Simple inputs These are simply added to the input window of SoapUI. In the example shown, the call is named findcustomerbyssn and it takes only one input: probably the SSN (social security number). I say probably, because BlazeMonster tells us only that there is one argument and that it is of type java.lang.string. It is able to read only the method signature so knows the number and type of inputs, but not what they represent. We can test our guess from within BlazeMonster using the Invoke selected button. Unfortunately, things are seldom that simple! If the AMF call takes more than one argument, guessing what they are becomes harder. Even if you can do that, guessing which is which becomes very hard, especially if more than one of them are of the same type (figure 11). The order in which the parameters are displayed now becomes important. You will need to find out where, in the source code repository, is the interface to the service, then read the code and find the method by 28 PT - June professionaltester.com

10 name, eg findcustomerbyname(string surname, String givenname). Then when creating the AMF step in SoapUI, give the inputs the same or similar names in the same order (figure 12). The name is not actually important, only the order is. Complex inputs Things now (as always in testing) get worse. We are accessing via AMF the underlying methods, right in the technology layer. That means the arguments of some of the calls are probably not simple numerical or string variables. They may be java. util.collection, or a custom object. Custom objects are easy to identify (their type starts with com.<your_company_name>) but difficult to deal with. Selecting a method with these argument types in BlazeMonster and clicking the Invoke selected button gives the error message Selected operation expects object types which are not supported by this application. Generate code for invoking this operation. And so we return to Groovy, the scripting language that can be used to generate the inputs for the call within SoapUI. Luckily for non-groovers, Groovy is source-compatible with Java. You can write simple Java code to create the needed object(s) and it should work in your SoapUI AMF step with very little modification. First, create a new AMF step in the way already described: enter the endpoint and AMF call and name the input. There could be more than one input, but in that case you should have a discussion with development about whether they would want to consider redesigning this call. Next, in BlazeMonster, select the appropriate method and click on Generate code. A new window appears (figure 13) containing AS3 (ActionScript) VO (Value Objects) code for Java classes. The lower pane Figure 13: AS3 VO code and custom objects 1. package com.company.dto 2. { 3. [Bindable] 4. [RemoteClass(alias= com.company.dto.customerdto )] 5. public class CustomerDTO 6. { 7. public var dateofbirth:date; 8. public var givenname:string; 9. public var id:number; 10. public var surname:string; public function CustomerDTO() 13. { 14. super() 15. } 16. } 17. } Figure 14: AS3 VO code for a specific custom object 1. import com.company.dto.customerdto 2. def scriptcustomer = new CustomerDTO() 3. scriptcustomer.givenname = Peter 4. scriptcustomer.surname = Parker 5. scriptcustomer.dateofbirth = new GregorianCalendar (1962, Calendar.AUGUST, 15).getTime() 6. parameters[ customer ] = scriptcustomer Figure 15: Groovy script PT - June professionaltester.com 29

11 lists the input and output custom classes. Select the class you are trying to create and click on Generate VO. This opens a new tab with more AS3 code, which gives a lot of hints about what is needed. Copypaste it into the script pane of the AMF call in SoapUI. An example of what it looks like is shown in figure 14. Now we will convert this to Groovy code SoapUI can understand. I usually start by enclosing all the code in a comment block. Now change lines 4 and 5 to lines 1 and 2 of the Groovy script as shown in figure 15. Figure 16: call inputs confirmed Next, we need the jar file that defines the object: place this in SoapUI s bin/ext folder and restart SoapUI to load it into the session. Now click the run button. If you get a dialog saying something like paramets {} amfheaders {} you re OK! If you get errors, don t move on until you resolve them. Then, continue building the script inputs (lines 3-5 in figure 15, replacing lines 7-9). 1. <flex.messaging.io.amf.asobject> 2. <dateofbirth> :00:00.0 PDT <dateofbirth> 3. <givenname>peter</givenname> 4. <surname>parker</surname> 5. </flex.messaging.io.amf.asobject> Figure 17: the response as XML (simple objects) 1. <flex.messaging.io.amf.asobject serialization= custom > 2. <string>dateofbirth</string> 3. <date> :00:00.0 PDT</date> 4. <string>givenname</string> 5. <string>peter</string> 6. <string>surname</string> 7. <string>parker</string> 8. </flex.messaging.io.amf.asobject> Figure 18: the response as XML (custom object) You will have to find out, from the source code (which may not be trivial) or from developers, which variables are required: in this example, we ll assume id is optional. Note also that our three new lines do not introduce the variables in the same order as the old ones: this is fine because it s Groovy. SoapUI cannot handle the situation where one of the variables is another custom object. Even if you do get all the pieces lined up, the call will probably fail. I don t have any way around this brick wall at present except that, again, the design might be questioned. Once the object has been built up, assign it back to the variable name (line 6 in figure 15) and run the script. If you see a dialog showing your inputs (figure 16), you are ready to run the test step. 30 PT - June professionaltester.com

12 Complex responses The AMF response is a binary object. You can see a representation of sorts on SoapUI s Raw response tab. However SoapUI also goes to great lengths to give you a more familiar representation: Response as XML. For simple response objects it usually does a pretty good job of guessing what belongs where so the output is quite readable (figure 17). For example, givenname could be validated using the XPath //givenname. For complex custom objects, the response can be more verbose (figure 18). In this case XPath axis is needed, eg //string[text()= givenname ]/ following-sibling::*[1]. Dealing with multiple architectures Unfortunately this method for handling complex inputs will not work for.net logic tiers, not least because SoapUI cannot import DLLs and serialization of objects is done differently by.net and Java. I have seen some workarounds tried: for example publishing the information on a special JMS test queue; duplicating information in the message headers; exposing methods as a SOAP call. All achieved some success but all require additional development work. If you will be testing a Flash application using.net the only suggestion I can offer at present is that you address this issue as urgently as possible Mark Lehky has worked in test automation in multiple industries since He blogs on the subject at and keeps in touch via LinkedIn ( Achieve test coverage of up to 95% Increase test automation up to 90% Reduce your time-to-market Free Demo