Industrial Adoption of Automatically Extracted GUI Models for Testing

Transcription

1 Industrial Adoption of Automatically Extracted GUI Models for Testing Pekka Aho, VTT Technical Research Centre of Finland International Workshop on Experiences and Empirical Studies in Software Modelling (EESSMod 2013) 1 Oct 2013, Miami Beach, Florida, USA

2 2 Agenda Acknowledgement and background Automatically extracting GUI models Experiences on extracting the models Using the models in industrial software testing Related work Benefits and shortcomings Conclusions and future research

3 3 Acknowledgement and background The actual engineering work on innovating and implementing the Murphy tool set was done by F-Secure, especially Matias Suarez The idea and motivation for the work came internally from F-Secure A lot of code and scripts related to test automation The maintenance of graphical user interface (GUI) related test automation scripts (or manually crafting the models) is tedious because the GUI changes fairly often The idea was to create a tool to extract a state model from the GUI by automatically crawling through the GUI application and using the model to support the automated testing

4 4 Automatically extracting GUI models Murphy tool set the approach: Dynamically analyses the GUI while automatically trying out all the possible user interactions of the application crawling the GUI The main idea is to traverse through all the possible states of the GUI application and automatically construct a GUI state model (based on finite state machine (FSM)) The goal is to discover as many states (nodes of the model) as possible (not to find all the possible paths between the nodes) To avoid state space explosion problems, the data values are not considered when distinguishing new GUI states from the previously visited states

5 5 Automatically extracting GUI models Murphy tool set the implementation: Provides generic UI crawling and window scrapping services as a library Various approaches (called drivers) for recognizing elements and windows of the GUI application are used to increase platform independency Comparing automatically taken screenshots to find the changing areas during automated interaction (e.g. the bounding rectangle of the selected element, the shape of the mouse cursor) and reason the structure and behavior of the GUI The process of extracting the model is mostly automated and fully customizable (hooks and callbacks provided) The script that is used for invoking the generic UI crawling library can be modified with application specific rules

6 6 Automatically extracting GUI models Murphy tool set the generated models: A directed graph is used to model the behavior of the GUI The nodes of the model are states of the GUI screen or window The edges are actions that the end user could perform in that specific state of the GUI, triggering changes of the GUI state A state of the GUI is defined by the appearance and the interactions provided for the user, but not by the data values (e.g. text in a text field) A dialog with an enabled OK button represents a different node than otherwise similar dialog with the OK button disabled Two similar dialogs with same enabled actions represent the same node regardless of the value in a text field of the dialog Screenshots of the GUI are used for visualizing the GUI state model

7 7 Automatically extracting GUI models Murphy tool set the generated models, an example:

8 8 Experiences on extracting the models Experiences on using the Murphy tool set to extract GUI models of commercial software products during the development: The Murphy tool has been able to satisfactory extract most parts of the UI flow of the GUI applications of F-Secure with very little user intervention The invocation scripts were usually between 10 and 200 lines of Python code and produced models capable of exercising in the range of 80% of the possible UI flows The generated models ranged between nodes (GUI states) Most of the GUI applications used in experimenting the approach and Murphy tool set were flow based GUI applications having a relatively low amount of the possible input values

9 9 Using the models in industrial software testing Experiences on using the generated models to support automated testing: The generated models were used to create test scripts to replace manually written test automation scripts, reducing the amount of code that has to be maintained In order to automate the testing of the internationalization of the GUI applications, Murphy was used to create models to cover all 29 different languages of the modeled products. As the basic functionality of the application is supposed be the same regardless of the selected language, the same invocation scripts could be used for creating the models of each language with a minor modification of changing the selected language into the script The models were used also to support manual testing actions. Murphy tool set has functionality to create and launch a virtual machine with the modeled GUI application and drive the application into the state selected from the model

10 10 Related work GUI Driver tool by Aho et al. Dynamic reverse engineering of Java GUI applications to generate an FSMbased state model and structural models for each of the GUI states Compares structural models to observe the changes happening in the GUI during the execution GUITAR tool by Memon et al. Using GUI ripping to create structural models, similar to GUI Driver Not able to model dynamically changing GUIs (the visibility of a GUI component depends on the state of another component) Models events (GUI actions) as nodes instead of transitions, creating a map of event sequences GUITAM by Miao et al. FSM based GUI Test Automation Model No details about the tools or syntax of the models available

11 11 Benefits and shortcomings Pros (-): Compared to the related approaches, the main advantage of Murphy tool set is that it does not restrict the modeled GUI application to a specific programming language or execution platform The tool set was built to integrate into an industrial development and testing environment (including continuous integration and virtual test environment) No dependency to updated specifications Cons (+): Generated models describe the implemented behavior instead of the expected or specified behavior -> the generated tests are based on the implementation instead of specifications Executable implementation required

12 12 Conclusions and future work Conclusions: Using automatically extracted models to support automated GUI testing can significantly reduce the amount of hand written code compared to manually written test scripts, reducing the maintenance effort related to test code With the help of Murphy tool set, it is possible to use the models to support and reduce the effort required for manual GUI testing Future work: The approach and Murphy tool set seems to work also on Android mobile applications, but the experiments are not yet finished Murphy tool set is in the process to be published as open source

13 13 Thank you! Any questions? For further information contact: