The Research and Design of NSL-Oriented Automation Testing Framework Chongwen Wang School of Software, Beijing Institute of Technology, Beijing, China wcwzzw@bit.edu.cn Abstract. By analyzing the Selenium and other open source testing tool, the lack of Selenium and the design of testing scripts are given to discuss and try to improve to resolve problems of NLS. These improvements include the using of page elements, enhancement of the response of the heavyweight component, optimization of testing scripts for multi-language versions. The parallel execution strategy for multilingual test cases has been provided, through which the users can execute test cases of multi-language in a great number of test servers at the same time, greatly improving the overall testing efficiency. The testing framework proposed has been applied to the actual web product globalization testing, and achieved very good results. Keywords: Automation testing, testing framework, NLS, Selenium. 1 Introduction As Global software production and service to become an important development strategy for more international software companies, Software products need an international software design, development, testing and services to gain more international market share. So Testing for the NLS (National Language Support) becomes more and more important. But currently the existed automation test framework does not cover the content of global test. Therefore, we need to expand and improve the technology of global testing. Currently NLS-oriented test is used after function test and do not own a single automated test for itself. Many functional test scripts are reused in NLS-oriented test, so the problems found in NLS-oriented test will cause deep impact even need to refractor the whole software design. Therefore a special framework of automation testing is needed for NLS-oriented test. Based on the actual project requirements, we proposed a NLS-oriented automation testing system which based on the open source tools of Selenium, ANT and JUnit. 2 Choice of Automated Testing Framework Software to be tested is collaboration office software based on B/S structure which uses WEB2.0 technology and provides all CRUD operation with basic data for the collaboration suite, with powerful data management capabilities. Testing work need to D. Jin and S. Lin (Eds.): Advances in MSEC Vol. 1, AISC 128, pp. 367 373. springerlink.com Springer-Verlag Berlin Heidelberg 2011
368 C. Wang support the globalization test with ten languages. The test script need to be reused and the operations which will be repeated need to cover all features. In common test automation frameworks, the test framework of keyword-driven/table-driven need to design a data sheet box of keyword and its development cycle is long. It is suitable for operation with large amounts of data and is not suitable for our project. The library test framework is not selected because the function libraries of our software are too complex. In this project, we select a hybrid testing framework with modularity test framework and data-driven testing framework for the script development. Modular testing framework needs to create the scripts. The small scripts with tree structure can be constructed for a particular test case which provides modular features of program design. Despite it is difficult to be recorded; the code can be instead of script in the design of framework. It is simple and high controllable which makes the development more effective. Data-driven testing framework is a framework which reads input and output data from files and loads the data into the variables in capture or hand-coded scripts. It reads the testing data from testing script and stores in external files or database. In the testing process, it reads testing data from files dynamically. Extending test script by replacing the testing data with parameters, the test processes are not limited only with the testing data when test script is recorded. Through the way of parameters, we can read the test data from external data source or data generator. Then we can extend the test coverage and improve test flexibility. 3 The Choice of Automated Testing Solution We intend to use the Selenium tool and use Java language to implement the automated test for our project. The test program is shown in Figure 1. Import Selenium and JUnit tools, we wrote the test scripts with Java language. By the Selenium tool we implemented the simulation of web operations. By the JUnit tool we organized the basic framework of scripts and verified the implementation checkpoints. Writing methods, using XML format file to parameter all data elements. Which data elements include basic configuration data, test record data, GUI element data and so on. Writing configuration file and using Ant tool to build the test project. Its main tasks include set environment parameters, start the Selenium Server, compile program, run the test cases and generate logs. Ant can be built automatically and JUnit can run the automation test. Using Ant and JUnit together, we can make the build and test process automatic. In order to ensure the labels of <JUnit> and <JUnitreport> can be identified by Ant tool, we need to copy three jar files which included by the JUnit to the directory of Ant library. Compared to the general Selenium test solution, our program takes the following advantages. Firstly, it is easy to control with the Ant tool. If the test task changed, we can restructure the configuration files of Ant tool to implement the modified task. Secondly, by using the data-driven testing framework, all data is stored in the XML
The Research and Design of NSL-Oriented Automation Testing Framework 369 Ant tool Define task by configuration file and drive test task Target <Selenium> Task<runtest> Start Selenium Server Initialize environment Compile program Run test case Generate report Parameter files Test_Data.params Test_GUI.params Test case program Action defined program Jar tool Sel-server.jar Sel-driver.jar Test case mode Setup(); Teardown(); SelTest(){ Sel.open(<url>); Sel.click<>; AssertTrue<>; } Fig. 1. Schematic diagram of the test program format files and is easy for maintenance and modification. Finally, by saving the environment parameters, test data parameters, test result parameters and extracting test reports, it is convenient for tester to view and invoke test cases. 4 Improve Selenium Tool Selenium tool has many advantages such as supporting multiple browsers, protocols and program languages and providing multiple ways to locate the elements of web applications. However, there are a lot of shortcomings for Selenium tool. Firstly, a lot of page elements such as name, id, xpath and so on were used in Selenium. Both the functional changes and UI reconstruction or interaction improvement will affect these elements which makes the test vulnerable. Secondly, it depends on the existence of specific data. The test will fail when individual data is no longer valid and such failure is not identified as the missing of function. Finally, Selenium is a lightweight testing framework and the script deals with simple-formed test case. In response to the above shortcomings, this paper proposed methods to improve Selenium tool and scripts. 4.1 Improvement for Page Elements In Selenium toolkit, the operating functions use a lot of page elements as arguments directly and also depend on the existence of these data values. When the data in an individual page element is no longer valid, the test will fail. And the failure is not identified as the missing of the functions which makes the Selenium test fragile and the attributes in the same page element need to be modified repeatedly. In order to fix the above problem, we write a program to replace parameters by extracting the page
370 C. Wang elements. The page parameter file is defined as xml format and named as the module name. For instance, the parameter file is named test_gui_user.params for user management module. The basic format of parameter file is as follows. <?xml version = "1.0" encoding = "UTF-8"?> <parameters file=""> <param name="newuserbutton"> _sbox: b5c109e: 0_1: b5c10a1 </ param>...... </ Parameters> In order to improve the robustness of the program, we intent to use xpath and other stable and well-marked page elements as the page argument. Meanwhile, in order to improve the determination logic, we will check page elements multiple times to confirm the correct page. 4.2 Improve Response of the Complex Test Cases Selenium is a lightweight testing framework and the script deals with simple-formed test case. The essence of Selenium is sending request to complete test case through the HTTP protocol. Therefore, Selenium command will do not respond to some heavyweight components and it is difficult to handle test cases with strong logic business relationship There are two solutions for the above situation. The first solution is to rewrite the JavaScript Event simulation code in Selenium toolkit. The disadvantage of this approach is that testers can not view the test process from the user perspective. The second solution is to use Java.awt.Robot to simulate the actual mouse movement. We need to rewrite the mouse movement code. In Selenium toolkit, the HTML element can be located by Xpath, DOM API or other attributes value. By calling the Selenium function, the coordinates of HTML elements are returned. Originally, Selenium Server communicates with browser by AJAX (XmlHttpRequest) directly. Now it could invoke the method of Java.awt.Robot to simulate actual mouse movement after the coordinates of HTML elements were returned. 4.3 Support for Globalization Test As the same with the majority automation testing tools, Selenium is also lack of globalization testing support. If the parameter file is not specified, Selenium Server will start a browser execution test with system default setting. The default language is the current browser version language. This test can not provide test data with multiple language and character set. In order to improve the above drawback of Selenium toolkit and extend it for the globalization test, we design the structure of test scripts and add the environment setting and parameter files for the Selenium RC-packaged Java toolkit. 1) Make the elements of language-specific independent of the test script code and support script reuse of multiple languages In order to reuse the whole test scripts for multiple languages, we make all elements which depend on the specific language independent of the test script code and store them in the parameter files. The structure design of the test script is as follows.
The Research and Design of NSL-Oriented Automation Testing Framework 371 The source files of test cases are stored in the src directory and the parameter files are stored in the param directory. The parameter files used xml format which are divided into three categories according to the purpose of parameters and the relationship with languages. a) The parameter files which stored fixed parameters. In such files, we stored the parameter data which was not changed for different language versions. These fixed parameters can be the input string during test process or the property values of interface elements such as the test machine IP address, the id value of the button UI, the xpath value of table UI and so on. b) The parameter files which stored the input random data according to the test language and character set. According to the requirements of test language and character set, we analyze test cases which deal with string input/output directly or indirectly and extract the random data which can be replaced with any international input type. When the test is running, we can replace the input data to test the software capabilities of handling different languages and character sets. For instance, a new user name was needed when the test case of create new user is running. In this design, the user name can be replaced of data with any international input type. 2) Select the languages and character sets dynamically and support for multiple languages testing process In order to meet the requirements of globalization test, we need initialize test environment such as add setting for test environment language and character set at first. The options of language and character set include not only the choice of current test software language versions, but also the choice of the current operating system language and character set. In our project, the major task is to achieve the options of the language and character set between the client operating system (Redhat4.0) and browser (Firefox3.0) If the parameter file is not specified, Selenium Server will start a browser execution test with system default setting. The default language is the current browser version language. The setting of specific browser language and character encoding can be implemented through the following ways. Firstly, the test language and character set will be stored in the configuration file according to the choice of testers. Secondly, according to the browser language and character set of test requirements, we set the user language preference in Firefox parameter file which named prefs.js. Finally, we add the option of "firefoxprofiletemplate" and point it to the modified parameter file directory of Firefox when the Selenium Server is starting. For instance, we select the language Chinese and GB2312 character encoding as the user preference and add the following statements into the specified parameter file of Firefox which named prefs.js. export LC_ALL = zh_cn.gbk export LANG = zh_cn.gbk user_pref ("intl.accept_languages", "zh-cn") user_pref ("intl.charsetmenu.browser.cache", "GB2312") In order to set the language and character set of client operating system (Redhat4.0), we can modify file in /etc/sysconfig/i18n or set the parameters of LC_ALL and LANG.
372 C. Wang Meanwhile, the above method can be extended to the situation with multiple languages and character set environment. Firstly, all of the languages and character sets were stored in the configuration files. Then, the test was ran serially or parallel in the test machine. The serial execution means initialize test environment and loop test process step by step. And parallel execution means pass test information to multiple target machines through socket communication and initialize test environment parallel and run all of the test process. 3) Use regular expressions to deal with the translation unit contains special characters In the above automated process of parameter file replacement, a large number of XML format translation file data need to be searched. However, some special characters written to the XML file will be replaced by entity references. Therefore, as shown in Table.1, the displayed string in target screen may be different with the translation string which stored in the XLIFF translation files. Table 1. The characters replaced in the translation files. Character shown in user interface & Character in XLIFF format translation file & ' " < < > > We can use regular expressions to solve the inconsistency displayed string between the original string and the translation string. Regular expressions provide a powerful, flexible and efficient way to handle text data. By the pattern matching method of regular expression, we can compare and match strings to determine the test results are correct or not. 5 Strategy for NLS-Oriented Test Task When NLS-oriented test task is running, multi-language test cases tend to make the number of test cases exponentially growing which extend test time greatly. And the probability that cause some unexpected error in the test process will be higher. Therefore, our project uses parallel execution strategy for multi-language test. When the test languages in the instruction are over a number of test cases, we will split it into small tasks. Test controller program will find the idle test machine according to the status of test machine and the corresponding task amount. Then the test commands are sent to the test server through the socket communication. This parallel test platform with remote control will reduce the execution time of software test, raise the efficiency of execution and provide a valuable engineering reference for the large-scale software test. The test workflows of remote control are as follows.
The Research and Design of NSL-Oriented Automation Testing Framework 373 The client proxy viewer and monitor were deployed in the test server which got the status of the test server by monitoring the relevant processes and updated the state property of test machines in the server database. Testers accessed the front controlled interface based APEX through the WEB page, set up the task parameters and controlled the task execution. In the control server, we find the idle test server according to the state information stored in the database, and then send the test command to the Agent program in test server through the socket communication. The proxy program which was deployed in test server accepted the test task, controlled and driven the test script. The client proxy viewer and monitor were deployed in the test server which got the status of the test server by monitoring the relevant processes. If the test end is detected, the test results will be sent to tester and the state property in controlled server will be updated. 6 Conclusion In this paper through the production testing process of a WEB collaboration suite, we proposed an automation testing system of NLS-oriented which based on the Selenium, ant and JUnit tools. For the problems in NLS test, we improved the Selenium tool and test scripts. And we provided a parallel execution strategy for multi-language test which can run multiple test process in multiple servers and improved the overall test efficiency greatly. Of course, the test for the NLS, there are many issues involved need to continue in-depth study, such as multi-language environment deployment of test cases, test cases quickly build multi-language and how to automate the maintenance of test case library, and so on. However, there are many issues in NLS-oriented test to research for such as how to deploy test environment for multi-language's situation, how to build test cases for multi-language quickly, how to maintain a test case library automatically and so on. References [1] McMahon, C.: History of a large test automation project using selenium. In: Proceedings - 2009 Agile Conference, AGILE 2009, pp. 363 368 (2009) [2] Bruns, A.: Web application tests with selenium. IEEE Software 26(5), 88 91 (2009) [3] Xu, D.: A Tool for Automated Test Code Generation from High-Level Petri Nets. In: Kristensen, L.M., Petrucci, L. (eds.) PETRI NETS 2011. LNCS, vol. 6709, pp. 308 317. Springer, Heidelberg (2011) [4] Patel, S.: TestDrive - A cost-effective way to create and maintain test scripts for web applications. In: SEKE 2010 - Proceedings of the 22nd International Conference on Software Engineering and Knowledge Engineering, pp. 474 477 (2010) [5] Selenium, http://wiki.javascud.org/display/sel/home [6] HtmlUnit, http://htmlunit.sourceforge.net/