Web Services Testing, the Methodology, and the Implementation of the Automation-testing Tool 1

Transcription

1 Web Services Testing, the Methodology, and the Implementation of the Automation-testing Tool 1 Ying Li, Minglu Li and Jiadi Yu Department of Computer Science and Engineering,Shanghai Jiao Tong University, Shanghai , China {liying, li-ml, jdyu}@cs.sjtu.edu.cn Abstract. Web Services testing is essential to achieve the goal of scalable, robust and successful Web Services especially in business environment where maybe exist hundreds of Web Services working together. In this paper, we give detailed explanation about the Web Services testing methodology and skill, which are very helpful to the testers. Compared with tradition programming testing, the Web Services testing has its own feature such as performance, authorization, and security. Based on the knowledge of the aspects of Web Services, we design and implement a testing tool to perform some tests automatically. 1 Introduction With the growing of using XML Web Services [1,2,3], we find that the Web Services testing technique should be enhanced in the Web Service developing cycle. Although Web Services are web application [4], we could use tradition web testing methods, but it has own feature. Testing is essential to achieve the goal of scalable, robust and successful Web Services, testers should concentrate in some key points of Web Services testing in order to design test cases specific to the task. In this paper, we give the basic concepts for the Web Services testing and implement an automatic test tool for some testing. 2 Web Services Testing Methodology The Web Services are modular, self-described and self-contained applications [5]. With the open standards, Web Services enable developers to build applications based on any platform with any component modular and any programming language. More and more corporations now are exposing their information as Web Services and what s more, it is likely that Web Services are used in mission critical roles, 1 This paper is supported by 973 project (No.2002CB312002) of China, grand project (No.03dz15027) and key project(no ) of the Science and Technology Commission of Shanghai Municipality.

2 therefore performance matters. Consumers of web services will want assurances that Web Services won t fail to return a response in a certain time period. So the Web Services testing is more important to meet the consumers needs. 2.1 Unit Testing Unit testing is much more like tradition program test. We could apply this technical into web service s unit test. 2.2 Functional Testing Functional testing ensures that the functionality of Web Services are as expected. In functional testing, we should not only examine the basic input/output, bounds testing, error checking and so on, but also include the basic security/authorization examination, and test if the service should support all the communications protocols it applied which is also very important to Web Services. Although Web Services have no user interface, but they provide web methods to invoke, which provide us a way to use automatic script to test them. 2.3 Performance Testing Performance testing is often to determinate the relevant product statistics. For example: How many messages per second? How many simultaneous users of a service are acceptable? There are basically three ways to conduct performance testing: Ad hoc performance testing; Observational testing; Measured testing. Compared with tradition program and web program, there exist variety factors to effect the performance of Web Services, which include: Differentiated Web Services solutions; System model; Workload model [6], Transaction model [5], or even Security model. There are some key parameters to determine Web Services performance; we can use tradition web testing parameters to describe the efficiency or ability of Web Services. These common performance measurements include: MCs(Megacycles), Memory footprint, BoW(bytes over wire, TTLB(time to last byte), user-perceived response time, and the TTFB(time to first byte). In paper [6], SLA (service level agreement) will rule the relationship between users and services providers. It is very important for testers. 2.4 Load/Stress Testing The aim of Load/Stress testing is to find out the Web Services scalability in the growing of the number of the clients invoke them. Load/Stress testing can be applied with performance testing together.

3 Through Load/Stress testing, the typically bugs would be found more easily than other testing methods, such as: Memory leaks: Memory leak would be common in tradition programs. In Web Services, the programming language such as Java, C# is designed to automatic deallocate memory when objects are no longer used. But it is still possible that the objects would not be de-allocated, such as in java s programming using JDBC. Memory leaks not only appeared in programming language in Web Services, but also in some circumstances in database server. If too many connections appeared at the same time, the database server would not release the cursor in time in some circumstances. The memory leak is extremely difficult to detect. With few use of Web Services, memory leaks are very rarely found since the test does not generate enough usage of the product before it completes. Even through Load/Stress testing, few memory leaks still cannot be found. Concurrency and Synchronization: In Load/Stress testing, the testing program will generate many threads to act as virtual users to invoke Web Services, while Web Services can be invoked by others. In such complex circumstance, many different code paths and timing conditions will be performed. In general, the more code paths performed, the more error would be shown. The load/stress testing is ideal for automatic testing [7]. The implementation of the testing tools will be introduced at section Security Testing Generally, there are two kinds of Web Services, the Web Services are used in Intranet and the Web Services are used in Internet. Both of them face the security risk since message could be stolen, lost, or modified. The information protection is the complex of means directed on information safety assuring. In practice it should include maintenance of integrity, availability, confidentiality of the information and resources that used for data input, saving, processing and transferring. The complex character of this problem emphasizes that for it solution should be realizing the combination of legislative, organizational and software-hardware measures [8]. So, it is very difficult to test these automatically, but testers should carefully design the test cases according to the real environments and the protocols. The WS-Security 1.0 specification provides varieties ways for Web Services security, such as XML DIGSIG, XML Encryption [9]. The information protection is just one aspect of the security. In Web Services, the main challenge we faced is to consider the protection of resources such as data and applications so that this important information should be only accessed by the appropriate entities. That calls authorization. 2.6 Authorization Testing In real world, one Web Services would invoke another Web Services, In EAI, there exist thunders of Web Services that could be used together. Not every one could

4 access these services. So testing the authorization is very important to protect the invaluableness data. Authorization denotes granting authority, in practice, we often use access control list (ACL) or role-based access control (RBAC) to map from the entities to resources in order to assign rights for each resource. Table 1[10] shows the principal Web Services from an existing trade faire web sites as well as the categories of target users for each service (Table 2)[10]. Authorization testing should test each entity (user categories) to Web Services to get an access table to determine if the authorization is assigned correctly. We designed a semiautomatic tool to perform the authorization test, which will be discussed later. Task T01 T02 T03 T04 T05 T06 T07 T08 T09 T10 T11 T12 T13 T14 T15 T16 Table 1. Principal services offered by existing trade fair Web sites. Description To obtain information on how to get to the fair (transportation, ticket costs, time schedule, etc.) To consult the calendar of the appointments and conventions To obtain detailed information on the exhibitors To obtain detailed information on the exhibited products To consult the map of the fair To have information on job offering in the fair To buy ticket on-line To buy fair catalogue on-line To obtain information on the receptive structures near the fair (possibly to make reservation) To consult the call for tender for service providers To consult the official news To contact the administrative secretariat To watch the fair through a webcamera To watch the fair in 3D vision To buy advertising banner in the fair web site User registration for user profiling Table 2. Distribution of service targets for the user categories within fair trade business T01 T02 T03 T04 T05 T06 T07 T08 T09 T10 T11 T12 T13 T14 T15 T16 Organizer X Exhibitor X X X X X X Professional visitor X X X X X X X X X X X Generic Visitor X X X X X X X X X Service Visitor X X X X X Press X X X X X X X X X X X X X Fair Worker X X X X X

5 3 The Design of the Automatic Web Services Testing Tool (AWSTT) 3.1 Related Work The nunit.org [11] developed an open-source unit-testing tool for all dot net languages. Some primary developing environments provide the functional testing tools, such as Visual studio.net, IBM WebSphere Studio Application Developer, Weblogic workshop. But they re basic functional tools, none of them provides full functional testing mentioned in 2.2. Few companies develop third party Load/Stress tools, but some tools just use http post and get to simulate the user request, none of them provides security or authorization testing environment. 3.2 The Framework of AWSTT Since there exist good unit and functional test tools, we put our focus on the design of the 2.3,2.4,2.6. Figure 3 shows the framework of AWSTT. Detail information will be discussed later. Recorder Runtime Engine Script Generator System Managerment Execution Scheduler System Configurator SOAP Composer Execution Collaborator Test Case Execution Configurator Monitor Fig. 1. The framework of AWSTT 3.3 Performance and Load/Stress Testing Lots of Load/Stress automatic testing tools perform several tests at the same time, for example calling a number of Web Services on the same server simultaneously. The

6 tools first record user s action, then change the action into scripts; the runtime engine executes the script concurrently. Some Load/Stress tools capture programs catch all the http requests URL and then concurrently simulate number of users to open the URL, using http post get request to simulate the virtual user, whereas in production most Web Services will be invoked directly using the Simple Object Access Protocol (SOAP)[12]. Here we use JavaScript as the script language to record the action that user invokes Web Services from browser. After record, the SOAP Composer transfers the http post request from the script to SOAP message and save as a configure file to let runtime engine to process. Because one Web Services can provide more than one web method, and one server can hold more than one Web Services, we consider Load/Stress testing includes two aspects: Repetition testing. Repetition testing is the basic testing for one function or web method. That means running one Web Services over and over again. Functional test is to examine if the Web Services are working well, while repetition test is to justify if the Web Services could continue to work repeatedly. Concurrency testing. Repetition testing is to test one special web method; the aim of concurrency testing is to simulate the real world of the using of Web Services, it performs several operations simultaneously. With the concurrency testing, we can find some problems like thread safety, transaction problems and so on. We designed the runtime engine which can simulate up to 120 users concurrency, with each one can call one or more Web Services according to the configure file. The performance testing can be a part of Load/Stress testing. The tradition way to analyze the performance is to use the site s log data by using web mining technique [13], or use cookies to analyze user s visiting [14]. But these are not very suitable for Web Services. We currently use SOAP extension to catching SOAP message [15]. A SOAP message is composed by three major parts: a SOAP envelope, a SOAP header and a SOAP body [12]. We apply a new SOAP extension to Web Services to catch the SOAP message and log it to a text file for testing analysis. Analyzing these data should be very important. In the concurrency testing, Web Services might have transaction; some Web Services would be broken by another Web Services, or even cannot be executed. Currently, we do not provide such analyze tools to help testers to found such problems. 3.4 Authorization Testing The authorization testing using AWSTT currently cannot be performed automatically, because there exist many authorization methods. In different organizations, there may exist different authorization models such as LADP, authorization services. We cannot get the users or roles information from these authorizations directly. In AWSTT, we can simulate authorization in these ways:

7 Invoke and record all Web Services manually through the recorder. That can reuse the test case of the Load/Stress testing. Or we can get such information from UDDI. Manually assign users (roles ) rights in authorization server with same token or password. The users rights should be cover all the Web Services and there are no two users have exactly the same rights. Export the users name form the authorization server and import them to AWSTT AWSTT uses the each username, password to assemble a SOAP message, to invoke each Web Services and recorder the state. Generate the report. The SOAP message assemble must be modified by hand, because we cannot prognosticate what encryption method Web Services are used. 4 Summaries and Future Work In this paper, we give some methods for the Web Services testing. Web Services testing includes unit testing, functional testing, performance testing, Load/Stress testing, security testing and authorization testing. We give detailed information about the key points of the Web Services testing. And designed an automatic testing tool for some of these testing. The automatic testing tool we currently used has some features, such as SOAPbased log analysis, Repetition and Concurrency Load/Stress testing, and the authorization testing. But it also has some weakness; some features will be implemented in next version: Web Services navigation diagram. The Web Services navigation diagram can show how Web Services invoke other Web Services. Given one Web Service, we can get the special one s navigation diagram to other services. If we test all Web Services, then we can get a clear picture of how the Web Services are interacting. Transaction monitor and analyze. In EAI, transaction is a critical factor, which affects the Web Services performance deeply. The analysis of the transaction can help optimize the performance of Web Services. References [1] Carolyn McGregor, Santhosh Kumaran, Business Process Monitoring using Web Services in B2B e-commerce, Proceedings of the International Parallel and Distributed Processing Symposium (IPDPS 02) [2] M. Aoyama, E. Kawaguchi, Intelligent Software Services over the Internet, Information Modeling and Knowledge Bases, IX, IOS Press, Feb. 2000, pp [3] P. Brereton, et al., The Future of Software, CACM, Vol. 42, No. 21, Dec. 1999, pp

8 [4] Yuichi Nakamur, Satoshi Hada and Ryo Neyama, Towards the Integration ofweb Services Security on Enterprise Environments, Proceedings of the 2002 Symposium on Applications and the Internet (SAINT 02w). [5] Akhil Sahai, Jinsong Quyang, vijay Machiraju, End to-end Transaction Management for Composite Web based Services, 2001, IEEE. [6] Valeria Cardellini, Emiliano Casalicchio, Michele Colajanni A Performance Study of Distributed Architectures for the Quality of Web Services, Proceedings of the 34th Hawaii International Conference on System Sciences [7] Alan Booth, Andrew Citron, Stress testing your software without stressing your testers: Automation is the key. [8] V.P. Shyrotchin, V.Ye. Mukhin, MEANS AND METHODS FOR THE INFORMATION PROTECTION IN THE GLOBAL NETWORK INTERNET, ies2000 [9] Web Services Security (WS-Security) Version April [10] Claudio Muscogiuri, Gerald Jaeschke, Aldo Paradiso, Matthias Hemmje, FAIRWIS: An Integrated System offering Trade Fair Web-based Information Services A R&D Case Study, Proceedings of the 35th Hawaii International Conference on System Sciences [11] [12] Simple Object Access Protocol (SOAP), [13] Sankar, K.P., Varun, T., Pabitra, M.. Web Mining in Soft Computing Framework: Relevance, State of the Art and Future Directions. IEEE Transactions on Neural Networks, 2002, 13(5) 1163~1177. [14] Cooley, R., Mobasher, B., Srivastava J.. Data Preparation for Mining World Wide Web Browsing Patterns. Knowledge and Information Systems, 1999, 1 1(1) 5~32. [15] A.Sahai, V.machiraju, J.Ouyang, K. Wurster Message Tracking in SOAP-Based Web Services, /02 IEEE 2002