Run-time test configurations for load testing

Transcription

1 Run-time test configurations for load testing Gábor Ziegler, Ericsson Hungary Ltd. Contents Introduction What is TITANSim Motivation for TITANSim Functional description of the parts of TITANSim CLL, Application Libraries and Control Logic Three different perspectives of TITANSim: HW, SW, Run-time configurations Comparison of the possible test configurations local scheduling vs. central scheduler PTC Conclusions and questions Ericsson AB 2007 ETSI TUC (25) Ericsson AB

2 Introduction About TITANSim Introduction TITANSim is aimed at performance testing With TTCN-3 as the specification language With Ericsson s internal TTCN-3 Executor and Compiler tool Performance testing needs highly optimized test suite code Two contradicting requirements: A framework shall be general generalization Optimization is task specific specialization Ericsson AB 2007 ETSI TUC (25) Ericsson AB

3 Motivation for TITANSim TTCN-3 and TITAN is widely used for functional testing throughout Ericsson TITANSim aims to achieve cost savings via reuse of the function test code base reuse of testers competence reuse of existing, in-house tools reuse of the new performance test solution by different projects through-out the company Ericsson AB 2007 ETSI TUC (25) Introduction Functionalities and applications of TITANSim Ericsson AB

4 Functionality of TITANSim CLL Run-time interaction with the test suite A dynamically configurable run-time GUI and Parameters Statistics Generic support for common programming tasks Memory management support: resource pools Scheduling support A logging framework Generic support for distributed scheduling: EventQueue data type + support functions Concrete support for central scheduling Ready-made scheduler component for central scheduling Load balancing, regulated load, external execution control, trafficmixer and a graphical console for all these Other useful data types and algorithms: Linked lists (FreeBusyQueue), hash tables, binary search tree (Red-Black trees) Ericsson AB 2007 ETSI TUC (25) Functionality of Application Libraries and Control logic Application libraries: simulated entity specific tasks Protocol message handling Inbound message routing in case of multiple generator PTC Protocol specific TITANSim parameters and TITANSim statistics Building blocks and state-machine support for Control Logic Control logic: realization of particular traffic cases Ericsson AB 2007 ETSI TUC (25) Ericsson AB

5 Introduction The 3 perspectives of TITANSim Views of a TTCN-3 load test library At least 3 perspectives have to be considered: 1. HW perspective: which hardware to use? 2. SW perspective: how to modularize your code? 3. Run-time perspective: what is the best run-time testconfiguration? Th is t a lk s ma in t o p ic Ericsson AB 2007 ETSI TUC (25) Ericsson AB

6 1. The HW perspective control SW SW and HW are separated One SW many HW To expand load capacity only HW units shall be added UE#1 IMS Core UE#1 IMS Core SIP UE#1 RegistrationIMS Core SIP UE#1 RegistrationIMS Core SIP SIP PUBLISH UE#1 RegistrationIMS Core SIP SIP PUBLISH UE#1 RegistrationIMS Core SIP SIP PUBLISH UE#1 RegistrationIMS Core SIP SIP 200 SIP PUBLISH UE#1 OK RegistrationIMS Core SIP 200 SIP SIP PUBLISH OK UE#1 RegistrationIMS Core SIP 200 SIP SIP PUBLISH OK UE#1 RegistrationIMS Core SIP SIP 200 SIP PUBLISH UE#1 OK RegistrationIMS Core SIP 200 SIP SIP PUBLISH OK UE#1 RegistrationIMS Core SIP 200 SIP SIP PUBLISH OK Registration SIP SIP 200 SIP PUBLISH OK Registration SIP SIP 200 PUBLISH OK SIP SIP 200 PUBLISH OK SIP 200 OK SIP 200 OK SIP 200 OK Ericsson AB 2007 ETSI TUC (25) 2. The SW perspective Three levels approach: Core Load Library (CLL) provides the generalization Application specific framework libraries (AppLibs) provides the specialization code provided in-cooperation with project experts, relies on core library code Control logic : can provided by non-experts, as well builds on both application specific and core libraries code Ericsson AB 2007 ETSI TUC (25) Ericsson AB

7 3 Run-time configuration perspective This talk s main topic Careful trade-off must be made between Efficiency Resulted code complexity Load testing means concurrency handling: Many(!) parallel traffic flows over some shared resource pools! TTCN-3 has a special concurrency model PTC-s are run concurrently A PTC is a single CPU system No concurrency support below PTC level by the language PTC-s are run isolated from each other no shared memory Our dilemma: on which level do we handle the concurrency? Ericsson AB 2007 ETSI TUC (25) Alternatives for run time configurations The simple approach is follow the usual TTCN-3 semantics Concurrency is to be handled on PTC-level A single PTC is responsible for a single transaction The advanced approach is to let a single PTC handle multiple concurrent transaction Concurrency is to be handled below PTC-level A single PTC is responsible for multiple transactions Ericsson AB 2007 ETSI TUC (25) Ericsson AB

8 TITANSim Run-time test configurations Central scheduling Central scheduling Run-time GUI/CLI CPU monitoring MainAdmin Host_Admin Host #i MTC LoadRegulator Execution Control External control Host #j XXX_Scheduler DB XXX_CT ZZZ_Scheduler DB ZZZ_CT Control logic and stack for protocol XXX XXX_YYY_Routing Traffic generator components ZZZ_Logger Host #m ZZZ_Logger Control logic and stack for protocol ZZZ ZZZ_WWW_Routing YYY_PT SUT Load measurement WWW_PT Ericsson AB 2007 ETSI TUC (25) Ericsson AB

9 Central scheduling Pros It is the most user-friendly It requires no coding-paradigm change with respect to function tests Suitable for ad-hoc load testing projects It can be used without an application library Can use ready-made scheduling functions that are totally independent of load generator component-type Cons Less efficient Wrong scalability Each traffic initiation requires internal communication with a single central entity: extra overheads and delays Sharing data of a run-time database across traffic cases / entities is difficult and inefficient Ericsson AB 2007 ETSI TUC (25) TITANSim Run-time test configurations Local (distributed) scheduling Ericsson AB

10 Local scheduling Run-time GUI/CLI MainAdmin MTC Host #i LoadRegulator Traffic generator components XXX_ApplAdmin PtcFunctions Control logic and stack for protocol XXX XXX_LGen_CT ZZZ_ApplAdmin Host #l Host #k Host #j Traffic generator components ZZZ_LGen_CT PtcFunctions Control logic and stack for protocol ZZZ XXX_YYY_Routing XXX_Logger Host #m ZZZ_Logger ZZZ_WWW_Routing YYY_TP SUT Load measurement WWW_TP Ericsson AB 2007 ETSI TUC (25) Local (distributed) scheduling Pros It is the most-efficient Less dependent on OS-scheduler Sharing data of a run-time database across traffic cases of the same PTC can be easy and efficient Load generator PTC-s schedule on their own no internal communication overhead needed for load generation They can run autonomously scalability! Cons: It requires some sort of an application library: Explicit concurrency handling shall be set up It requires coding-paradigm change with respect to FT: eventbased logic Writing and using reusable ready-made scheduling algorithms requires dirty-tricks w.r.t. TTCN-3 language Ericsson AB 2007 ETSI TUC (25) Ericsson AB

11 Thank you for your attention! Questions? Ericsson AB 2007 ETSI TUC (25) Ericsson AB