Software Test Management Involving Client Relationship and Application Virtualization

Software Test Management Involving Client Relationship and Application Virtualization Abstract Senior Manager,IBM India Private Limited, Bangalore Dr. Avijit Kar, Professor of Computer Science and Engg. Jadavpur University Kolkata In the last decade software testing saw a phenomenal change because of distributed development due to globalization and cloud computing environment. Now-a- days in testing we find systematic planning, management, use of latest technologies and metrics. In this paper an attempt has been made to discuss two aspects in software test management namely the client /customer relationship and application virtualization. The various issues for the implementation of these two aspects as faced by test manager are discussed. The whole idea is to provide customer centric testing and providing the test team with 24/7 environment availability. The approaches described in this paper are illustrated with case studies. The advantages and future scopes are illustrated. 1. Introduction We have seen a lot of changes in software testing methodologies in the last few years. This is because of several reasons like advanced technologies, use of cloud computing environment, use of testing teams at different places even in different countries in a distributed manner, use of customer`s feedback during testing and use of scientific metrics. Hence for software testing a systematic planning and management of the activity has become a necessity. We find the use of several new techniques in the software testing methods. Client relationship and application virtualization are two such management tools. Client / customer have become focus for all activities [3] of the organization. In fact we have moved towards customer centric activity [2]. Planning, communication, sharing, network time management, workflow, deadline of the work and review are some of the activities in which customer is very much involved. Hence client relationship has become a necessity. Client relationship has become integral part of many management activities including software testing. In virtualization, we do simulation of the hardware, software and other physical systems so that this simulated environment can be used in place of the physical system for several purposes [7,8]. This is done so that we can simplify the system, manage the system better, have an easier design, reduce time and cost. Virtualization has been used for (i) Operating systems where several operating systems uses the same hardware (ii) Networks where the allowed bandwidth is distributed into several channels which is then given to real time server. By doing this, network complexity is simplified. (iii) Storage Virtualization where several storage devices of storage area networks behave 18

as a single storage so that these can be controlled by a single console device (iv) Server virtualization where the physical resources are shared by several users so that resources are better managed, utilized thereby making more cost effective (v) Desktop Virtualization where same terminal services are shared by several users (vi) Application Virtualization where in distant systems software is used and the applications can be anywhere. We are encountering the tremendous challenge of application complexity increasing day by day. At the same time the application development processes are not changing at the same pace. This leads to missed project deadlines and rise in costs and compromise in quality for optimization of costs and deadlines. Application Virtualization is the practice of simulating the behavior of an application that interfaces with the application under test through a virtual image [9]. This paper has been divided into several sections. Section 2 describes the need of client relationship management (CRM), various issues and the steps needed to strengthen the relationship management. Section 3 describes a test case of CRM. Application virtualization practices are given in section 4. A case study of application virtualization practices is given in section 5. Concluding remarks with scope for further work has been given in section6. 2. CLIENT RELATIONSHIP MANAGEMENT [1] Client relationship management (CRM) is needed for improving the customer satisfaction which will lead to increase in the profit and will retain the client for a much longer period of time. Here one has to determine strategies, one has to understand customer behavior and it involves the entire process [4,5,6]. [2]. From Fig. 3 one can see how the business of an organization is dependent on customer/client 19 Fig-3: Relationship between Customer Satisfaction and organization s business We can see that with customer satisfaction there will be long term relation with client, i.e., customer retention, repeat business and referrals to other clients. In addition to core product the other

aspects like processes and support, technical performance of the application / product, interaction with organization at various levels and emotional elements are also very important for customer satisfaction which has been shown in Fig. 4 [2]. Fig-4: Drivers for Customer Satisfaction The client relationship has four levels (steps) namely the service offering based, need based, relationship based and trust based as shown in Fig. 5 [2] 3.1 Feedback in CRM : 20 Fig-5: Levels of Customer Relationship

There are two types of feedback. Formal feedback or the engagement level feedback is a regular part of the process and can be taken at definite intervals say quarterly during the completion of the software. Informal feedback is usually taken during coffee/lunch break to know the customer need and to help the project in between milestones so that one need not wait till the formal feedback. Both formal and informal feedback help us in the following three aspects [2] (i) (ii) (iii) One can determine effective tests so that testing effort is reduced. Client, will consider the project team as 'resource provider' and 'trusted advisor' rather than a 'solution provider' There will be feeling in clients that client relation is enhanced without much other strategy from their side. 3.2 CRM steps and Client's Advantages : For client relationship management we take the following three major steps [2]. (i) (ii) (iii) Re-usability of common library modules and test cases. This will lead to reduction in test case preparation time and in test case preparation productivity. In order to do this one has to decide the guide lines for the use of the library modules. Process Improvements - This has four steps (a) Selective testing of critical functionalities (b) Defect review (c) Defect error reduction (d) Bench mark productivity internally and externally. For each of these guidelines are decided. These steps help us to reduce test cycle time, reduce defect error and enhance quality of testing. Knowledge management initiatives - This involves transitions, training, learning, induction training and planning. With this the functionality issues in the team is resolved, domain competency in the team is built, reduction in people dependency and one can identify the things to be shared with client. 3.Test Case Monthly status reports of a test case are shown in Fig. 6 through Fig. 10. Test Case Preparation Productivity (TC/ Person Day) 7.4 7.2 7 6.8 6.6 7.37 7.24 7 7 6.91 Mar07-TCA Mar07-TCB Mar07-TCC Jun-TCB Jun-TCC Fig-6: Test Case Preparation Productivity for various release cycles- Increasing Trend 21

Test Case Execution Productivity (TC/ Person Day) 5 4.89 6.5 7.1 4.8 4.6 4.4 4.57 4.85 4.2 Mar07-TCA Mar07-TCB Mar07-TCC Jun-TCB Jun-TCC Fig-7: Test Case Execution Productivity for various release cycles- Increasing Trend Test Coverage (%) 95 90 85 80 91 87 85 83 79 75 70 Mar07-TCA Mar07-TCB Mar07-TCC Jun-TCB Jun-TCC Fig-8: Test Coverage Reduction (Risk Based Testing) for various release cycles- Reduced Test Execution Time % Defect Error 15 10 13.2 10.1 10.4 9.56 9.52 5 0 Mar07-TCA Mar07-TCB Mar07-TCC Jun-TCB Jun-TCC Fig-9: Defect Error Reduction with every release cycle Improved Testing quality 22

Review Effectiveness (%) 8 7 6 5 4 3 2 1 0 7.01 7.3 6.6 5.4 5.42 Mar07-TCA Mar07-TCB Mar07-TCC Jun-TCB Jun-TCC Fig-10: Review Effectiveness increased with every release cycle One can see the advantages mentioned in the previous subsection. 4. APPLICATION VIRTUALIZATION PRACTICES[10,11] The AV project is conducted using a software simulation testing tool. The software simulation testing tool is a testing workbench that supports the virtualization of applications, which accomplishes virtualization through the recording of interactions between applications and then exposing the applications, variables and test parameters as a stored test program executable codes[12,13]. The three phases of AV activities are: a) AV Startup Phase This phase consists of installing and configuring LISA software, verification of network connectivity and doing the Business Value Assessment (BVA).This phase will happen during the start of engagement and will happen only once. It has four tasks: (i)task1: Install and Configure LISA Software.This involves deploying the itko LISA Application Virtualization software into the designated enterprise test environment server and workstations, contingent upon the software License Agreement. The work product of this task is deployed itko LISA software. (ii)task2 : Develop Application Virtualization Business Value Assessment (AV BVA).This involves collecting and analyzing data and business requirement related to the BVA, generate and review the business case, compile findings covering reductions and test efficiency opportunities with estimated investment and overall savings. The work product of this task are Business Case and Business Value Assessment. 23

(iii)task3: Develop AV Strategic Roadmap.This includes utilizing the Business Value Assessment information to develop an AV Strategic Roadmap of the key applications and interfaces to be virtualized. The deliverable of this task is AV Strategic Roadmap. (iv) Task4: Develop AV Services Catalog. This involves utilization of AV Strategic Roadmap to develop an AV Services Catalog which will be used to manage, track and re-use Virtual Service Models (VSMs).The work product of this task is the AV Services Catalog. b) AV Implement Phase This phase consists of the primary task involved in virtualizing services. The tasks within this phase will be performed on each named application as contained in the AV Strategic Plan that is targeted for virtualization. It has 9 tasks: (i)task1: Implementation Startup. This includes preparation for the Implementation Startup meeting, setting up test server and workstations, preparing list of applications, interfaces, test cases, test environment constraints and dependent downtime test requirements, establish working teams, develop AV Project WorkPlan based on the project objectives, schedule and high level project plan. The deliverable of this phase is the AV Project WorkPlan. (ii)task2: Build Test Environment Requirements. This involves building of the test server and workstation environments based on the data available. This involves performing a gap analysis and developing the Gap Analysis Report. The deliverable of this is the Gap Analysis Report. (iii) Task3: Conduct Application Virtualization workshop. This involves an assessment workshop with the application owner and system administrator to review and refine the data collection materials. The deliverables of this phase are workshop notes and minutes. (iv) Task4: Development of Master Test Plan. This involves development of the Master Test Plan based on the prioritized list of test cases and the Virtual Service Environment (VSE) architecture. The Application Virtualization Master Test Plan is intended to be integrated as a subset of the actual test plan for the applications in scope. An Application Virtualization Assessment workshop will confirm the Master Test Plan with the Project Manager. The deliverable of this task is the AV Master Test Plan with scheduled applications to virtualize. (v)task5: Assess Test Environment and develop Logical Architecture for the VSE. This task involves assessing the current test environment and develop the Virtual Service environment logical Architecture diagram. An Application Virtualization Assessment workshop will be delivered to confirm the logical architecture for the VSE with Project Manager. This also involves review of the test environment, physical and logical documentation and deployment models relative to the systems that are targeted for virtualization. The prioritized application components are reviewed in relation to the test environment. This task also involves review of test cases to be executed against 24

VSE and review of regression test cases, design VSE model framework and service images and finally develop the AV Logical Architecture Diagram depicting the VSE architecture solution. The deliverables of this task are VSE model framework and AV Logical Architecture Diagram. (vi) Task 6 : Design and Build the Virtual Service Models(VSMs).This tasks involves the designing and building the VSMs of virtualized applications or application components to enable testing of the test cases for the applications and interfaces contained in the AV Strategic Plan that was developed during the AV Startup Phase. This involves a) Reviewing and analysing AV requirements b) Designing the AV Components. This phase may include designing data handlers, designing proprietary protocols and other tasks involved in designing components. c) Capture business transactions. This phase consists of listening to and capturing the characteristics and behaviour of business transactions. d) Model Virtual services to meet business requirements. e) Deploy the VSMs into the designated enterprise test server. f) Execute test cases that exercise the Virtual Service interfaces. Deliverables of this task are the initial VSMs. (vii)task7: Conduct Unit, Integration and Regression Test of VSEs and VSMs with test cases. This task involves conducting the unit, integration and regression tests with prioritized list of test cases using the VSE and VSMs for the interfaces designated by the AV Logical Architecture Diagram. This testing will be used to validate the functionality of VSE and VSMs that have been developed. VSMs will be modified based on test results and environment needs. The AV Completed Test Reports will be developed and will report on completed test executed against VSMs. The deliverables of this task are AV completed test reports and modified VSMs. (viii)task8: Review Test Environment recommendations. This involves review of test environment recommendation based on the VSE and VSMs that have been implemented. The deliverables of this task are the test reports. (ix)task 9: Develop and review Final Report.This involves developing the Application Virtualization final report and review the content of the same with project manager in a closing workshop. The deliverable of this task is AV Project Final Report. c) AV Manage Phase This phase consists of tasks required to manage and sustain the virtual models after they have been implemented. This phase has the following four tasks: (i)task1: Conduct AV Quarterly Planning session. The objective of this task is to review the progress of the AV implementation and plan the implementation for the next quarter and support work. This involves reviewing the completed activities over the last quarter and plan the next quarter s activities. The deliverable of this task is AV Quarterly Planning Report. 25

(ii)task2: Manage the AV Strategic Plan. This involves reviewing the AV Quarterly Planning Report developed in Task1, review additional AV requests, review and assess updates to the AV Strategic Plan as changes occur and present updated AV Strategic Plan to the Project Manager. The deliverable of this task is the updated AV Strategic Plan. (iii)task3: Manage the Virtual Models. The objective of this activity is to perform ongoing management of the virtual models. This involves reviewing requests from application owners on modification to virtual models and update AV Logical Architecture Diagram as changes occur. The deliverable of this task is updated AV Logical Architecture Diagram. (iv)task4: Perform Virtual Models Support. This task involves reviewing Virtual Service Model incidents (e.g. service issues or outages) and resolve the same, modify VSMs as necessary to resolve incidents, version VSMs in the appropriate version control tool, update AV Logical Architecture diagram as changes occur and develop the virtual models support report.the deliverable of this task is Virtual Model support Report. 5. A CASE STUDY OF APPLICATION VIRTUALIZATION PRACTICE The following challenges were being faced by a project and the managers and team members started looking for alternatives: Enterprise applications are highly interdependent High data dependency to perform system testing Interface unavailability at every step either due to Schedule mismatches Blocking code defects Non Intelligent simulators. Simulators had to be developed for each technology. Lots of home grown simulators with low reusability Ever changing interfaces increases the simulation maintenance effort Testers are responsible for ensuring data consistencies across simulators Learning curve higher to learn simulators for each technology Large effort spent on regression testing Application Virtualization AV was predominantly applied during Integration Testing (IT) and System Testing (ST) to break all interface dependencies. Several benefits were observed and shown to customer: Elimination of dependency on interfaces Reduced down times Better data management Single tool used instead of heterogeneous simulators Reduction in effort 26

Improved quality and reduced defect escapes Fig.11 shows the reduction in cycle time by the use of virtualized images. We could achieve cycle time reduction of almost 60% 6. Conclusion 27 Fig. 11: Cycle time reduction by the use of virtual images In this paper we have shown several benefits of customer relationship management technique and application virtualization in software testing. We have seen that using CRM there will be saving of time, cost and effort for the organization. There can be further in this area to showcase some metrics like testing effectiveness, detection efficiency, defect rejection ratio etc. he use automation to improve productivity is an area of further work. One can also work with some new mathematical techniques. The concept of soft computing can also be used if there is gray knowledge and imprecision. References 1. Banerji, S., Project Client Relationship Management - Moving From a "Service Provider" to a "Trusted Partner", 4 th Annual Project Management Leadership Conference, Bangalore, 2008a, p 1-12. 2. Infosys Technologies Limited Sources, www.infosys.com, 2008. 3. Srinivasan, D. & Gopalaswamy, R., Software Testing Principles and Practices, Pearson Education, 2007. 4. Barnes, J.G., Secrets of Customer Relationship Management, McGraw Hill, Blacklick, OH, USA, 2000. 5. Sharp, D.E., Customer Relationship Management Systems Handbook, Auerbach Publishers, Boca Raton, FL, USA, 2002. 6. Bligh, P., CRM Unplugged: Releasing CRM's Strategic Values, John Wiley and Sons, Hoboken, N.J, USA, 2004. 7. Service Virtualization 101, by John Michelsen, itko Chalk Talk, February 1, 2010 8. Service Virtualization for Modern Applications by Gaurish Hattangadi, Virtual Strategy Magazine, November 28, 2010 9. Managing Test Environments by Liz McMillan, Cloud Computing Journal, December 2011

10. Application Behavior Virtualization by Elizabeth White, Cloud Computing Journal, December 2011 11. Database Virtualization For Development and Test by Wayne Ariola, ST & QA Magazine, March 2012 12. An Intro to SOA and Virtualization by John Michelsen, WebServices.org, August 2007 13. The Next Generation of Test Environment Management, Wayne Ariola, Virtualization Journal, July,2011 28