Implementation of Process Performance Models in Application Service Maintenance Projects

Proc. of Int. Conf. on Advances in Computer Science, AETACS Implementation of Process Performance Models in Application Service Maintenance Projects Dr. Basavaraj M.J DELL International Services India Pvt. Ltd, EPIP Phase II, Whitefield Industrial Area, Bangalore-560 066, India basavaraj_mj@dell.com; basavarajmj@hotmail.com Abstract In this era of IT, Process Performance Models helps in estimating, analyzing, and predicting the process performance behavior associated with the processes in the organization s set of process standards. These Models helps the organization to predict, monitor and control metrics parameters. Implementation of Process Performance Models influences better Delivery Management, Productivity improvement and increased in Customer Satisfaction. Process Performance Model is built for Process Y - In Process Defect Density. The aim is to increase the pre shipment delivery defects and to decrease the delivered defects by controlling the critical sub processes. This Model is built by analyzing the six months past data of the Enhancements executed from various Application Maintenance Projects for Medium Enhancements. The challenges with data stratification, segmentation of data, deriving logical meaning for the Process Performance Models by mapping to practical working flow of projects, validation to implement these models are discussed in this paper. Index Terms Process Performance Model, Regression, Correlation I. INTRODUCTION A Process Performance Model [6][7][8] explains description of the relationships among various attributes of a process and its work products those are developed from historical process performance data and calibrated or validated using collected process and product measures from the project and which are used to predict results to be achieved by following a process [1]. These are used to predict the value of a process performance measure from the values of other process. PPMs are useful in monitoring Process Y s Measure by controlling Process X s measure. II. ARRIVE QUALITY AND PROCESS PERFORMANCE OBJECTIVES The organization's Quality and Process Performance Objectives (QPPO) need to address the following attributes [9]: i. Depend on the organization's business objectives ii. Depend on the past performance of projects iii. Defined to process performance measure Organization normally contains Missions, Vision and Goals are as influential parameters for its Performance Measure. Qi where i=1, 2, 3.,..n represents Quantitative Business Objectives and each i to be Elsevier, 2013

measured with relative importance rating j where j=1,3,5 in order of Highest relative importance rating. Process Y Measure = π Q i * j i=1,2,3 n, j =1,3,5 Multiple Process Y s Measure are arrived by : Process Y = π Q i * j k=1,2,3 m QPPOs / Process Objectives / Process Ys Measure at Organization level are arrived by brainstorming with relevant stakeholders, referring to the historical data and aligning with management goals, mission and objectives to focus on : i. To achieve a specified productivity ii. Deliver Enhancements / work packets with no more than a specified number of latent defects iii. Profitable Growth with specified Margin iv. Net Promoter Score with Specified Value v. Customer Operational Excellence Process Objectives are listed in below Fig 1. Fig. 1 Process Objectives Process Y Measures are listed in below Fig 2. In Process Defect Density came as Critical Process Y Measure among the listed in Fig 2 based on rating of influence. This was also supplemented with result by analyzing the past baseline data as well as brainstorming with concerned stakeholders. Fig. 2 Process Y s Measure 573

III. SUB PROCESS XS MEASURE Sub Process Xs data are collected for Process Measure Y(In Process Defect Density) by collating the six months past data of the Enhancements executed from various Application Maintenance Projects for Medium Enhancements. Medium Enhancements are those which requires Estimated Efforts greater than 80 hours and lesser than 496 hours for each delivered Enhancement. This is to adopt the segmentation and stratification to make model more robust and strong for applicability to the scope in the context. Maintenance Projects constitutes Enhancements as one of their functions. Enhancements are adding features to the existing applications. Software Development Life Cycle phases like Requirement Analysis Specification, Design, Coding and Testing and Maintenance which are applicable to Development projects are also applied to Enhancements to maximum extent. Actual efforts spent for the Enhancement is spread across phases like Requirement Preparation, Requirement Review, Requirement Rework, Design Preparation & etc as mentioned in Table 1. In Process Defect Density is defined as Number of In Process Defects / Actual Efforts. One of the triggering point to come out with PPM for In Process Specification data since the analyzed data for In Process Defect density is beyond the baseline specification limits. In Process Defect Density is measured as Defects/Person Hour. Breakup of Actual efforts spent among the phases in terms of efforts (Person Hours) for each Enhancement are listed below Table. 1. TABLE. I PHASE WISE EFFORTS Requirement. Preparation Requirement Review Requirement Rework Design Preparation Design Review Design Rework Coding Code Review Code Rework Unit Test Case Preparation Unit Test Case Review Unit Test Case Rework Unit Test Case Execution System Test Case Preparation System Test Case Review System Test Case Rework System Test Case Execution User Acceptance Testing Breakups of defects among the phases are listed below in Table 2. Phase wise efforts and Phase wise defects listed in Table 1 and Table 2 are considered as Sub Processes for Each Medium Enhancement for the Project. IV. CHALLENGES Accuracy of the data logged is always a very challenging one. Sampling is done to study a representative portion of population. Stratification [3] technique is used to analyze a population of data into homogeneous groups of data collected(sample) for In Process Defect Density. 574

TABLE II. PHASE WISE DEFECTS Review Defects -Fatal Review Defects-Major Review Defects-Minor Testing Defects-Fatal Testing Defects-Major Testing Defects-Minor User Acceptance Testing Defects-Fatal User Acceptance Testing Defects-Major User Acceptance Testing Defects-Minor V. BUIDLING PROCESS PERFORMANCE MODEL In Process Defect Density PPM is built by analyzing the data of all Sub Processes. Minitab tool is used for all stastical tests. Correlation test is conducted among the Sub Processes to know which sub processes are strongly correlated and their impact on In Process Defect Density. Correlation test measures a linear association between two variables with the help of correlation coefficient. The value of correlation coefficient ranges from -1 to 1[2]. i. -1 explains a relationship where an increase in one variable is accompanied by a predictable and consistent decrease in the other in Fig 3. ii. 0 describes there is no relationship in Fig 4. iii. 1 describes a relationship where an increase in one variable is accompanied by a predictable and consistent increase in the other in Fig 5. Fig 3 : -ve Correlation Fig 4 : 0 Correlation Fig 5: +ve Correlation Correlation test results display the Pearson correlation and p-value for each variable. In Fig. 7 example, the Pearson correlation between Design Prep Efforts and Requirement Prep, Efforts is 0.567, between Design Review Efforts and Requirement Prep. Efforts is 0.428 and Design Review Efforts and Design Prep. Efforts is 0.815. Correlation test also displays the p-values for the individual hypothesis tests of the correlations. Appropriateness of rejecting the null hypothesis in a hypothesis test is determined based on p-value. Strongly correlated Sub Processes are selected for those Pearson correlation value >= 0.5 for building into the model. Basic Statistics - Descriptive Statistics test is also conducted and result is displayed below: Fig 6. Descriptive Statistics test results In Fig.6, N represents 112 data points are collected as a sample size for analysis, Mean is 0.017 Defects/ Person hour which is beyond the baseline Lower Specification Limit 0.034 Defects/Person hour and Upper Specification Limit of 0.039 Defects/Person hour, whereas target to achieve is 0.036 Defects / Person hour. Following Sub Process Xs are selected to construct the Model by analyzing the correlation coefficient: Requirement. Prep. Effort Design Prep. Effort Design Review Effort Coding Effort Code Review Effort UTC Prep. Effort 575

UTC Review Effort UTC Execution Effort Fig 7. Correlation test results VI. MULTIPLE REGRESSION Multiples Regression is used to know the relationship between several independent or predictor variables and a dependent or criterion variable. In this PPM Model, In Process Defect Density becomes dependent variable. Requirement Prep. Effort, Design Prep Effort, Design Review Effort, Coding Effort, Code Review Effort, UTC Prep. Effort, UTC Review Effort and UTC Execution Efforts are considered as independent or predictor variables. Regression analysis is to build the model to establish the relationship between an independent variable and one or more predictors which are dependent variables[4][5]. n For the data set { y i, a i 1,, a} i = 1, linear regression model assumes the relationship between the dependent variable y i and the m-vector of regression a i is linear. Error variable of the Model expressed is ɛ i. This adds noise or disturbance to linear relationship between the dependent variables and repressors. Mathematically Model is represented by : 576

Y i = β 1 a i 1 +... + β m a im + ɛ i. i = 1.n Where a i 1, a i 2, a im are referred as independent variables or repressors. β is an m-dimensional factor. Its elements are referred as regression coefficients. Here a i 1, a i 2,, a im represents Sub Processes like Requirement. Preparation Effort, Requirement Review Effort.. Y represents an In Process Defect density. VII. MULTIPLE REGRESSION RESULTS Fig 8. Regression Equation Output from Minitab Residual Plots for Inprocess Defect Density Normal Probability Plot Versus Fits 99.9 Percent 99 90 50 10 1 0.1-0.050-0.025 0.000 0.025 Residual 0.050 Residual 0.050 0.025 0.000-0.025-0.050 0.0 0.1 Fitted Value 0.2 60 Histogram Versus Order Frequency 45 30 15 Residual 0.050 0.025 0.000-0.025 0-0.04-0.02 0.00 0.02 Residual 0.04 0.06-0.050 1 10 20 30 40 50 60 70 80 Observation Order 90 100 110 Fig 9. Residual Plots for In Process Defect Density 577

PPM regression equation : In process Defect Density(Defects / Person Hour) = 0.00436 + 0.000476 Design Review Effort + 0.000597 Code Review Effort - 0.00787 UTC Prep. Effort + 0.0159 UTC Execution Effort VIII. INTERPRETATIONS By referring Fig 8 & Fig 9 interpretations are listed below: i. The p-value in the Analysis of Variance table (0.000) shows that the model estimated by the regression procedure is significant at an a-level of 0.05. ii. The p-values for the estimated coefficients of Design Review Efforts, Code Review Efforts, UTC preparation Efforts and, UTC execution efforts are < 0.005, indicating that they are significantly related to In process Defect density. iii. The R-Sq value indicates that the predictors explain 80.1% of the variance in In process defect density. The adjusted R-Sq is 79.2% which accounts for the number of predictors in the model. Both values indicate that the model fits the data well. iv. Variation Inflation Factor(VIF) -Indicates the extent to which multicollinearity (correlation among predictors) is present in a regression analysis v. VIF values > 10 may indicate multicollinearity is unduly influencing regression results. In this case, reduced multicollinearity by removing unimportant predictors from the model vi. Residuals versus fits. This plot should show a random pattern of residuals on both sides of 0. Also, there should not be any recognizable patterns in the residual plot IX. VALIDATION Model is validated by substituting the Sub Process X s in PPM Equation and compared Model In Process Defect Density value with actual value of In Process Defect Density obtained from the data for some of the Projects. Data collected for next six months period which was not used for earlier model building. Observed results are fine by residual analysis with tolerable error value less than 0.5%. X. FUTURE WORK AND CONCLUSION PPM Model validated at Organization level to be rolled out across to the Projects for the stream Application Maintenance Medium Enhancements. The plan is to monitor the In Process defect density and to control the Critical Sub Process X s with in specified limits which is to be arrived by Simulation. Target is to achieve the Probability of Success greater than 80% for In Process Defect Density. REFERENCES [1] http://www.tutorialspoint.com/cmmi/cmmi-glossary.htm - 1 [2] http://www.minitab.com/en-us/training/tutorials [3] http://www.isixsigma.com/dictionary/stratification/ [4] http://en.wikipedia.org/wiki/regression_analysis [5] T.M Kubaik, Donald W. Benbow The Certtified Six Sigma Black Belt Hand Book, Second Edition, Pearson Publication-2010 [6] JianQiang Li, YuShun Fan, and MengChu Zhou, Performance Modeling and Analysis of Workflow, IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS PART A: SYSTEMS AND HUMANS, VOL. 34, NO. 2, MARCH 2004 [7] Robert W. Stoddard II, Dennis R. Goldenson, Approaches to Process Performance Modeling: A Summary from the SEI Series of Workshops on CMMI High Maturity Measurement and Analysis Software Engineering Institute, Technical Report-2010 [8] Tianying Chen; Bosheng Zhou; Shaomin Xing; Wenjie Luo, A Process Optimization Method Based on Process Performance Models, IEEE, 2009 [9] http://hci-itil.com/cmmi/references/sp_process_4_1_3_performance_select_quality_objectives.html AUTHOR BIOGRAPHY Dr. Basavaraj MJ is currently working in DELL International Services India Pvt. Limited. He had an industry experience of more than 20 Years. He is a having a very good experience in Project Management, Software Estimation, Software Quality, Simulation and Modeling, Data Mining and Image Processing. He has finished his B.E in Computer Science and Engineering from Bapuji Institute of Engineering & Technology Davanagere, Karnataka and his M.Tech in Computer 578

Science and Engineering from Karnataka Regional Engineering college Surathkal, Karnataka and PhD from Computer Engineering Department from National Institute of Technology (NITK)-Surathkal. He is a Six Sigma Black Belt certified and a Certified Software Test Manager. He has published several papers in International Journals/Conferences. His areas of interest include Software Estimation, Requirement Analysis, Project Quality/Quantitative Management, Simulation & Modeling, Data Mining, Six Sigma and Image Processing. 579