Outsource ERP Help Desk support: a customer value approach

Transcription

1 Outsource ERP Help Desk support: a customer value approach By Bianca Pretorius Submitted in partial fulfilment of the requirements for the degree of BACHELORS OF INDUSTRIAL ENGINEERING In the FACULTY OF ENGINEERING, BUILT ENVIRONMENT AND INFORMATION TECHNOLOGY UNIVERSITY OF PRETORIA October 2013

2 Executive Summary Following the implementation of an ERP system, the support of the system is vital for business and system success. Support functions can be handled in-house or outsourced depending on the company needs. The opportunity exists for EOH SAP Services to implement and run a centre of expertise whereby they will outsource ERP support services and offer post-implementation ERP support to customers. This project looks specifically at outsourcing the Help Desk function of an ERP system. The aim of the project was to develop a value proposition to present to customers to promote the new outsourcing service and underline the benefits of outsourcing to customers. Qualitative and quantitative aspects were necessary in order promote the benefits of outsourcing support functions. Qualitative aspects were researched and a quantitative statistical approach was used to determine whether time savings would be a benefit of outsourcing Help Desk support. The Cox Proportional Hazards Model was identified and data from an existing customer s Help Desk was fit to the statistical model. The idea of improving customer relationships by calculating responsive Service Level Agreement times formed and Predicted times would replace standard SLA times and be unique to each problem. Predicted times can provide a measure of performance, not only to the customer in terms of the Outsource Provider s performance, but also to the Help Desk management, in terms of consultant performance, giving the opportunity for two measures to arise. During data validation, it was found that the Cox model will not be feasible in practice as external valuation requires the time of solving to be known, the computing of the predicted time is not possible with available resources. Other methods of forecasting SLA times should be explored as it will remain a value-adding tool for customers and the company alike. The qualitative customer value proposition is based on the principles of a customer-centric strategy and may form a baseline for what the Centre of Expertise should strive to achieve and best practice IT service management approaches are mentioned. i

3 Table of Contents Chapter 1 INTRODUCTION AND BACKGROUND The Company ERP Support Activities Support Methods... 2 Chapter 2 PROBLEM STATEMENT... 4 Chapter 3 PROJECT AIM... 5 Chapter 4 PROJECT APPROACH, SCOPE AND DELIVERABLES Approach Scope Deliverables... 7 Chapter 5 LITERATURE REVIEW: Outsourcing risks from a customer perspective Benefits of outsourcing ERP support The global shift to customer-centric markets Failure and Survival Data Parametric Survival Model The Cox Proportional Hazards model Chapter 6 STATISTICAL MODEL - The Cox Proportional Hazards Function CONCEPTUAL DESIGN ACTUAL DATA ANALYSIS SOLUTION SOLUTION VALIDATION RESULTS Chapter 7 CUSTOMER VALUE PROPOSITION Best practice IT service management approaches ITIL best practice framework ISO standard and Code of Practice Chapter 8 CONCLUSION Quantitative Results Qualitative Results Further Discussion BIBLIOGRAPHY ii

4 List of Tables Table 1: Perceived customer risks of outsourcing... 8 Table 2: Perceived customer benefits of outsourcing IT management... 9 Table 3: Variables to be recorded in data collection Table 4: Description of variables Table 5: Data Analysis Table 6: SLA time allocation List of Figures Figure 1: Cox PH model 'R' output Figure 2: Cox Survival Function Figure 3: Problem Risk and solving time comparison Figure 4: Very High and High Priority Problems Figure 5: Medium Priority Problems - Predicted SLA Times Figure 6: Low Priority Problems - Predicted SLA Times Figure 7: Expected values of an independent dataset Figure 8: Schoenfeld residuals plots Figure 9: Schoenfeld residuals test Figure 10: Framework for understanding how the critical success factors affect the user satisfaction in client organization. (Bairi et al. 2011) Figure 11: Customer Value Proposition iii

5 Chapter 1 INTRODUCTION AND BACKGROUND 1.1 The Company EOH is a technology, consulting and outsource services provider. The company partners with and implements various ERP systems, which are customised to meet customer needs. EOH SAP Services has been implementing SAP solutions in the public and private sector for a number of years, which has granted a wide range of experience. With over 240 SAP consultants and experienced project management teams, they aim at low risk, quick return SAP implementations to meet customer needs. The experience has led to the development of template solutions to enhance the low risk, quick return ideology. The SAP market in South Africa is becoming saturated and while many medium to large businesses may still look at implementing SAP, there are other ERP systems and other companies to choose from. In order to gain customers and encourage strong customer relationships and customer retention, the company should expand in the SAP support area to meet the changing marketplace. 1.2 ERP Support Activities The sustainability aspect in ERP implementation is vital to the success of an ERP system. Most companies who decide to use ERP systems see the implementation or go-live stage as the finish line. Large amounts of time and money are invested in ERP implementation and the cost of maintaining the system is not considered at the same level. The postimplementation costs are usually incurred in the IT support component and according to Deloitte, attributes to 70% of the Total Cost of Ownership (TCO) of an ERP system, over a five year period. These costs are considered to be a fixed cost or necessary expense (2011). Many customers do not attempt to lower the TCO by outsourcing support as they wish to use in-house support to maintain their systems. This may be due to the perception that outsourcing relinquishes control of the ERP system. TCO can be lowered by outsourcing support functions or by a combination approach of in-house and outsourcing of different functions. 1

6 IT and ERP technology is a constantly expanding and merely implementing a system will not add benefit to a company if it is not seen as a long term tool for growth. Acknowledging the importance of maintaining and continuously improving ERP systems is vital for a system to be a success (Goldstein, 2006). ERP Support requires a team of knowledgeable employees with experience in the relevant ERP departments. If current IT employees will be trained for the purpose, this may result in salary increases and a larger scope of the work agreement. It should be considered that placing a larger workload and scope on the IT support employees may be detrimental to the company. An employee now has to focus on ERP support and application development, which may be repetitive, time-consuming tasks in the case of basic level support. According to Beaubouef, an IT ERP Director, IT ERP support teams should look for opportunities to free up their staff to focus higher up the value chain. (2012). ERP Support consists of the following functions: Administration Maintenance Application Support and Help-desk This report will focus on the help desk area of outsourcing. Help Desks provide a central point of contact for users of a system to report any IT problems and a consultant or analyst will attempt to resolve the user problems (Gonza Lez, Giachetti, & Ramirez 2005). Help desks and IT support are expensive to set up and maintain due to necessary hiring or training of employees; staffing forms a large part of ERP support expense. Small and Medium sized businesses may not have the funds or capacity for this and outsourcing is thus required. 1.3 Support Methods Support activities can be managed in-house at the company who have implemented an ERP system. In-house support staff ratios are high, as each ERP department or sector (Finance, Materials Management, etc.) needs an employee knowledgeable in that department. This requires training of existing employees or the employment of skilled ERP consultants. 2

7 Outsourcing requires entrusting the support to an external company (in this case, EOH s centre of expertise). The different support functions can be outsourced independently, but, according to Kahn, outsourcing them all would increase the benefit of utilising an outsource service provider (2011). The need for outsourcing may depend on the size of the business and complexity of the ERP system and may also depend on whether a company is in the public or private sector. Different market segments tend to have different approaches to support with a ratio of inhouse to outsourced functions. There is much debate over the benefits of in-house versus outsourced support but this report focusses on outsourcing, in order to design an effective Centre of Expertise for implementation at EOH. 3

8 Chapter 2 PROBLEM STATEMENT The opportunity exists for EOH SAP Services to implement and run a centre of expertise whereby they will outsource ERP support services and offer post-implementation ERP support to customers. In promoting a centre of expertise to outsource IT help functions, customers will want to see the benefits of outsourcing, specifically those customers who are in the design phase of an ERP system. Implementation of an in-house Help Desk will require planning and allocation of resources and this is not usually the first thing customers look at when implementing ERP; even though maintaining the ERP system is one of the most critical factors to ERP success. If an outsourcing option is readily available, customers can consider ERP support early in the design and implementation phases of an ERP system. A Centre of Expertise may be of financial benefit to both EOH- where the resources needed to run the centre are already in place - and to its existing and future customers. By providing an outsourcing opportunity, the customer can consider the option of outsourcing help-desk or support components with the added benefit of a dedicated and experienced team. The time taken to solve user problems will be a major consideration in help desk environments. The adage time is money is apt in situations where real-time sharing of information is required and down time is costly. Any problems in ERP systems must be addressed as quickly as possible to avoid a delay and possible backlog in the system. The success of the centre of expertise lies in convincing the customers to outsource their help-desk function to EOH. A customer value proposition will be necessary in order to promote outsourcing and address concerns about solving time of problems. Since the centre is not yet in existence, the value proposition may be difficult to create and to carry weight without physical evidence of past performance. 4

9 Chapter 3 PROJECT AIM The opportunity of developing a centre of expertise in EOH SAP Services will introduce a new Business Process Outsourcing opportunity at EOH; namely a Support Centre that allows customers to outsource the support functions required to run and maintain existing ERP systems. The aim is to develop a value proposition to present to customers to promote the new service and underline the benefits of outsourcing ERP support services to customers. Ultimately, the aim is to promote outsourcing. Deducing whether outsourced problems can be solved faster, or whether any benefits - time-related, or otherwise - can be detected in the outsourcing of Help Desk functions will be the foremost outcome. Statistical methods and research will be used to approach the main goal of highlighting customer benefit of outsourcing Help Desk support functions to EOH SAP Services Centre of Expertise. 5

10 Chapter 4 PROJECT APPROACH, SCOPE AND DELIVERABLES 4.1 Approach This report will attempt to build up the value of EOH s centre of expertise through research based on existing help-desks, and through statistical methods. The time taken to solve any Help Desk problem will be analysed at an existing help-desk centre. Using a statistical approach, the difference between in-house solving time and the potential solving time of the new centre of expertise will be predicted. Perceived customer risks and benefits of outsourcing support functions will be researched and will form a qualitative argument for outsourcing. However, a quantitative approach will better appeal to prospective customers. In order to show how outsourcing could benefit a company, a comparative study should be conducted to show whether outsourcing has benefit over an internal help desk. Several factors will influence the processing time of a problem and must be identified; changing these to adapt to a hypothetical outsourcing environment could show the difference between problem-solving time at an external and internal Help Desk. A statistical model may allow for this comparison as the centre of expertise s solving time can be modelled based on the existing data. The program R will be used. R is a free software environment for statistical computing and contributors continually update the system. The program will be explored and the programming language must be learned in order to complete the project. 4.2 Scope The project is management oriented and aims to look at possible new tools for management of ERP Help Desks that have not likely been used. The environment is SAP-specific but the ERP market is large and the benefits of outsourcing to a specific company may be unique. To keep the results consistent and unbiased, basic SAP solutions will be considered, while industry-specific solutions (such as Mining or Engineering, Construction and Operations) may be considered at a later stage but do not fall in the scope of the project at present. At this stage of investigation, the focus will lie on manufacturing-specific SAP systems. 6

11 Research will be discussed in the Literature Review and qualitative findings expressed in the conclusion. All statistical modelling will be done in R. 4.3 Deliverables The findings will be used to promote the centre of expertise, add to the value proposition and support the customer service level agreement, which can be referenced to the statistical output. In future, such a statistical model may be used to monitor the performance of EOH employees to check that they conform to the solving time within given specification limits as well as provide customer value. The project must; Promote outsourcing, Show qualitative outsourcing benefits, Address qualitative outsourcing risks, Give quantitative outsourcing benefits in the form of a statistical model in R, and Give a visual representation of quantitative results. 7

12 Chapter 5 LITERATURE REVIEW: 5.1 Outsourcing risks from a customer perspective Many of the reasons customers choose against outsourcing could possibly be attributed to wanting a high level of control over systems. Various reasons were adapted from a journal article Evaluation of ERP Outsourcing, (Olsen, 2006) and the selected drawbacks were; data security and privacy, vendor dependency, availability, performance, reliability, critical key applications may be necessary to address in-house, and corporate culture that does not deal well with working with partners (Olsen,2006). The drawbacks and concerns are listed and discussed in Table 1. Table 1: Perceived customer risks of outsourcing Data security and privacy Vendor dependency Availability Performance Reliability Outsourcing puts data in the hand of an external company and this opens data up to risks including, but not limited to, data loss or lowered privacy and confidentiality. Customers understand that there is a risk of vendor lock-in if they outsource their support and that they may only be able to utilise vendor-specific services or products. This means that they do not have flexibility in the choice of support offered. Most problems within an ERP system require immediate attention. This is also a valid concern in South Africa as the problem of slow internet speeds and the possibility of communication networks being down is a likely possibility. A support centre offering outsource services will cater to various customers and this may mean that performance could be lower if another company s problem is given preference. This is highly dependent on the staff and resource allocation at the outsource service provider. The outsource service provider must be able to be consistent in delivering a service and the customer wants to feel like they can rely on them as they are relinquishing a large level of control to the provider. Critical Key applications to be done in-house Corporate culture Key applications would largely depend on the type of company, the ERP system, company policy or other factors. The company may wish to have a high level of control over the functions as outsourcing could lead to delays in maintaining or updating and consequently to loss of income. In companies encouraging internal growth, outsourcing may move away from this culture. Other reasons could be that the corporate culture of the customer does not align with that of the outsource service provider. 8

13 The biggest factor customers will consider in deciding to outsource would be the cost of inhouse versus outsourced ERP support. This should not be the only consideration, as solving time, resource utilisation and required staffing ratios are also key elements in ERP support. 5.2 Benefits of outsourcing ERP support The benefits of outsourcing support range from cost to resource utilisation. Customers stand a chance of gaining much by outsourcing even one of the support functions. The most obvious gain would be the fact that the outsource provider has a dedicated team with experience in ERP support. The team will gain knowledge through serving multiple companies and their time and skills will be utilised better than they would if focusing on one stable project where problems and issues may not arise often. The knowledge and experience of such a team will be invaluable as experience is not a skill that can be bought. There are many more benefits to outsourcing; Olsen reports that outsourcing may improve flexibility and replace obsolete systems. Furthermore, access to expertise that a company cannot afford to maintain or gain in-house is a big advantage, specifically to small or medium businesses. Capital expenditure may be reduced as the need for overheads related to support and the added expense of support staff salaries are lowered (Olsen, 2006). When outsourcing, support functions become monthly operational expenses rather than capital expenditure (Muldoon, 2011) The benefits given in Table 2 were outlined by Putra (2011). Table 2: Perceived customer benefits of outsourcing IT management Cost Reduction Access to Specialist Resources Improved Focus Better Risk Management Outsourcing can reduce the TCO of an ERP system as well as free additional finances for use in business areas with strategic values. As mentioned, the skills of a dedicated team will grow with experience gained from other projects and refine the specialisation of outsource providers By outsourcing IT functions that a company does not have in-depth knowledge of, to one that specialises in IT support, they free their resources and are able to focus on the core business processes and can focus on developing their own level of expertise in their own scope of business. An outsource service provider will have an in-depth knowledge of risks to ERP systems and can easily identify and rectify area where risks may become problematic. 9

14 A study including 1335 industry stakeholders was completed by HfS Research and the London School of Economics Outsourcing Unit on the state of outsourcing in Findings showed that over 95% of the current buyers saw cost savings as the most relevant benefit gained by outsourcing. Beyond savings, most service providers met the regulatory requirements but other benefits were not as visible to customers (HfS Research, 2011a). Buyers have concerns about outsourcing that must be overcome and addressed before entering into a relationship, hence the importance of a Customer Value Proposition to highlight the Service Provider s capabilities. Throughout industries, the focus of promoting products has shifted from a product-driven to customer-centric approach. Customers seeking Outsource Providers can be seen to align to this trend by their perceptions of critical attributes they expect when selecting candidates. Among these attributes are financial stability and the ability to support change management and governance, which is an attribute many providers did not consider as important as their customers did (HfS Research, 2011b). In an environment where customers prefer to base outsourcing decisions on peer-reviews (HfS Research, 2011c), service providers will benefit greatly from utilising a customer-centric approach to business by building strong customer relationships and listing business value above cost-savings in terms of the promotion of their services. Customer-Provider communication will be a key component in achieving a customer-centric business model and Service Level Agreements must be responsive. In ERP environments, one faces a realtime, integrative environment so in a help-desk scenario, SLA times that are adapted to each problem and more accurate than a standard SLA time could benefit the Customer-Provider relationship. Time savings is not a listed benefit, and this project will aim at finding out whether it can be added to the list of benefits. 5.3 The global shift to customer-centric markets In 2010, the Harvard Business Review included articles explaining the shift of a marketing focus from product- to customer-centric (Bhalla et al.) and the shift from measuring business success by shareholder value to measuring it by customer-value (Martin). In a service environment, keeping abreast of the shift is vital to success. In the case of a service provider, success will mean customer satisfaction and loyalty. This research topic will not be 10

15 expanded because it is only important to understand that it plays a large role in shaping a customer value proposition. 5.4 Failure and Survival Data In a statistical problem that considers the time to an event, the assessment of product reliability could be a good approach to assess solving time. By looking at the online Engineering Statistics Handbook (NIST/SEMATECH 2012), lifetime distribution models used for non-repairable populations were identified. These can possibly be used by replacing the time to failure by the time taken to solve a Help Desk problem. Due to the fact that there are several factors affecting the solving time, a multivariate statistical approach would be a good approach as it will assess the influence of variables on the solving time. As mentioned in the scope, the statistical analysis program R will be used. A survival analysis package has been developed by contributors to the open source program to allow for the use of several survival-analysis statistical models in R (R-project 2013). Parametric Survival Model In the case of parametric models, the time to failure of a set of data is assumed to follow a known distribution. Common distributions used in parametric survival analysis include Weibull, Exponential, Log-logistic and Lognormal. If the data fits well to these distributions, a parametric model can be used to estimate the regression coefficients. (Kleinbaum& Klein, 2012) The Cox Proportional Hazards model The Cox Model is a multivariate survival analysis model. While more often used in survival analysis and medical testing than in engineering practice, the Cox proportional hazards model considers the impact of several factors on survival and ultimately determines the survival time, or in the case of assessing product reliability in engineering application, the failure rate of non-repairable populations (NIST/SEMATECH 2012). In the help desk scenario, this time to failure will be defined as the solving time. 11

16 The Proportional Hazards Model is semi-parametric, as there are no assumptions about the shape of the baseline hazard function. Proportional Hazards (PH), however, are an assumption (Boston University School of Public Health 2013). The assumption translates to the fact that the hazard ratio remains constant with time. (Fronza et al. 2011) In order to apply proportional hazards regression analysis, data that indicates the amount of time it has taken to solve a problem and various other explanatory variables that contributed to this time will be required. The variables may be continuous or indicator variables (NIST/SEMATECH 2012) and must be independent of time. The Cox Proportional Hazards model h(t,x) = h 0 (t) exp (b 1 X 1 + b 2 X b p X p ) h(t,x) expected hazard at time t given an indicator variable x h 0 (t) baseline hazard (X 1, X 2 X p) predictors (or independent variables) (b 1, b 2,..., b p ) regression coefficients (Boston University School of Public Health 2013) The expected hazard at time t will be the time of solving of a Help Desk problem. In the Cox PH model, no assumptions are made about the form of the baseline hazard and it is not assumed to be constant (Kleinbaum& Klein, 2012). When fitting data to the Cox PH model, the output will be the regression coefficients corresponding to the independent variables. Regression coefficients indicate the proportional change these variables have on the hazard rate when they are adjusted (Walters 2009). Due to the complexity of the model, it must be fit using a computer program (Walters 2009). The survival package in R will be used, and given the time to failure and an indicator (i.e. whether the product has failed or not) from a set of data, the program will return the regression coefficients for each variable. The Cox model will be useful to analyse existing data and adjusted variables to coincide with expected centre of expertise performance as discussed in the project approach. 12

17 Chapter 6 STATISTICAL MODEL - The Cox Proportional Hazards Function 6.1 CONCEPTUAL DESIGN Data collection To test whether the Cox Proportional Hazards model would work under the assumptions, some fictitious data was created, and the variables were filled in with predicted values based on basic help desk solution observation and knowledge. 20 random samples were created as test data. This data includes the following variables: Table 3: Variables to be recorded in data collection MANUFACTURING problem type accounting procurement sales production costing problem level software ABAP config user error hardware network user level low med high consultant experience trained employed consultant area of expertise? no yes problem referred? no yes method phone web time (0-inf) solved? no Table 4: Description of variables yes Variable problem type problem level user level consultant experience consultant area of expertise? problem referred? method time Description Which scope of the ERP system does the problem fall under? On what level was the problem that occurred? What logon abilities did the user who reported the problem have? Did the company train an existing employee or employ a skilled consultant? Does the problem type fall under the consultant's area of expertise? Did the consultant have to refer the problem (if they could not resolve it)? How did the user contact the help desk? How long did it take to solve the problem? Data Analysis The test data was fit to the Cox Proportional Hazard function in the program R. The statistical program was researched and a survival model plugin found to fit data to the Cox model. The model requires that the user identify the time and an event indicator in a set of data. In this case, an extra variable must be introduced to specify that the problem is solved. This is the event indicator and shows that all the individuals in the population being considered fail in the survival analysis (similar to non-repairable populations). The sum of the remaining variables will be the variables that affect the solving time. 13

18 Findings The coefficient of determination (R 2 ) was which showed that the data fit the model with 56% accuracy. The results returned the coefficients of each variable, which shows on what level they affect the solving time.the results identified that the most significant of these were the problem level, whether the problem was in the consultant s area of expertise and the consultant s experience (whether trained or employed). The results are promising given that fictitious data was used, once the actual data is fit to the model, a more accurate representation of the fit and the most relevant variables can be returned. The Cox model may be a valuable tool in assessing the solving time of a proposed centre of expertise by adjusting the variables and substituting them into the hazard function and seeing how long it would take to solve the same problem at the centre. In the case of a centre of expertise, the problems will always go to a consultant who is experienced in the problem type and level, which will according to the coefficients, result in a shorter solving time. The model can also become a future tool for control. If enough relevant data is collected, consultants and problem types can be monitored and compared to the standard time to solve a specific type of problem. This is a way to locate problem areas or see that consultants are performing on target with their expected performance. 6.2 ACTUAL DATA ANALYSIS The actual data collected had different variables to the proposed design, but the concept remained the same. Out of 60 possible variable fields, the detailed descriptions were eliminated, and possible factors that may influence solving time were identified. For each of the variables, the possible values had to be converted to numbers so as to be able to fit the data into the statistical model. The following tables show the variables and description corresponding to the translated numerical values. 14

19 Table 5: Data Analysis Variable Ticket Type I P R Customer A B C D E F G H I J Department Ticket WEB Created Via Ticket Very High High Med Low Other Priority Ticket No Yes Resolved Ticket SLA No Yes Violation Variable Category APO Basis BW CO FI HR Mast MM PM Portal PP Pricin Scann SD WM XI er Data g ing Symptom Error Out Requ Slow Unabl Unabl Mess of est e to e to age Scope Open Print Root Cause Acces s BPS Break fix CBI Chan ge Order Dupli cate Requ est Lack of User Respo nse Perfo rman ce Issues Requ est Cance lled Techn ical After the variables had been identified, the data had to be filtered to include SAP-specific problems only, only problems that have been resolved, and the Category, Symptom and Root Cause variables had to be broken down and the level of each also had to fall under SAP. During testing, it was found that the actual data had an R-square value of when fit to the Cox PH model, which is not as satisfactory as hoped. Various multivariate failure rate data was explored to see if the data could fit to another model, specifically parametric models as they were considered in the early stages of the project. The solving times were fit to the various possible distributions that are commonly used in parametric survival analysis. R s survival analysis tool fits the data to survival distributions but the data did not fit well to any of the distributions, with chi-squared values in the hundreds and up, thus parametric models were not considered further as an alternative approach for estimating the solving times. 15

20 This led to considering the data itself. The data is from an in-house help-desk that is in operation at one of EOH s customers. The guidelines for defining the variables were explored. Firstly, the Priority of a problem should be addressed, and secondly the Service Level Agreement Time, as a Service Level Agreement Violation had a large effect on the solving time, as identified when testing the model as can be seen in the R output (Figure 1). The guidelines are discussed below; Ticket priority - classification rules 1) Very High Very serious consequences for normal business transactions and urgent work cannot be performed by the End User. Requires immediate processing as this type of malfunction can cause serious losses. Possible causes include: Absolute loss or in-operation of the SAP system; Malfunctions of the central SAP functions in the production system of the End User; Delays to the planned production start-up or upgrade within the next 2 business days. 2) High Normal business transactions are seriously affected and necessary tasks cannot be performed. This is caused by incorrect or inoperable functions in the SAP system necessary in the actual situation. The problem message requires immediate processing because the malfunction can seriously disrupt the entire productive business flow. 3) Medium Normal business transactions are affected. The problem is caused by incorrect or inoperable functions in the SAP system. 4) Low Low priority issues have little or no effect on normal business transactions. The problem is caused by incorrect or inoperable functions in the SAP system that are not required daily, or which are rarely used. 16

21 5) Other The Priorities included another problem level, with a priority of 5 that was classified by the user as other. The data was included in the model, but the service Level Agreement Time was not stipulated, and was thus considered to be the same as that of a Low Priority Problem. Allocation of Service Level Agreement Times: For priority 1 very high incidents, the time is measured in real time, regardless of Local Office Times. For Incidents with any other priority, the time is measured in working hours during Partner s normal business hours. Table 6: SLA time allocation Priority Severity Response Time Resolution Time Very High (1) Emergency: 1 hour (24h x 7) 4 hours (24h x 7) High (2) Critical: 4 hours (business hours) Mon-Fri 2 business days. Medium (3) Non-Critical: 8 hours (business hours) Mon-Fri 4 business days. Low (4) Minor: 16 hours (business hours) Mon-Fri 8 business days. Out of the 3643 resolved problems (excluding the problems classified as Other ), only 2 were of a High Priority. This, along with the extra problems with priority classified as other, may mean that problem priority classification guidelines should be re-addressed or classification rules be made more clear to help desk consultants correctly define the priority of a problem. Even though these concerns were raised, it was decided that the data would be used as-is in order to identify if the Cox Proportional Hazards Model holds promise in a help desk environment where certain classification rules exist and are most likely complied to. Using the existing data allowed exploration of combining and excluding variables, limiting the data by priority or time and to become familiar with the statistical programming language and functions used in the program, R. 17

22 In terms of the centre of expertise, actual data will need to be collected from the new outsourced Help Desk to fit a completely separate Cox PH model, but the existing data is being used only to test whether the Cox model could identify and magnify any customer benefits in a Help Desk environment that is outsourced. 6.3 SOLUTION Several models were considered but the Cox Model was chosen because of its broad application and semi-parametric nature. The solution looks at using the predicted Cox PH model analysis to better predict SLA times and in that way, add value to customers by providing a more responsive expected solving time than a standard SLA time. After fitting the data to the Cox PH model in R, the following output was obtained: Figure 1: Cox PH model 'R' output While the R squared value was lower than expected, the output was still used to observe the Cox model. All the variables mentioned in the Data Analysis were used, but the output in Figure 1 shows only the variables that have the highest effect on the solving time, or the highly significant statistical coefficients, identified by the z-value (Fox, 2002). The coefficients for each variable were returned with negative coefficients indicating that a 18

23 variable increases the solving time and positive coefficients decreasing solving time; this is attributed to positive variables having a positive effect in the case of survival where a longer survival time is beneficial. In our case, a shorter survival time is favoured. SLA violations have the largest effect on solving time and Figure 2: Cox Survival Function Figure 2 shows the survival function, namely the probability that any subject will still be unsolved at a specific time, with a 95% confidence interval. The survival can be explained as follows, half a day after any specific problem has arrived, the likelihood that said problem is still unsolved is about 60% and after 2 days there is a 20% chance that this problem is still not resolved. It is worth noting that the probability of still being unsolved after 8 days is close to 0% thus the standard SLA times of even low priority problems may be too high. The predicted risk for each problem in the dataset was calculated and can be seen in Figure 3. In our scenario, the risk is translated as the likelihood of solving a specific problem relative to the entire sample. On average, higher likelihoods corresponded with shorter solving times and vice versa. This is particularly evident in the case of SLA time violations. Since the risk measure is related specifically to the original sample fit to the Cox model (R- Project, n.d.), the likelihood of independent data from the help desk producing similar results during validation must still be determined. 19

24 Figure 3: Problem Risk and solving time comparison The variation between real solving time and the predicted Cox PH times is difficult to solve directly. During early stages of the project, the expected predictor was identified to return predicted Cox times, however, research indicated that the expected value returned by the predict function is the expected number of events per individual unit of time (Fox 2006). In other words, the expect prediction returns the value of the cumulative hazard at a given time for a problem with specified covariates. In cases where the probability is higher, the problem has, in most probability, been solved and the probability could even represent the number of times the same problem could have been solved in the given amount of time. It must be noted that this was determined through observation and may not be a completely accurate translation of the expected; variable. In order to calculate an expected value, a data set requires all the covariate as well as survival time values for each subject. This makes the prediction of future values unlikely and forecasted solving times will not be able to be forecasted in this way. A question to ponder is- if the solving times are known, could the expected value have been a better indicator of SLA time or a measure of consultant performance? For the different priority levels (Very High - Low), the expected values were compared to the standard SLA times stipulated by the Help Desk seen in Table 6. The graphs that follow are divided by priority to compare the Actual, Cox and SLA times per problem. The graphs exclude the SLA violations and these values are not used to adjust the SLA times, as it is 20

25 assumed that in the EOH Centre of Expertise, SLA violations will be minimised and avoided at all costs. However, the SLA violations are important to consider when fitting the data to the Cox PH model to get an accurate representation of the fit. The ideal of achieving no standard SLA violations is possible to achieve as there are many capable consultants available at the proposed centre of expertise to address a large number and wide variety of problems at once. When a new problem arrives at the Help Desk and its data is fit to the Cox model to determine a Predicted Cox time, the Ticket.Resolved indicator will be set to 1 (indicating that the ticket is solved). The problem may not be resolved yet, but our assumption is necessary to return the required hazard (the Predicted Cox Time ). The next assumption is that the SLA Violation indicator will be set to 0 (meaning no standard SLA violation has occurred when solving the problem). This does not, however, mean that the actual time to solve a problem will always be lower than a newly Predicted SLA time, it is only necessary in terms of the Cox PH model hazard output Very High and High Priority Help Desk Problems The first graph (Figure 2) shows the SLA time for Very High and High level priority problems. It is apparent that the actual solving time of a problem is close to the standard SLA time. It is thus wise to treat such problems with a standard, quick-response SLA time, as these problems are vital to the functioning of the entire ERP system or specific business functions in the system and must be solved as soon as possible to avoid down time and allow for further business transactions. The expected value does not show much benefit in this case. Figure 4: Very High and High Priority Problems 21

26 6.3.2 Medium Priority Help Desk Problems The Medium Priority Problems graph (Figure 3) shows that the expected Cox PH value had a closer fit to the actual solving times than the High and Very High Priority Problems. On average, the expected value was higher than that of the actual time. If this Predicted Cox value had been used as an SLA time, theoretically very few SLA violations would have occurred. The actual time was subtracted from the predicted time for the 993 Medium priority problems that were solved within standard SLA time. This observation did not extend to the SLA violations, as considering these would have affected the observations that follow negatively. There were 49 instances out of the 993 problems where the solving time took longer than the predicted Cox PH time with an average time overrun of days. If testing were possible, any new problems could be entered into the Cox model in R as they arise and the output given in the form of the hazard would be the Predicted SLA time. Figure 5: Medium Priority Problems - Predicted SLA Times Low Priority Help Desk Problems For the Low Priority Problems in Figure 5, the same approach was used. Actual solving times were subtracted from the Predicted Cox times and the average difference was , meaning that the Actual time to solve a problem was generally higher than that of the Predicted Cox time. Once again, these Predicted SLA Violations (values where the Actual time exceeds the Predicted Cox time) were analysed and out of 2215 values, 772 violations were detected, which amounts to 34.85% of the Low Priority Problem population. The 22

27 average overrun time in problems where the Predicted SLA time was violated amounted to days. If the Predicted SLA times were used as new SLA times, a 38.6% violation rate would likely not be classified under satisfactory performance by customers. Figure 6: Low Priority Problems - Predicted SLA Times Summary There is a possibility that Medium and Low Priority Problems can be fit to the Cox PH model but only as a post-solving measure of consultant performance. The predicted hazard returned by the model can produce a SLA time that is unique to the problem. High and Very High Priority Problems will be measured by the existing standard SLA times and Problems classified with the priority other must be further expanded and the classification of these should be discussed with the current Help Desk to determine the cause of the unlisted classification. This is necessary in order to determine whether a standard SLA does exist for the problem type, or whether either the standard SLA for Low priority problems must be used or priority classification must be reclassified and standardised across the help desk. Testing the feasibility of using calculated expected SLA times will be done by using separate data collected from the Help Desk which will be fit to the Cox PH model in R and adjusted to find the Predicted SLA Value. The problem solving times of any new data used in testing will eventually be required to compare the Actual Time and the Predicted SLA Time and see whether the use of the Cox model could work as a SLA determination tool. 6.4 SOLUTION VALIDATION RESULTS External validation - Testing with new data The idea of predicting time would work but only if the solving time is available. In Figure 7, a second dataset collected from the same help desk was fit to the original model. The 23

28 expected values show promise as SLA times but the program requires the solving time to return this value. The tool will not work as a prediction of SLA times, but could be used to measure past performance. Figure 7: Expected values of an independent dataset Check Proportional Hazards Assumption The assumption of proportional hazards (PH) is important to model validation. Hazards must remain constant with time to meet the assumption. The smooth scaled Schoenfeld residuals plots can be used to check the PH assumption these plots are shown in Figure 8. The test will check each variable and if there are visible trends present over time, the PH assumption may be rejected. Generally data that violates the assumption is time-related (Therneau 2000). None of the covariates used are linked with time but may still violate the assumption. As can be seen in graph, most of the covariates do show slight trends over time. RootCause, Ticket.SLA.Violation and Ticket.Support.Group all show an increasing trend after a number of days. The trend is not large but the fact that it is present may mean that the PH assumption must be rejected. To investigate further, the Schoenfeld residuals test was also done to get a summary of the above that may be simpler to interpret. The results are shown in Figure 9. 24

29 Figure 8: Schoenfeld residuals plots Figure 9: Schoenfeld residuals test A large global chisq value shows high likelihood of non-proportionality. P values lower than 0.05 also indicate that PH is not satisfied (Klein 2005). It thus follows that the only PH covariate is Ticket.Priority. Reasons for non-proportionality may be outliers in the data, 25