Impact of Knowledge Repository Use on the Performance of Technical Support Tasks (In Progress)

Transcription

1 Impact of Knowledge Repository Use on the Performance of Technical Support Tasks (In Progress) Introduction Mani Subramani*, Mihir Wagle*, Gautam Ray*, Vallabh Sambamurthy+ *University of Minnesota +Michigan State University How Information Technology (IT) investments create value is an enduring question in Information Systems research. The large body of work examining the relationship between IT and performance outcomes reflects three approaches: a) examinations of the firm and industry level effect of IT investments on productivity, b) a process level approach to articulate the path from IT investments to firm level outcomes through intermediate benefits created by IT in specific processes and c) the articulation of contingencies influencing the relationship between IT investments and outcomes. IT and Firm level Productivity: This stream of research (e. g. Brynjolfsson and Hitt 1996, Dewan, Steven and Min 1998) has been influential in providing empirical evidence of the productivity benefits of IT. The significance of these studies lies in their use of the production function framework and in suggesting that investments in IT capital and IT labor should be viewed as factors of production in addition to non-it capital and non-it labor. The work of Dewan and Min (1997) provides a clearer articulation of the specific role of IT by relaxing the constant elasticity of substitution assumption in the Cobb Douglas production function. They argue that more flexible forms of the production function such as the Translog and the CES-Translog may be more appropriate to capture the influence of IT Capital. Dewan and Min find that IT capital is a net substitute for both ordinary capital and labor. More recently, Chwelos et. al (2010) suggest that IT substitutes for other factor inputs like ordinary capital and labor in earlier periods (i.e., in the1990s) periods while IT is a complement to other factor inputs in later periods. IT and Firm processes: In contrast to the impact of IT on firm level productivity, these studies have specifically focused on the changes in business processes facilitated by IT, and have opened the black box to examine IT related changes to business process and transactions. Studies adopting this approach have found positive effects of IT accruing to firms through changes to business processes and transactions and have provided a more granular tracing of the intermediate effects of IT investments. For instance, ITenabled processes have been examined in mail sorting at the U.S. Postal Service (Mukhopadhyay et al. 1997), and order-processing in supply chains (Mukhopadhyay and Kekre 2002). Recent work further extends this approach to provide a more nuanced articulation of the effects of technology deployment on business processes and firm outcomes. Kumar and Telang (2011) examine the impact of providing a web based self-service channel on customers use of the conventional but costly phone based customer service interactions. Their findings suggest that such IT investments are substitutes for the labor intensive phone based customer service only in the case when customers need information that is unambiguous and easily structured. However, these IT investments have the opposite effect of increasing rather than decreasing the use of the phone based customer service channel in instances when customers faced ambiguous situations. Using panel data from a Japanese bank, Maggio and Alstyne (2011) study the information sharing activities in a bank. They find that low skill agents gain productivity benefits by acquiring information (i.e., answers) from high skill agents; whereas high skill agents get promotions by providing information (i.e., answers) to low skill agents. Aral, Brynjolfsson and Van Alstyne (2012) link the productivity of recruiters to the variety of knowledge that they can access through their information networks enabled by information technologies.

2 While these studies articulate the value created by the enhancement of processes and the ability to support novel ways of working, there has been little prior research on whether IT substitutes or complements other process level resources such as human capital. How IT investment influences knowledge work whether IT substitutes for or whether it is a complement for other process level resources is an important theoretical issue with considerable practical significance. We build on the work of Kumar and Telang (2011) by examining the role of IT in the setting of a firm s customer service process where IT can either serve as a substitute or as a complement for labor to enhance the productivity of labor inputs. Data and Empirical Context Archival data on field support calls were collected from the North American division of a large industrial manufacturing firm selling and servicing commercial HVAC systems. Warranty and ongoing maintenance services to US customers are provided by field technicians in office locations distributed throughout the 50 states. Field technicians are aided in their tasks by a Technical Support group located at the firm s headquarters location. To support information access regarding equipment and trouble shooting by field technicians, the Knowledge Management group in the firm had implemented a document repository. This repository contained engineering documents as well as the documentation of commonly occurring problems and their solutions authored by technical support engineers. This knowledge repository was also accessible to field technicians over the web. All field support technicians carried company issued cell phones in case they needed to call Tech Support and were also provided with ruggedized laptops with wireless cards that they could use to access the knowledge repository. The knowledge repository was populated with documents submitted by technical support engineers. Many of the calls from field technicians involved complex issues not covered in the repair manuals carried by field technicians. Technical support engineers taking calls from the field technician often improvised solutions that involved considerable knowledge, experience and ingenuity. The goal of the firm was to have such solutions documented so that others field technicians facing the same or similar problem could potentially reuse the solution. Document submissions were voluntary and not compensated but support engineers were encouraged to document any solutions and novel approaches in solving field problems and submit them for inclusion in the document repository. The document repository was continuously growing with new documents being added to the repository every week. Field technicians were encouraged to look up solutions in self-service mode before calling the support group. Details of all calls to the technical support group were documented in a ticket management system. We were provided access to the tickets created by technical support engineers, which had name of the caller, details of the equipment, the nature of the problem and the solution that the support engineer had suggested. The support engineer also recorded the nature of the call. Product information calls were considered the simpler and trouble-shooting calls were considered the most complex. The record also included the duration of the call. All accesses of the knowledge repository by field technicians were logged and we had access to these logs with records of the beginning and end times of sessions, queries used for searching, documents retrieved in each query as well as a record of the documents that were opened for examination. We had detailed data on all tickets as well as all searches performed in the 25 month period from 9/2008 to 8/2011. At the end of the period the repository contained knowledge documents. On a typical day there are 59 tech support engineers available to take calls. On average each tech support engineer takes 12 calls per day, where each call lasts about 10 minutes.

3 Empirical Analysis The analytical framework used in the empirical analysis is a production function where the output (the dependent variable of interest) is the average time taken by technical support engineers to handle a call and the number of calls handled by them each day. The inputs that influence the production process are the labor hours logged in by technical support engineer on a given day and the number of knowledge documents available on that day. We also collected data on monthly service revenue, and the number of field service agents in the field. The field support headcount and the revenue from service contracts directly reflect the volume of field service incidents that influences the number of calls to the technical support center. We therefore use these variables as controls in our specification. The basic analytical framework is represented as: Output ij = + 1 ln K j + 2 ln L ij + Controls + ij Output ij refers to the average response time of calls, or the number of calls handled by technical support engineer i on day j. K j refers to the knowledge capital (number of documents in the repository on day j) and L ij refers to the number of available hours of technical support engineer i on day j. Model 1 Duration of Call Model 2 Duration of Call Model 3 Duration of Call Ln_L *** (0.033) (0.09) (0.354) Ln_K *** (0.155) *** (0.409) *** (1.015) Ln_headcount (0.746) (2.077) (2.068) Ln_sales (0.085) (0.237) (0.237) Complexity *** (0.018) *** (2.319) Complexity_ln_L (0.079) Complexity_ln_K *** (0.223) Lambda 0.304** (0.149) Constant *** (6.983) Table 1: Individual Transaction Level Analysis (0.804) ** (19.266) (0.798) *** (21.251) We want to examine the impact of the number of knowledge documents on the response time to handle a call. However, a field technician may access the knowledge repository before deciding to (or not) to call the help desk. Thus, we need to correct for self-selection and unobserved heterogeneity among individuals calling the help desk. Since we had information on accesses by field technicians on their repository accesses as well as their calls to the technical support engineers, we were able to distinguish cases in which the caller (i.e., the field technician) had accessed the repository prior to calling (or not calling) the helpdesk. We estimate a probability of calling the helpdesk using the number of knowledge documents in

4 the repository and the designation of the field technician. Then we incorporate the inverse Mills ratio in examining the impact of the number of knowledge documents on the time to respond to a call. Selection Model: Call_Made = f (Ln_K, Designation) Performance Model: Duration = f (Ln_L, Ln_K, Complexity, Ln_L x Complexity, Ln_K x Complexity,, Controls) Table 1 presents the analysis. In model 1, the self-selection term is positive and significant. However, when we include call complexity in model 2, the self selection term is not significant but the coefficient of call complexity is positive and significant. This suggests that if a field technician calls the help desk, it is a more complex call. The main effect of the number of knowledge documents is negative and significant in all the three models. This suggests that the time to answer a call goes down as the number of repository documents increases. The interaction of the complexity and the number of knowledge documents has a positive and significant impact, suggesting that for a given level of number of knowledge documents, a more complex call takes longer. However, the total impact of the number of knowledge documents on the response time is ( *Complexity). This suggests that the number of knowledge documents reduces response time, though more complex calls take more time to respond. Difference between no-access and before-access calls: To further examine the impact of the knowledge repository, we distinguish between calls answered by a helpdesk engineer where the field technician had accessed the knowledge repository before making the call (referred to as before-access call) and compare the performance of those calls with the calls responded by the same help desk engineer on the same day where the caller (i.e., the field technician) had not accessed the knowledge repository before calling (referred to as no-access call). This analysis is in the spirit of Difference-in-Difference (DID) analysis. When we consider the complexity of the call, we examine for a specific help desk engineer, on a specific day, the difference in time to respond to a no-access and a before-access call of the same complexity. We define two new variables, Ln_Difference_Count = Ln (No-Access Count) Ln (Before-Access Count) Ln_Difference_Duration = Ln (No-Access Average Duration) Ln (Before-Access Average Duration) We then examine the relationship between the inputs: number of knowledge documents and labor hours, and the performance outcomes, Ln_Difference_Count and Ln_Difference_Duration. This is a fixed effects model. Table 2 presents this analysis. We used robust standard errors to control for heteroscedasticity. The number of knowledge documents has a positive impact on the difference in the number of calls taken with no-access compared to the number of calls with access to the knowledge repository before calling. This suggests that the repository increases the help desk engineers ability to take more calls with no -access to the repository compared to calls that had prior access to the repository before calling. However, the interaction between complexity of the call and the number of documents in the knowledge repository has a negative impact on the difference in the number of calls taken. This suggests that the for a given number of knowledge documents, the number of calls taken increases for a more complex calls when the field technician had accessed the knowledge repository before calling the help desk. The total impact of the number of documents in the knowledge repository on the difference in number of calls taken is *complexity. This suggests that the help desk engineer takes more

5 calls if the field technician had not accessed the repository before calling, though this difference decreases for more complex calls. The analysis in table 2 also indicates that the number of knowledge documents has a positive impact on the difference in the average duration to respond to a call with no-access compared to the call with access to the knowledge repository before calling. This suggests that the number of knowledge documents in the repository reduces the time for a before-access call more compared to a no-access call. However, the interaction between complexity of the call and the number of documents in the knowledge repository has a negative impact on the difference in duration in responding to the call. This suggests that the for a given number of knowledge documents, the time to respond to a call increases for a more complex call when the field technician had accessed the knowledge repository before calling the help desk. However, the total impact of the number of documents in the knowledge repository on the difference in duration to respond to the call is *complexity. This suggests that the time to respond to a call is lower if the field technician had accessed the repository before calling, though this difference reduces for more complex calls. count of tickets count of tickets duration of call duration of call Ln_L 0.272*** (0.059) 0.233** (0.105) (0.046) (0.104) Ln_K 0.49*** (0.17) 0.713*** (0.134) (0.081) 0.333* (0.168) Ln_headcount (0.834) (0.536) ** (0.583) (0.750) Ln_sales *** (0.124) *** (0.076) (0.079) (0.082) Complexity *** (0.487) ** (0.539) Complexity_ln_L (0.03) (0.029) Complexity_ln_K *** (0.037) ** (0.043) Constant (5.754) (4.546) 9.111** (4.676) (5.572) Table 2: Comparing calls with No-Access and Access-Before calling the help desk engineer. Substitution vs. Complementarity: To further study the impact of repository access we compared calls with no prior access and calls with access to the repository before calling, in terms of how the relationship between labor (i.e., labor hours) and IT/knowledge capital (i.e., number of knowledge documents in the repository) differs for the two types of calls. It is likely that when the field technician had not accessed the repository before calling, the tech support engineer can use the knowledge repository to answer the call. In other words, when there is no access to the repository before calling, there is a substitution relationship between the tech support engineer s labor hours and the number of documents in the knowledge repository. However, when the field technician has accessed the knowledge repository before calling, the tech support engineer has to use his/her specialized knowledge and time, and the knowledge in the repository together to respond to the call. In other words, in the case of prior access to the knowledge

6 repository there is a complementary relationship between the tech support engineers labor hours and the number of documents in the knowledge repository. The Cobb-Douglas production function assumes that there is perfect substitution between labor and capital. However, the Translog and the CES Translog production functions present more general production functions. The Translog and CES Translog production functions are presented below. The CES Translog production function is estimated using non-linear regression. Translog: Ln Duration ij = β 1 * ln K ij + β 2 * ln L ij + β 3 * ln K ij * ln L ij + β 4 * (ln K ij ) 2 + β 5 * (ln L ij ) 2 + controls + ε ij CES-Translog: Ln Duration ij = α (1/ρ) * ln[delta * K ij -ρ + (1 delta) * L ij -ρ ] + β 3 * ln K ij * ln L ij + β 4 * (ln K ij ) 2 + β 5 * (ln L ij ) 2 + controls + ε ij Allen elasticity of substitution (AES) is calculated as AES ij = (Σf i * x i ) * H ij / (x i * x j * H ) Where f i = marginal product of input i, H = the determinant of the Bordered Hessian, X i = input i, H ij = the co-factor of f ij A positive (negative) AES indicates a substitution (complementarity) relationship between labor and knowledge capital as increase in the price of labor increases (decreases) the demand for knowledge capital. We find a positive and significant Allen Elasticity of Substitution (AES) for calls with no prior access to the knowledge repository indicating that for these calls there is a substitution relationship between tech support engineer s labor hours and the number of knowledge documents in the repository. However, for the calls with prior access to the knowledge repository before calling the AES is negative. This suggests that for the call with prior access to the knowledge repository, the relationship between the tech support engineer s labor hours and the number of knowledge documents in the repository is somewhat different, and not a substitution relationship. Conclusion The results of this study contribute a more nuanced view of the relationship between knowledge capital and labor in the context of knowledge intensive tasks. Our results provide evidence that knowledge capital can serve both as a substitute as well as a complement for highly skilled labor. The results based on detailed information on repository access and calls to a tech support center by field technicians indicates that in general, calls to the support center are made in instances where field technicians encounter more complex problems. The productivity of tech support engineers is higher in terms of the number of calls taken when field technicians call without accessing the knowledge repository than when they call after accessing the repository. In this case the document repositories are substitutes for of the tech support engineer labor. However, if the field technician calls the help desk, after accessing the knowledge repository, the call is on average a more complex call than a no-call, or no-access call. In this case, the tech support engineer uses his/her time/labor and the knowledge repository in a complementary manner to reduce the time to respond to such calls. This research contributes to the emerging stream of work in the IT Value literature by highlighting the contingent role of information technologies in knowledge intensive tasks. References are available at: