CRITICAL SUCCESS FACTORS OF PROCESS PERFORMANCE MANAGEMENT SYSTEMS: RESULTS OF AN EMPIRICAL RESEARCH

Transcription

1 CRITICAL SUCCESS FACTORS OF PROCESS PERFORMANCE MANAGEMENT SYSTEMS: RESULTS OF AN EMPIRICAL RESEARCH Blasini, Josef, University of Regensburg, Universitätsstraße 31, Regensburg, Germany, Abstract A process performance management system (PPMS) helps to plan, monitor, and improve the performance of business processes. In PPMS, process-focused key performance indicators (KPIs) are used to evaluate the actual process performance and to improve business processes and, subsequently, corporate performance. The empirical study focuses on PPMS and its success factors. The study is based on a causal model of hypotheses, which was developed under consideration of the results of a literature review and a multiple-case study. The structural equation modeling approach is used to analyze the causal relation hypotheses. Using data collected in an online-survey, multivariate analyses show that actual PPMS usage is strongly influenced by management support while information quality and system quality benefits process performance. However, the intention to use a PPMS is positively influenced by expected net benefits and not by user satisfaction with the actual PPMS. The study shows important findings for practical and theoretical issues of how PPMS success can be influenced. Keywords: Process Performance Management, Performance Measurement, Success Factors, Empirical Research 1

2 1 Introduction More and more companies use process-focused key performance indicators (KPIs) to evaluate and to monitor the performance of their business processes. Process KPIs like throughput time, quantity of output, error rate, or the number of complaints can be used to make better decisions and to support the management level as well as the operative level. A process performance management system (PPMS) (Kueng and Krahn, 1999) which comprises the underlying concept, methodological procedures, and supporting information systems (IS) helps to monitor and manage business processes by using process KPIs. This leads to better transparency of the process performance and makes a major contribution to information processing in companies. Consequences of poor process performance can be the change of resource allocations, schedules, and priorities (Galbraith, 1974), the redesign of processes, or reconsideration of planned performance targets. These process improvement activities lead to higher process performance and consequently to a higher firm performance. But what is critical for a successful PPMS? Although every company faces the problem of how to measure process performance, some companies implement numerous KPIs but ignore underlying management procedures, which are important as well. Other companies focus on the technical system, ensure functionalities or properties like up-to-dateness, adaptability of KPIs, and availability, but fail to identify the company-specific process KPIs. In addition to that, PPMS is a special type of knowledge management systems. A PPMS supports not only the continuously needed measurement but also the improvement phase. As we discovered in several case studies, if process performance is very low, the PPMS is needed and used more than before. Subsequently, the performance improves and PPMS usage can decrease again. In summary, PPMS, its usage, and its success are thus an interesting research field, which reveals the need for a PPMS-specific IS success model helping to understand the relations between the PPMS success factors and the successful application of a PPMS. Thus, in this paper the following central research question is addressed: What are the critical success factors which contribute to the success of a PPMS? Following this short introduction, underlying background theories concerning PPMS and the impact of success factors will be presented. We derive a causal model of hypotheses based on the results of a prior literature review and a multiple-case study. The causal model is validated with empirical data collected in an online-survey. We present the results of a multivariate analysis and evaluate both the validity of the measurement models and of the structural model. The paper ends with a short conclusion which includes the summary and interpretation of the results, limitations of the research, and the next research steps. 2 Background Theories 2.1 Process performance management Performance management is a company-wide concept comprising a control cycle of the four steps plan, improve, control, and communicate, each of which affects the performance and underlying mental models of involved protagonists (Krause, 2005). The recorded data are often financial ones and are used for corporate controlling supported by management information systems. When performance management focuses explicitly on process controlling and the collection of process relevant data, it is called process performance management (PPM) (Melchert et al., 2004). PPM (see (Jeston and Nelis, 2008a; Heckl and Moormann, 2010)) can be defined as the use of performance measurement information to effect positive change in processes (see (Amaratunga and Baldry, 2002; Folan and Browne, 2005; Cleven et al., 2010)). Focusing on processes, PPM narrows the view of performance management and restricts it to the measurement and management of process-related aspects. PPM can therefore be seen as the active management of business processes through planning, monitoring, and controlling of process performance by using process-focused KPIs. Within the business process 2

3 management (BPM) lifecycle, PPM particularly supports the monitoring and measurement phase. PPM is generally supported by IS (e.g. workflow management systems, BPM suites, and Business Intelligence (BI) systems), which collect data from operational systems to merge and analyze relevant process information. IS disseminate the results (current, historical, and target values, trend) to the process actors (Kueng and Krahn, 1999) and decision makers at managerial and operative levels. The term PPMS comprises PPM as a conceptual management approach as well as the related technical IS. The performance measurement of business processes is a significant part of the sustainability of process improvement and management within an organization (Jeston and Nelis, 2008b). PPMS provides the prompt and continuous monitoring of past and present process performance, allows to estimate future process performance, and supports the identification of optimization potentials in business processes (Dinter and Bucher, 2006). This on-going management of business processes within the enterprise includes amongst others the following tasks (Jeston and Nelis, 2008b): Establish (negotiate) process performance targets, ensure consistency with the organizational strategy and objectives, report and analyze the actual performance levels as compared to the established and agreed targets, and progressively move from a reactive to a proactive and, finally, predictive state. Performance reports include different indicators, measures, and figures (Heckl and Moormann, 2010) which are generally subsumed under the term process KPIs (e.g. throughput time, output quantity). 2.2 Information systems theory models Numerous empirically validated models for measuring the general IS success were proposed (Urbach et al., 2009) which led to a common understanding of success and possible success specifications. Several success and acceptance models were developed over the last few decades to make the success of methods, IS or the acceptance of technologies measurable. The DeLone & McLean IS success model (DeLone and McLean, 2003) is still the dominant basis of IS success measurement (Urbach et al., 2009). Several empirical studies confirmed the significance of this success model (Petter and McLean, 2009) which comprises information quality, system quality, service quality, and especially the dependent success variables (intension to) use, user satisfaction, and net benefits (DeLone and McLean, 2003). Another well-known and intensively used model is the Technology Acceptance Model (TAM) by Davis (1989), which consists of two main categories: perceived usefulness and perceived ease of use. The re-specified version of the IS success model by Seddon (1997) who integrates both approaches, adds expectations about net benefits, and emphasizes the different levels of IS success measurement individual, organization and society. Performance and success are linked to efficiency and effectiveness (Budäus and Dobler, 1977). The benefit from information use is the maintenance or change of mental models (Schäffer and Steiners, 2004). This leads to higher efficiency and effectiveness, and thus to a higher organizational and financial performance of the company (Barr et al., 1992). Hence, PPMS success can be defined as achieving of high process performance represented by efficient and effective business processes. 2.3 Research design Since DeLone and McLean (2003), Petter et al. (2008), and Petter and McLean (2009) motivate to identify IS-specific success models, we aim to identify reliable success factors for PPMS. Our research follows the 12-stage multi-method research design according to Gable (1994), which integrates qualitative (case study method) and quantitative (survey approach) research methods to one single research design to increase the robustness of results (Kaplan and Duchon, 1988). This paper focuses on the stages 6 to 12 which represent empirical validation of the model based on data of a survey. A short summary of the results of the first five stages (literature review and multiple-case study) is given in the following. Then the structural model including the constructs and the hypotheses and the measurement models will be presented. The models and hypotheses will be tested using empirical data, followed by a short discussion and interpretation of the findings. 3

4 3 Related Literature, Literature Review and Multiple-Case Study A major topic of research is the impact of IS on process effectiveness and organizational performance (Chang and King, 2005; Melville et al., 2004). The benefit of IS for companies has been debated from several viewpoints as well (see (DeLone and McLean, 2003; Petter and McLean, 2009; Urbach et al., 2009)). Critical success factors for different domains and IS were identified, e.g. in BPM (Trkman, 2010; Rosemann et al., 2004), BI (Olbrich et al., 2011), performance measurement (Huan et al., 2010), or in business performance management (Ariyachandra and Frolick, 2008). However, very little discussion has so far been dedicated to PPMS. Bucher and Winter (2006) identified that the use of established methods and standards is crucial for successful PPM users. Cleven et al. (2010). analyzed four stereotype problem situations of PPM and highlighted optimization potentials. For example, KPI enthusiasts define and implement a large amount of KPIs, but only monitor a small number of processes. Limitations of performance management systems are discussed, too (see (Pidun and Felden, 2011)). Based on these findings, we conducted a two-step literature review. A review of representative literature (vom Brocke et al., 2009) helped to get an overview of possible success factors. A second, intensified literature review of 33 contributions revealed more stable success factors and related measurement items. We resolved overlappings and described the success factors more precisely: individual knowledge and competence, process knowledge, integrated performance management, information quality, system quality, service quality, process quality, management support, incentive system. A multiple-case study increased the reliability of the success factors. Four companies from different economic sectors (service sector, energy industry, manufacturing sector, and financial services) were selected to obtain reliable and valid results. After analyzing project documentations, conducting interviews and surveys most of the candidate success factors could be identified as validated in practice as well. The only success factor which was not supported was incentive system. As service quality showed high correlations to process quality, representing the quality of process support before, during, and after the application of PPMS, we integrated both success factors into service quality. 4 Theory Development Based on the prior research results and adopting background theories, we propose our research model and the hypotheses, which express the relationships between the different constructs, in Figure 1. Information Quality H1(+) H2(+) Perceived Net Benefits H10(+) Usage H3(+) H12(+) H4(+) H11(+) H14(+) System Quality H5(+) H6(+) User Satisfaction H13(+) Management Support H15(+) H7(+) Service Quality H9(+) H8(+) Expected Net Benefits H16(+) Intention to Use Figure 1. Research model 4

5 4.1 Theoretical constructs Our research model consists of nine constructs (measurement instruments/items in italics): Information Quality: The first success factor in Figure 1 is information quality which had the highest numbers of mentions in the analyzed literature. Processes can only be controlled if the necessary information about their performance is measured and made visible. Therefore information quality, which is the main basis for controlling activities, is assumed to represent the main success factor for PPMS success. Completeness, accuracy, understandability, reliability (see (DeLone and McLean, 1992)), balance, and consistency (Janz, 2008) of information and/or of the underlying performance indicators can be subsumed under this variable as well as measurement depth (Gleich, 2001; Janz, 2008) and up-to-date data quality (see (Cleven et al., 2010)). System Quality: PPMS can be seen as a methodical approach which may be supported by an underlying technical IS. The independent variable system quality refers to usability, availability, reliability, adaptability, responsiveness (DeLone and McLean, 2003), navigation, design, searchability, structure, functionality, accessibility (Urbach et al., 2009), support of the entire PPMS lifecycle (Krause, 2005), flexibility, and integration (Bandara, 2007). This success factor sums up all desirable system properties of the PPMS and reflects the construct ease of use of TAM (Petter et al., 2008). Further items from the case studies: scalability, understandability. Service Quality: It is the last of the three independent success variables and uses the dimensions of tangibles, reliability, responsiveness, assurance, and empathy to measure service quality (DeLone and McLean, 2003). Due to the high correlations we integrated items of the construct process quality of Urbach et al. (2010). Methodological support in the implementation phase (Krause, 2005) is related to service quality as well as to process quality in the application phase (PPM process efficiency, reliability, regularity etc.) (Bucher and Winter, 2006; Gleich, 2001; Krause, 2005). Thus the construct service quality includes questions whether the implementation of the PPMS and its processes are methodically supported and whether there is a standardized PPM process (regular controlling, documented procedures, easy to understand). Perceived Net Benefits: Net benefits are measured in terms of perceived usefulness (Petter et al., 2008). It is the first dependent variable and represents the extent to which PPM is contributing to the success of individuals, groups, organizations, industries, and nations (Petter et al., 2008). Since the goal of PPM is to increase efficiency and effectiveness of operational processes, this construct represents improved business processes and high process performance: efficient processes, minimal input, effective processes, output meets target. Usage: This construct is represented by an intensive usage of the current PPMS on a regular basis. As DeLone and McLean (2003) discovered that the extent of IS usage is an indicator of its success. Successful IS are much more used than unsuccessful ones: used a lot, intensively used, regularly used. User Satisfaction: This instrument measures the overall satisfaction with an IS (Petter et al., 2008). It is a common measurement construct to determine the degree to which the expectations of the system user are fulfilled. Together with the other two dependent variables perceived net benefits and usage it represents the success or non-success of a PPMS: good PPM, PPM possibilities, PPM functions. Intention to Use: The future extent of PPMS usage is an indicator of PPM success. Petter and McLean (2009) found out that relation strengths to intention to use have the highest values. High intention to use thus indicates that the PPMS is expected to be used in the future and used more. Expected Net Benefits: Initiated by Seddon (1997), the question on how to integrate expectations on an IS into IS success models is still not answered. Although Petter et al. (2008) think the changes introduced by Seddon complicate the model, they strongly motivate to change and extend the current IS success model and show possible modifications by disconnecting use and intention to use, distinguishing between individual and organization level, or by introducing new relationships towards 5

6 net benefits into their model. The case studies showed that, although the degree of PPMS usage was very low and the company staff was very dissatisfied with the current version of the PPMS, the intention to use the PPMS was, however, very high. The interviewees said they saw the benefits they could reach if they further improved and developed their PPMS. Thus they expected higher benefits for the future, which is why we integrated the measurement of expected net benefits in our model. In contrast to perceived net benefits, expected net benefits assumes the highest process performance possible by using the most improved PPMS possible. Since pretests showed that items asking for the expected process performance are not useful (all questions were rated as 7), we rather focused on the individual level. We included items of desired individual benefits during information usage according to the three types instrumental, conceptual, and symbolic (see (Pelz, 1978)): making decisions, enforce decisions, inform others, up-to-date, future decisions. Using Seddon s construct of expectations thus supports the distinction of current usage and intention to future use of a (possibly improved) PPMS which fully meets expected net benefits. Management Support: Regardless of the unit of analysis, the support of (top) management is one the most frequently mentioned success factors for IS (see (Bandara, 2007; Urbach et al., 2010; Wixom and Watson, 2001)). Bucher and Winter (2006), in particular, stress the importance of top management support and of professionalizing process management in the organization. Petter and McLean (2009) strongly motivate to include this construct in future IS success studies. Items: Implementation supported, superior supports, superior expects, management supports, management expects. We assume that the remaining candidates individual knowledge and competence, process knowledge, and integrated performance management are factors of second order or rather moderating control variables, because the number of mentions in the literature was very low, which was also confirmed by the case studies. In this paper, we concentrate only on direct success factors, being aware of the fact that further research is needed concerning possible moderating effects. 4.2 Hypotheses H1, H2, and H3: The quality of information is positively influencing perceived net benefits, PPMS usage, and user satisfaction. Information quality measures have been proven to be strongly associated with net benefits, system use, and user satisfaction (DeLone and McLean, 2003). Transferring this concept to PPMS, we can identify three different interrelationships. High quality of the desired characteristics of information products leads, on an organizational level, to higher process performance and, on an individual level, to better decisions (H1). On the other hand, there is an impact only working on the individual level, too. High information quality leads to higher usage of the PPM and the supporting IS (H2) and to higher user satisfaction (H3). H4, H5, and H6: System quality is positively influencing perceived net benefits, PPMS usage, and user satisfaction. Similar to information quality, the system quality of the PPMS has individual as well as organizational impacts. Being aware of the fact that the strong support for the interrelationships between system quality and net benefits, use, and satisfaction were empirically proven (Petter and McLean, 2009), we postulate that higher PPM system quality leads to higher process performance (H4), higher PPMS usage (H5), and higher user satisfaction (H6). H7, H8, and H9: The quality of the supporting services is positively influencing perceived net benefits, PPMS usage, and user satisfaction. Relying on generic frameworks or measures can be a limitation of performance measurement systems (Pidun and Felden, 2011). Thus a methodically supported PPMS with a high service quality leads during implementation as well as during application to better processes (H7), to a more consistent and intensive use of the PPMS (H8), and consequently to higher user satisfaction (H9). H10 and H11: PPMS usage is positively influencing perceived net benefits and user satisfaction. The more the PPMS is applied, the more efficient and effective the business processes become (H10). 6

7 According to the IS success model, positive experience of IS usage leads to higher user satisfaction (H11) in a process sense as well as in a causal sense (DeLone and McLean, 2003). H12: Perceived net benefits are positively influencing user satisfaction. If the goal to improve process performance is reached, the expectations of the users are matched and their satisfaction is high. H13: User satisfaction is positively influencing the intention to use the PPMS in the future. According to the often applied IS success model of DeLone and McLean (2003), users who are satisfied with an IS will use it even more in the future or have a higher intention to use it. H14 and H15: Management support is positively influencing PPMS usage and intention to use. Top management commitment is a reason for continuing performance management initiatives (Bourne et al., 2002) and is therefore a driver for current PPMS usage (H14) and for future PPMS usage (H15). H16: Expected net benefits are positively influencing the intention to use the PPMS in the future. The intention to use is high if the user believes that the benefits connected with the use of a PPMS are high (see (Seddon, 1997)). 5 Research Methodology and Data Collection Our study used an open online-survey as data collection instrument. A pre-test was conducted prior to the online-survey to test the research instruments for reliability, content validity, and construct validity. In addition, all questions, statements, and selectable answers were intensively tested for comprehensibility. The results of the pre-test helped to phrase the questions and statements more precisely and to make sure that the questions and statements did not overlap. All data collected in the pre-test were excluded from the analysis sample. Measurement scale and operationalization: We used a 7-point Likert scale to identify the level of agreement or disagreement of a participant to each statement made. All constructs were measured by using different indicators. Albers and Hildebrandt state that, if success factors are to be identified, latent constructs have to be measured by means of formative measurement models (Albers and Hildebrandt, 2006). Thus the five independent variables in our research were operationalized as formative indicators. They are aggregations of different manifestations and the causality is from indicator to construct. In contrast to this, the four dependent variables were operationalized as reflective indicators representing similar manifestations of the underlying construct and have to be, thus, highly correlated. The causality is from construct to indicator. Data collection: With the focus being on identifying success factors of PPMS application in practice, we explicitly excluded students from the sample. Moreover, we focused on employees of real-world companies and invited an array of companies to take part in our research study. We published the invitation to participate in our online-survey in PPMS-relevant discussion groups, we informed PPMSinterested persons by means of direct marketing, and we used different mailing lists where PPMS-related topics were discussed. The overall sample size amounted to 441 participants, who started to fill out the online survey. To improve the data quality of the sample, we excluded datasets which were not completed (193 datasets) and datasets with more than 30% of the statements left unanswered. In addition, we excluded participants who had completed the survey in less than 5 minutes, had always checked the same boxes, or demonstrably had given inconsistent answers. After excluding 226 datasets in summary, the sample size amounts to n=215 participants. This can be interpreted as a response rate of 48.75%. Analysis of participants: Prior to the multivariate analysis a descriptive analysis of the participants and their demographic characteristics was performed. Most of the participants were from small and medium enterprises (SME; no more than 500 employees), in summary 141 (65.58%). Nevertheless, each of 11 different company size categories is represented by 4-13% of all data sets, a quite good distribution. In addition, 31 of the companies were listed at a stock exchange, mainly at a German stock exchange (16). 180 participants said that their company was not listed at any stock exchange. 7

8 Asked for their position, the 66 participants belonged to the top management, 33 to the (process) controlling, 30 were process managers, 30 process consultants, 25 were process participants executing the controlled process, and 27 identified themselves as process owners. 6 Multivariate Data Analysis To test the proposed research model, we used the structural equation modeling (SEM) approach. As we formulated five formative constructs representing independent variables, variance-based partial leased squares SEM (PLS-SEM) including bootstrapping and blindfolding, which are supported by SmartPLS (version 2.0 M3), and linear regression supported by IBM SPSS Statistics 20 were used. Hair et al. (2011) provide rules of thumb for model evaluation as well as for the desired thresholds. The minimum sample size in SEM is an intensely discussed topic of research and different rules have been proposed (Westland, 2010). Our sample size of n=215 is higher than the proposed minimum sample size of 30 to 100 (see (Urbach and Ahlemann, 2010)). There are more observations (215) than 10 times the highest number of formative indicators (9 at information quality) and also more than 10 observations per construct (9). Missing values were substituted and replaced by means to avoid biased data which would be the result of eliminating all incomplete observations. We followed the two-step process of PLS-SEM, which involves separate assessments of the measurement models and the structural model (Hair et al., 2011). 6.1 Testing the measurement models To test the quality of the underlying nine constructs, the two different types of measurement models have to be distinguished: reflective and formative ones. Quality of reflective measurement models: Internal consistency reliability: Hair et al. (2011) state composite reliability to be higher than Table 1 shows that the composite reliability of all four reflective constructs is 0.88 or higher. Cronbach s Alpha coefficients are all higher than the minimum of 0.70 (Hair et al., 2011). Construct Type Comp. reliab. AVE Cronb. Alpha R² Q² n=215 Threshold perc net benefits reflective, endo usage reflective, endo satisfaction reflective, endo intention reflective, endo Table 1. Reliability and validity of our research model Indicator reliability: All indicators of reflective measurement models have indicator loadings of more than 0.70 (see Table 2) and thus fulfill this requirement (Hair et al., 2011). Convergent validity: The average variance extracted (AVE) of all four reflective measurement models are higher than the threshold of 0.50 (Hair et al., 2011) (see Table 1). Discriminant validity: Indicator loadings are all higher than all of its cross loadings (no table). To evaluate internal consistency reliability we additionally used Cronbach s Alpha coefficients which are all higher (see Table 1) than the minimum cutoff score of 0.70 (Nunnally, 1978). Quality of formative measurement models: Indicators loadings and weights (no table): To test the significance of formative indicators coefficients we ran a bootstrapping procedure which used 215 cases, 5000 samples, and a missing value algorithm. All the indicator loadings are higher than 3.15 and thus significant at the 1 percent level. Most of the indicator weights are higher than the minimum cutoff score of 1.65 (significance level = 10 percent). This indicates that with respect to the results of the literature review and the statements of experts in our multiple-case study our indicators can be interpreted as demonstrably important due to their theoretical and empirical relevance. 8

9 Construct Indicator Indicator Loading pn_effective perc net pn_effective benefits pn_efficient pn_efficient us_alot usage us_intensively us_regularly sa_expectations sa_measures satisfaction sa_possibilities sa_ppmgood sa_support intention in_future in_more Table 2. Indicator loadings Hyp. Constructs T-value Sig. lvl H16 exp net benefits intention H13 satisfaction intention H14 man support usage H1 inf qual perc net benefits H6 sys qual satisfaction H3 inf qual satisfaction H4 sys qual perc net benefits H11 usage satisfaction H9 serv qual satisfaction H12 perc net benefits satisfaction H8 serv qual usage H2 inf qual usage not sig. H15 man support intention not sig. H5 sys qual usage not sig. H7 serv qual perc net benefits not sig. H10 usage perc net benefits not sig. Table 3. T-statistics 6.2 Testing the structural model While the minimum cut-off score of R² is 0.19 (Chin, 1998), values of 0.25 for endogenous latent variables can be described as weak, 0.50 as moderate, and 0.75 as substantial (Hair et al., 2011). The four R² values are higher than 0.19 (see Table 1). Thus intention to use, perceived net benefits, and usage are weak latent variables and satisfaction can be described as moderate. Using bootstrapping we calculated the path coefficients (see Figure 2) and the path coefficients significance (see Table 3). Table 3 and Figure 2 show that 10 of our 15 hypotheses are supported by the data. Path coefficients (Figure 2) should be at least 0.10 to be considered meaningful for further discussion. Statistical significance (see t-values and significance levels in Table 3) supports the interpretation of the findings. Information Quality ns 0.271*** 0.403**** Perceived Net Benefits 0.142** ns ns Usage Notes: 0.193** 0.163** 0.417**** sample size n = 215 System Quality 0.177*** User Satisfaction **** Management Support ns **** = significant at the level *** = significant at the 0.01 level ** = significant at the 0.05 level * = significant at the 0.1 level ns = not significant ns significant Service Quality 0.221** Expected Net Benefits 0.532**** Intention to Use not significant 0.166* Figure 2. Structural model results In summary, 4 interrelationships are highly significant (p<0.001). H16(+) (exp net benefits intention to use) shows the highest path coefficient (0.532) and confirms our assumption that theoretically feasible and expected net benefits of an PPMS are the main drivers for a positive attitude towards increased PPMS usage in the future. The second highest path coefficient (0.413) belongs to management support usage. This empirical result supports our hypothesis H14(+) that management support has a strong positive and highly significant influence on PPMS usage. The third highest path coefficient is user satisfaction intention to use (H13). Although this relation is empirically proven highly significant (0.001 level), the path coefficient is negative and implies that the more users are 9

10 unsatisfied with their PPMS, the more they want to use it or an improved version of the PPMS in future. In contrast to our hypothesis H13(+), which stated a positive correlation, the relation in our PPMS study is empirically proven to be negative. That is why the original H13 cannot be stated as accepted. The fourth highly significant path coefficient belongs to H1(+) inf qual perc net benefits. 2 relations show high, 4 moderate, and 1 shows low path coefficients and low significance levels (see Table 3). Thus the following hypotheses can be seen as accepted: H3, H4, H6, H8, H9, H11, and H12. 5 of the 16 assumed interrelationships are not supported by our results due to low path coefficients and significance levels: H2, H5, H7, H10, and H15. In summary, the results show that most of our hypotheses are fully supported, but some are not which highlights the need for further interpretation. 6.3 Discussion and interpretation The results of our empirical analysis show, at least, five interesting findings worth to be discussed: 3 success factors: Information quality, system quality, and service quality have strong and significant influence on the success of the PPMS currently used by the participants of our study. Key drivers for efficient and effective processes are information quality and system quality, while service quality supports PPMS usage. Further descriptive analyses confirmed the underlying theory concerning the importance and the value of information and information processing in enterprises. While iq_uptodate (process measures are up-to-date) had the highest mean value (5.44, range of scale = 1-7), iq_depth (depth of measurement = all important aspects are measured) and iq_balance (balance among the process measures) had the lowest mean values (4.41 and 4.33). Thus the participants of our study think that more in-depth process KPIs are needed and the balance among them must be improved. This fact confirms our opinion that many practitioners use too many KPIs which are not very well balanced among each other. Expected net benefits: We used Seddons s construct of expectations and integrated it into our PPMS success model. It turned out to be the key driver for increased future PPMS usage in contrast to user satisfaction towards the current PPMS. We think that the construct expected net benefits fully supports and simultaneously expands the existing IS success model and, thus, is worth to be taken into account when developing success models on other IS or conducting surveys on specific IS success by other researchers. User satisfaction: On the one hand, user satisfaction is the only construct the totality of whose relations to and from other constructs are fully supported and significant. On the other hand, the relation of satisfaction to intention to use (H13) turned out to be negative, but statistically significant. Descriptive analyses showed that the participants of our study are relatively unsatisfied with their PPMS, but have a high intention to use the system in a possibly improved form. In contrast to the IS success model, which states a positive influence of satisfaction to intention to use, the findings of our study rather show a negative influence. Thus, we assume that the relation satisfaction intention to use can be positive in cases where the IS cannot be further improved or the degree of satisfaction is already very high, while in other cases such as our PPMS study where the participants think the IS can be further improved or where the degree of satisfaction is very low, the relation can actually be negative. This finding is in accordance with the metaanalysis of Petter et al. (2008) who described several studies which discovered negative relations (see (Adams et al., 1992; Weill and Vitale, 1999) ). Management support: The research results show that management support strongly influences the current usage of PPMS (H14), but not the intention to use it (H15). This reveals that management support is rather interpreted as negative pressure on users to use the PPMS because it strongly forces the current usage and not in a positive way the intention to use it in the future. Usage perceived net benefits: Surprisingly, there is no significant influence of PPMS usage on perceived net benefits (H10). Instead, some interviewees of our case studies assumed an opposite relationship. Whenever they detected low process performance (= perceived net benefits), they intensified their PPMS usage until process performance reached the desired level. Thus, there could be causal relationships in both directions, which could explain why H10 turned out to be 10

11 insignificant and why the impact of increased PPMS usage on higher process performance could not be empirically proven in our study. 7 Conclusion In this paper, we presented a PPMS-specific IS success model including several success factors for the application of PPMS in enterprises. Being aware that a PPMS is a special type of decision and knowledge management system, this research model helps to understand the relations between the PPMS success factors and the successful application of a PPMS. We state that this research model can be tested and verified by means of collected data of an online-survey. The results of a multivariate analysis show that information quality, and system quality positively influence PPMS success, whereas management support is a key driver for current PPMS usage, but not for the intention to use it. Our theoretical contribution is the new construct expected net benefits, which strongly influences the intention of PPMS usage and extends the DeLone and McLean model. In addition to this, we think that the impact of user satisfaction on intention to use depends on the specific type of IS. If the IS can be improved by the system user or the using company, then the impact of current user satisfaction on intention to use can be low or even negatively correlated and expected net benefits are the key driver for high intention to use because the user thinks he can realize them when improving the IS. In contrast to this, IS which cannot easily be modified and improved (e.g. a specific app on a mobile tablet) must subsequently show a high dependency of intention to use on the user s satisfaction, as discovered by DeLone and McLean (2003). Nevertheless our research has some limitations, e.g. concerning the external validity. Our sample consists of data coming from almost only German speaking participants (204). Although an English version of the online-survey did exist and much effort has been made to enrich the data by intensely promoting the English version, only eleven datasets based on the English version could be considered in the statistical analysis. A larger sample size would lead to even more reliable results. Although we think that the sample including only PPMS experts is representative, the participants answers were influenced by knowing to be part of a PPMS study (Hawthorne effect). In addition, although descriptive analyses provide first insights into the data, further analyses will have to be conducted. Since in this paper we focused on the main constructs, we excluded control variables which may have a moderating effects. We assume two categories of control variables: external, meaning unchangeable, and organization-internal variables. Dynamics (within the sector, of technology, etc.) and competitive environment (see (Eckey, 2006; Janz, 2008)) belong to the first category, while competence, process knowledge, and integrated performance management belong to the internal control variables. We will use them in the next steps of our research to compare different extensions of our model. References Adams, D., Nelson, R., and Todd, P. (1992). Perceived usefulness, ease of use, and usage of information technology: a replication. MISQ, 16 (2), Albers, S. and Hildebrandt, L. (2006). Methodische Probleme bei der Erfolgsfaktorenforschung - Messfehler, formative versus reflektive Indikatoren und die Wahl des Strukturgleichungs-Modells. zfbf, 58 (1), S Amaratunga, D. and Baldry, D. (2002). Moving from performance measurement to performance management. Facilities, 20 (5), Ariyachandra, T.R. and Frolick, M.N. (2008). Critical Success Factors in Business Performance Management - Striving for Success. Information Systems Management, 25 (2), Barr, P., Stimpert, J.L., and Huff, A. (1992). Cognitive Change, Strategic Action, and Organizational Renewal. Strategic Management Journal, 13 (S1), Bourne, M., Neely, A., Platts, K., and Mills, J. (2002). The success and failure of performance measurement initiatives: Perceptions of participating managers. IJOPM, 22 (11), Bucher, T. and Winter, R. (2006). Classification of Business Process Management Approaches - An Exploratory Analysis. Banking and Information Technology, 7 (3), Budäus, D. and Dobler, C. (1977). Theoretische Konzepte und Kriterien zur Beurteilung der Effektivität von Organisationen. Management International Review, 17 (3),

12 Chang, J.C.J. and King, W.R. (2005). Measuring the Performance of Information Systems: A Functional Scorecard. Journal of Management Information Systems, 22 (1), Chin, W.W. (1998). Issues and opinion on structural equation modeling. MISQ, 22 (1), vii-xv. Cleven, A., Wortmann, F., and Winter, R. (2010). Process Performance Management Identifying Stereotype Problem Situations as a Basis for Effective and Efficient Design Research. DESRIST 2010, p , St. Gallen, Switzerland. Davis, F.D. (1989). Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information Technology. MISQ, 13 (3), DeLone, W.H. and McLean, E.R. (1992). Information Systems Success: The Quest for the Dependent Variable. ISR, 3 (1), DeLone, W.H. and McLean, E.R. (2003). The DeLone and McLean Model of Information Systems Success: A Ten-Year Update. Journal of Management Information Systems, 19 (4), Dinter, B. and Bucher, T. (2006). Business Performance Management. (Chamoni, P. and Gluchowski, P. Eds.). Analytische Informationssysteme - Business Intelligence-Technologien und -Anwendungen. Springer Berlin. Eckey, M. (2006). Kontrolle von Beteiligungen als Aufgabe des Controllings. DUV, Wiesbaden. Folan, P. and Browne, J. (2005). A review of performance measurement: Towards performance management. Computers in Industry, 56 (7), Gable, G.G. (1994). Integrating case study and survey research methods: an example in information systems. EJIS, 3 (2), Galbraith, J.R. (1974). Organization design: an information processing view. Interfaces, 4 (3), Gleich, R. (2001). Das System des Performance Measurement: Theoretisches Grundkonzept, Entwicklungs- und Anwendungsstand. Vahlen, Munich. Hair, J.F., Ringle, C., and Sarstedt, M. (2011). PLS-SEM: Indeed a Silver Bullet. JMTP, 19 (2), Heckl, D. and Moormann, J. (2010). Process Performance Management. (Brocke, J. and Rosemann, M. Eds.). Handbook on Business Process Management 2. Springer Berlin. Huan, Y., John, F.Y.Y., Albert, P.C.C., Chiang, Y.H., and Daniel, W.M.C. (2010). A critical review of performance measurement in construction. Journal of Facilities Management, 8 (4), Janz, S. (2008). Performance Management: Eine empirische Untersuchung der Konzeption, Erfolgsauswirkungen und Determinanten am Beispiel der Marketingkommunikation. Universität St. Gallen, Schesslitz. Jeston, J. and Nelis, J. (2008a). Business Process Management: Practical Guidelines to Successful Implementations. Jeston, J. and Nelis, J. (2008b). Management by Process: A Roadmap to Sustainable Business Process Management. Elsevier, Kaplan, B. and Duchon, D. (1988). Combining qualitative and quantitative methods information systems research: a case study. MISQ, 12 (4), Krause, O. (2005). Performance Management - Eine Stakeholder-Nutzen-orientierte und Geschäftsprozess-basierte Methode. Technische Universität Berlin, Berlin. Kueng, P. and Krahn, A. (1999). Building a Process Performance Measurement System: some early experiences. Journal of Scientific and Industrial Research, 58 (3/4 (March/April 1999)), Melchert, F., Winter, R., and Klesse, M. (2004). Aligning Process Automation and Business Intelligence to Support Corporate Performance Management. AMCIS 2004, p , New York. Melville, N., Kraemer, K., and Gurbaxani, V. (2004). Information Technology and Organizational Performance: An Integrative Model of IT Business Value. MISQ, 28 (2), Nunnally, J.C. (1978). Psychometric theory. 2. Edition. McGraw-Hill, New York et al. Olbrich, S., Pöppelbuß, J., and Niehaves, B. (2011). BI Systems Managers' Perception of Critical Contextual Success Factors: A Delphi Study. ICIS 2011, p. 1-21, Shanghai, China. Pelz, D. (1978). Some Expanded Perspectives of Use of Social Science in Public Policy. (Yinger, M. and Cutler, S. J. Eds.). Major Social Issues: A Multidisciplinary View. Free Press New York. Petter, S., DeLone, W., and McLean, E. (2008). Measuring information systems success: models, dimensions, measures, and interrelationships. EJIS, 17 (3), Petter, S. and McLean, E.R. (2009). A meta-analytic assessment of the DeLone and McLean IS success model: An examination of IS success at the individual level. Information & Management, 46 (3), Pidun, T. and Felden, C. (2011). Limitations of Performance Measurement Systems based on Key Performance Indicators. AMCIS 2011, p. 1-8, Detroit, USA. Rosemann, M., de Bruin, T., and Hueffner, T. (2004). A Model for Business Process Management Maturity. ACIS 2004 Proceedings, p. 1-6, Hobart, Tasmania, Australia. Seddon, P.B. (1997). A Respecification and Extension of the DeLone and McLean Model of IS Success. ISR, 8 (3), Trkman, P. (2010). The critical success factors of business process management. IJIM, 30 (2), Urbach, N. and Ahlemann, F. (2010). Structural Equation Modeling in Information Systems Research Using Partial Least Squares. JITTA, 11 (2), Urbach, N., Smolnik, S., and Riempp, G. (2009). The State of Research on Information Systems Success. Business & Information Systems Engineering, 1 (4), Urbach, N., Smolnik, S., and Riempp, G. (2010). Industry-Specificity of Employee Portal Success: A Multi-Group Comparison. AMCIS 2010, p. 1-11, Lima, Peru. vom Brocke, J., Simons, A., Niehaves, B., Riemer, K., Plattfaut, R., and Cleven, A. (2009). Reconstructing the Giant: On the Importance of Rigour in Documenting the Literature Search Process. ECIS 2009, p , Verona, Italy. Weill, P. and Vitale, M. (1999). Assessing the health of an information systems applications portfolio: an example from process manufacturing. MISQ, 23 (4), Westland, C.J. (2010). Lower bounds on sample size in structural equation modeling. Electronic Commerce Research and Applications, 9 (6), Wixom, B.H. and Watson, H.J. (2001). An Empirical Investigation of the Factors Affecting Data Warehousing Success. MISQ, 25 (1),