The DeLone and McLean Model of Information Systems Success Original and Updated Models Satu-Maria Hellstén satu.hellsten@tut.fi Maiju Markova maiju.markova@tut.fi ABSTRACT In this paper we introduce the original as well as updated DeLone and McLean Model of Information Systems Success. The original model has six interrelated dimensions of success: System, Information, Use, User Satisfaction, Individual Impact, and Organizational Impact. The dimensions of Service and Intention to Use were added into the updated model and the original dimensions of Individual and Organizational Impact were combined into one new dimension, Net Benefits. We also present five examples of applying or testing the model together with concrete success measures of IS. We believe that IS success model provides a good framework for identifying and developing different measures. IS success model might provide a practical way to evaluate, for example, user satisfaction and impacts of that satisfaction on the use of information systems. Author Keywords information systems success, evaluation of IS ACM Classification Keywords H1 Information Systems: Models and Principles: Human factors, K4.3 Computers an Society: Organizational Impacts INTRODUCTION Effective measurement of information system (IS) success is an important issue for both practitioners and researchers. The measurement of success is critical in order to understand the value of IS management actions and IS investments [1, 2]. However, IS success is a multidimensional concept that can be assessed at various levels. In addition, various stakeholders may have different opinions of the success of the same information system, for example, product developers may value technical success, end-users easiness of use, and managers increased revenues gained through the use of IS. In the literature, there are nearly as many IS success measures as there are studies, and therefore, different studies and systems are not comparable [1]. The aim of this paper is to introduce the first attempt to capture all different aspects of IS success into a one comprehensive model IS success model as well as its updated version. Based on the literature, this study also describes how this model has been used in several studies and how it could help the development of human-centered technology. This paper is organized as follows. First, the original and updated DeLone and McLean model of IS success are introduced with their dimensions and examples of success measures. After that the utilization of the model in different studies is discussed and concrete ways to measure various dimensions of success are presented. Finally conclusions are drawn. THE DELONE AND MCLEAN MODEL OF IS SUCCESS In order to organize diverse research and to present more integrated view of the concept of IS success, DeLone and McLean introduced a comprehensive, multidimensional model of IS success [1]. The purpose of this model is to be a framework for measuring different dependent variables in IS research. Based on the communications research of Shannon and Weaver from 1949 and the information influence theory of Mason from 1978, as well an empirical management information systems (MIS) research studies from 1981-1987, they categorized IS success into six major dimensions: 1) System, 2) Information, 3) Use, 4) User Satisfaction, 5) Individual Impact, and 6) Organizational Impact. In their study, they reviewed all those empirical studies that have attempted to measure some aspects of MIS success in seven publications (Management Science, MIS Quarterly, Communications of the ACM, Decision Sciences, Information & Management, Journal of MIS, and ICIS Proceedings). Additionally, some other articles were included to contribute to theoretical or conceptual issues. Altogether 180 articles were referenced in the paper. Figure 1 presents these six interrelated dimensions of success: System and Information singularly 1
System Use Individual Impact Organizational Impact Information User Satisfaction Figure 1 IS Success Model [1, p. 87]. and jointly affect both Use and User Satisfaction. In addition, the amount of Use can have a positive or negative effect on the degree of User Satisfaction and vice versa. Use and User Satisfaction are direct antecedents of Individual Impact, and this impact should eventually have some Organizational Impact. [1] In DeLone and McLean IS success model, these six dimensions are examined at three different levels: technical level, semantic level, and effectiveness or influence level. First dimension of the model, Systems, studies the success at technical level. It focuses on the desired characteristics of the information system itself which produces the information. Second dimension, Information, focuses on the information product instead, and characteristics at the semantic level, i.e. the success of the information in conveying the intended meaning. Some examples of success measures of Systems and Information are listed in Table 1. Table 1 Examples of Success Measures Systems and Information System Ease of use Ease of learning Convenience of access Realization of user requirements Usefulness of system features and functions Data and system accuracy Information Importance Relevance Usefulness Timeliness Readability Content At the influence level, Use and User Satisfaction are measured in order to analyze the interaction of the information product with its recipients. Some examples of success measures of Use and User Satisfaction are listed in Table 2. Table 2 Examples of Success Measures Use and User Satisfaction Information Use Amount/duration of use Actual vs. reported use Nature of use: use for intended purpose, appropriate use, type of information used Motivation to use User Satisfaction Satisfaction with specifics Overall satisfaction Information satisfaction: Difference between information needed and received Enjoyment In addition, the influences which the information product has on management decision (Individual Impact) and on organizational performance (Organizational Impact) are measured at the influence level. Some examples of success measures are presented in Table 3. Table 3 Examples of Success Measures Individual Impact and Organizational Impact Individual Impact Learning Decision effectiveness: Decision quality, Improved decision analysis, Correctness, time to make decision Improved individual productivity Task performance Problem identification Willingness to pay for information Organizational Impact Operating cost reductions Staff reductions Overall productivity gains Increased revenues, sales, market share, profits Increased work volume Service effectiveness This model already helps IS researchers in understanding different aspects of IS success, but as DeLone and McLean 2
argues this success model clearly needs further development and validation before it could serve as a basis for the selection of appropriate IS measures [1, p. 88]. Updated D&M IS Success Model As the original IS success model needed further validation, DeLone and McLean proposed an updated model in 2003, again based on a literature review [2]. They added Service (e.g., IS support) as one important dimension. In addition, they added Intention to Use as an alternative measure because an attitude is worthwhile to measure in some context. Finally, they combined Individual and Organizational Impact to one dimension, named Net Benefits; to broaden the impacts of IS also to groups, industries and nations, depending on the context. Information System Service Intention to use Use User Satisfaction Figure 2 Updated D&M IS Success Model [2] Net Benefits In 2002 Rai et al. [4] tested empirically and theoretically DeLone and McLean s model and Seddon s model. Both models deal with IS success, but Seddon s model treats IS Use as a behavior, not as a process leading to individual or organizational impact as in the original model of DeLone and McLean. Seddon s model focuses on the causal aspects of the interrelationships among the taxonomic categories. According to Rai et al, the principal difference between these two models is in the definition and placement of IS use. Seddon says that the use must precede impacts and benefits, but it does not cause them. He considers IS use to be a resulting behavior that reflects an expectations of net benefits from using an information system. More information about the empirical test conducted by Rai et al. is discussed later. Next, various studies where these previously presented models have been used are presented. UTILIZATION OF THE SUCCESS MODEL DeLone and McLean [3] applied their updated model to organize the e-commerce success metrics identified in the literature and demonstrated how the model can be used through two case examples. In both cases, usability was seen as an important measure of System leading to increased number of visits in web sites (Use) and repeat purchases (User Satisfaction). In addition, they suggested that e-commerce studies should include net benefits measures (e.g., incremental sales, market valuation) and not be content to collect only surrogate measures, such as Web site hits (Use). On the other hand, to understand these net benefit results, they argue that the quality of the user s experience and the customer s usage of, and satisfaction with, the system should be measured. [3] Rai et al. [4] did an empirical test in quasi-voluntary IS use context regarding a Student Information System (SIS). The SIS provides online access to a database of students personal and academic information. The use of SIS was not mandatory. The findings support DeLone and McLean s observation that IS success models must be carefully specified in a given context. They also suggest that future research should examine how IS success models perform in different context, including settings that range from strictly voluntary to strictly involuntary use, and recommend refinements as appropriate. Iivari [5] tested the IS success model by using field study of a mandatory information system. The test was conducted with Oulu City Council. The council was working on the adoption of a new information system and trying to accomplish its organizational acceptance. Iivari collected data with questionnaires which were given to new information system s primary users. The questionnaire was based on standard measures. System was measured with six scales: flexibility of the system, integration of the system, response/turnaround time, error recovery, convenience of access, and language. Information was also measured with six scales: completeness, precision, accuracy, reliability, currency, and format of output. The results showed that perceived system quality and perceived information quality were significant predictors of user satisfaction with the system, but they did not matter to system use. User satisfaction was a strong predictor of individual impact. Bryd et al. [6] contributed to IS success research through the development and empirical testing of a process-oriented model of IS success that was based on the model of DeLone and McLean. They examined the influence of lower-level intangible IS and information technology (IT) benefits on higher-level financial measures. They also introduced IS plan quality as an antecedent to the model s input variables. The results supported a process-oriented view of the benefits from IS and showed how the effects of IS along a path can lead to better organizational performance, in their case, lower overall costs. Concrete measures are introduced in the appendix of their study. In their study, they used 7- point Likert scale anchored by Strongly Disagree to Strongly Agree or scale from Not Much to Extensively. Wu and Wang [7] proposed and empirically assessed a knowledge management systems (KMS) success model. Based on an analysis of current practice of knowledge management as well as the DeLone and McLean s model, 3
they used five dependent variables (system quality, knowledge or information quality, perceived KMS benefits, user satisfaction, and system use) in evaluating KMS success. The definitions and success measures are presented in Table 4. Table 4 Construct definition and success measures [7] System quality: How good the KMS is in terms of its operational characteristics Q1. KMS is easy to use Q2. KMS is user friendly Q3. KMS is stable Q4. The response time of KMS is acceptable Knowledge or information quality: How good the KMS is in terms of its output 1) Content quality KQ1. KMS makes it easy for me to create knowledge documents KQ2. The words and phrases in contents provided by KMS are consistent KQ3. The content representation provided by KMS is logical and fit KQ4. The knowledge or information provided by KMS is available at a time suitable for its use KQ5. The knowledge or information provided by KMS is important and helpful for my work KQ6. The knowledge or information provided by KMS is meaningful, understandable, and practicable KQ7. The knowledge classification or index in KMS is clear and unambiguous 2) Context and linkage quality KQ8. KMS provide contextual knowledge or information so that I can truly understand what is being accessed and easily apply it to work KQ9. KMS provide complete knowledge portal so that I can link to knowledge or information sources for more detail inquire KQ10. KMS provide accurate expert directory (link, yellow pages) KQ11. KMS provide helpful expert directory (link, yellow pages) for my work User satisfaction: The sum of one s feelings of pleasure or displeasure regarding KMS US1. I am satisfied that KMS meet my knowledge or information processing needs US2. I am satisfied with KMS efficiency US3. I am satisfied with KMS effectiveness US4. Overall, I am satisfied with KMS Perceived KMS benefits: The valuation of the benefits of the KMS by users PKB1. KMS helps me acquire new knowledge and innovative ideas PKB2. KMS helps me effectively manage and store knowledge that I need PKB3. KMS enable me to accomplish tasks more efficiently PKB4. My performance on the job is enhanced by KMS PKB5. KMS improves the quality of my work life System use: The extent of the KMS being used SU1. I use KMS to help me make decisions SU2. I use KMS to help me record my knowledge SU3. I use KMS to communicate knowledge and information with colleagues SU4. I use KMS to share my general knowledge SU5. I use KMS to share my specific knowledge Lai et al. [8] attempted to extend the DeLone-McLean model by adding a new concept, dependability. To test their new concept, they did a questionnaire survey in internationalized companies in Taiwan. In their study, Lai et al. had questions that were related to Information (IQ), System (SQ), Dependability (DEP), Perceived Usefulness (PU), User Satisfaction (US) and Intention to Use (IU). They found out that SQ had the largest total effect on DEP, PU and IU. Their finding imply that when dealing with enterprise applications, System can help to build users beliefs regarding dependability, satisfaction, and intention to use. Because 1) employees need to have the right information from the right place at the right time, 2) employees efforts must be maximized and 3) enterprise applications must provide integrated service to help employees to complete their daily tasks despite different systems, dependability is a significant factor for the success of enterprise applications. Lai et al. feel that researchers need to understand the importance of dependability. System and Information affect Intention to Use and User Satisfaction through dependability. Linking IS success model with TAM An interesting link between IS success model and Technology Acceptance Model (TAM) was also pointed out in the literature. The TAM model is developed by Davis [9] and it is an adaptation of the Theory of Reasoned Action, and the Theory of Planned Behavior, which are two of the most popular models used to explain IS behavior. According to TAM, Perceived Usefulness and Perceived Ease of Use affect users behavioral intentions. In addition, this effect impacts on IS Use. [4] CONCLUSION DeLone and McLean s IS success model seems to provide a good framework to identify and develop different measures for several important dimensions. Therefore, it could be used also in the field of human-centered technology and usability studies to understand different aspects of IS success. For example, it could provide a practical way to evaluate why user satisfaction is not good and what 4
problems does the usability of the system create to users and organization. Both the original and updated models are based on literature reviews and other researchers have tried to validate, use and develop these models further. Instead of having ready-to-use measures, there is a lot of work to be done when modifying the model for own purposes. However, the model and empirical studies offer great advices and concrete measures for future work for evaluating information systems in different contexts. REFERENCES 1. DeLone, W.H., and McLean, E.R. Information systems success: The quest for the dependent variable. Information Systems Research, 3, 1 (1992), 60-95. 2. DeLone, W.H., and McLean, E.R. The DeLone and McLean Model of Information Systems Success: A Ten- Year Update. Journal of Management Information Systems, 19, 4, (2003), 9-30. 3. DeLone, W.H., and McLean, E.R. Measuring e- Commerce Success: Applying the DeLone & McLean Information Systems Success Model. International Journal of Electronic Commerce, 9, 1, (2004), 31-47. 4. Rai, A., Lang, S. S., and Welker, R. B. Assessing the Validity of IS Success Models: An Empirical Test and Theoretical Analysis, Information Systems Research, (2002), Vol. 13, No. 1, March 2002, 50-69 5. Iivari, J. An empirical Test of the DeLone-McLean Model of Information System Success. Database for Advances in Information Systems (1), April 2005, 8-27 6. Bryd, T.R., Thrasher, E.H., Lang, T., and Davidson, N.W. A process-oriented perspective if IS success: Examining the impact of IS on operational cost. Omega, 34, (2006), 448-460. 7. Wu, J-H., and Wang, Y-M. Measuring KMS success: A respecification of the DeLone and McLean s model. Information & Management, 43 (2006), 728-739. 8. Lai, J-Y., Yang, C-C., and Tang, W-S. Exploring the Effects of Dependability on Enterprise Applications Success in e-business. SIGMIS-CPR 06, April 13-15, 2006, Claremont, California, USA. 9. Davis, F.D. Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly, September (1989), 318-340. 5