Performance Management for Inter-organization Information Systems Performance: Using the Balanced Scorecard and the Fuzzy Analytic Hierarchy Process Y. H. Liang Department of Information Management, I-SHOU University, Kaohsiung City, Taiwan, R.O.C. german@isu.edu.tw Abstract - Under the dramatic change of business environment, organizations must break up the boundary of organization, share and transfer information with strategic alliance partners or customers more closely to promote collaboration each other in order to improve enterprises competitiveness. Inter-Organizational Information Systems (IOIS) can traverse the formal boundaries of an organization and are utilized by organizations to connect partnership and customers to promote connection and information integration and support transactions-based information and communication technology. Therefore, enterprises increasingly place more importance on IOIS. The purpose of this study is to propose a model for assessing IOIS performance. This study develops the framework based Balanced Scorecard (BSC) and uses the fuzzy analytic hierarchy process approach to consider the relative importance of diverse performance indicators in assessing IOIS performance for Taiwan s information and electronic industries. The results of this study provide references for managers and future researches when assessing the performance of IOIS. Keywords - inter-organization information systems, balanced scorecard, fuzzy analytic hierarchy process I. INTRODUCTION Previous inner organization information systems were primarily constructed according to the requirements of the various functions of the information systems of departments, such as finance, production, and sales departments. These information systems operated separately, did not integrate or connect to one another, and did not communicate with external organizations. Now, because of the dramatic change in the business environment, organizations must eliminate organizational boundaries and partake in more information sharing and transfer with strategic alliance partners or customers to promote collaboration to improve enterprises competitiveness. Therefore, enterprises are increasingly placing more importance on Inter-Organizational Information Systems (IOIS). IOIS involve value-added network-based electronic data interchange, Internet-based EDI, extranets, electronic exchange in supply chain management, and other decision support systems that utilize a usual technology under a sole ownership [1]. IOIS can traverse the formal boundaries of an organization and are utilized by organizations to connect partnership and customers to promote connection and information integration and support transactions-based information and communication technology (ICT). IOIS consist of boundary-spanning technologies that link suppliers and buyers [2]. Performance management is the most effective tool used for organizational reengineering. Performance management aims to aid people in achieving their strategies, missions, visions and goals. Good performance management systems can enable departments in an organization to allocate reasonable resources, prioritize resource investment, and further improve departmental effectiveness and efficiency, and organizational members adopt identical methods to pursue their goals, encourage morale, and promote member focus on organizational vision. The Balanced Scorecard (BSC), which was developed by Kaplan and Norton [3] in 1992, is a useful and popular method of identifying business performance using lagging and leading indicators based on the foundation of visions and strategies. The BSC implies that organizational performance is evaluated through the utilization of not only financial indicators but also, simultaneously, nonfinancial indicators. The BSC built a framework to transform organizational vision and strategies into a series of consistent performance indicators and thus executes and controls organizational administration, allows organizational members to more concretely familiarize themselves with the organization s vision and strategies, and assists managers in tracking the outcomes of implemented strategies. BSC can assist managers of enterprise organizations in holistically evaluating information technology (IT) investments, as well as the performance of information system (IS) departments. This study builds a framework for evaluating enterprise MIS departments based on the BSC. The study summarizes how to combine the BSC and the fuzzy AHP to create a decision tool for enterprise organizations. The tool can be used to not only assess the contribution of a specific enterprise information system but also analyze the performance and direct the activities of enterprise information systems. Satty s (1980) [4] analytical hierarchy process (AHP), which is a multi-criteria technique, is considered appropriate for solving complex decision problems [5]. 978-1-4799-0986-5/13/$31.00 2013 IEEE
Clinton et al. (2002) [6] supposed the AHP is a single method that offers a means of selecting proper metrics, and this method is capable to satisfy each requirement of metric choice and scorecard construction. The AHP is based on theory and offers information on the relative weight of the BSC performance indicator [7] [8]. Zadeh (1965) [9] developed fuzzy theory for handling problems involving fuzziness and vagueness. Lee et al. (2010) [10] posited that traditional BSC failed to consolidate diverse performance indicators. Lee et al. (2010) [10] also suggested the fuzzy AHP method as a solution to this problem. Consequently, the study explored IOIS performance from the perspectives of the inner and outside organizations. This study develops the Balanced Scorecard (BSC) framework, and uses experts pairwise comparisons in the fuzzy analytic hierarchy process approach to consider the relative weight or importance of diverse performance indicators in assessing IOIS performance for Taiwan s information and electronic industries. The result of the study will allow organizations to assess the performance of IOIS. A. IOIS II. LITERATURE REVIEW In the 1980s, enterprise departments stressed the value of performance and the pursuit of quality. Enterprise departments gradually learned the importance of information technology. Enterprise departments must adjust their organizational structures to match the rapidly changing information technology environment and increase in numbers of users, and information management is the most challenging among the various required adjustments. However, enterprise departments usually developed their information systems according to their individual requirements and, hence, did not communicate with one another. The concept of IOIS stemmed from the interorganizational information sharing system, introduced by Barrett and Konsynski (1982) [11]. IOIS were first formally defined by Cash and Konsynski (1985) [12], and Cash and Konsynski (1985) [12] proposed that IOIS implied the sharing of automated information systems between two more companies. Kumar and Soliman ( 2003 ) [13] believed that IOIS, which are automated information systems, can traverse organization obstacles through the use of various technologies. B. Balanced scorecard Drucker (1973) [14] indicated that management requires constructing a system for assessment and evaluation. Performance appraisal system thus is important for performing management tasks. Profit organizations were the first to implement performance appraisal systems. Performance appraisal systems for profit organizations have traditionally measured performance financially. For non-profit organizations a different approach is used, since for profit is not a main objective for such organizations, and possibly is even a constraint. Financial performance represents a subjective measure of how well a firm can use assets from its primary mode of business to generate revenues. Notwithstanding, for government organizations and other non-profit organizations, financial performance primarily represents a measure of how efficiently a government organization can use its budget. Using financial performance to measure is not sufficient to measure the government organization performance. Current strategic management suggests that there should be a strong linkage between strategic plans and performance measures [15]. Kaplan & Norton (1992) s work [3] and determinants framework can provide this linkage. Kaplan & Norton (1992) [3] developed BSC to reform the traditional performance appraisal system and as a means to evaluate corporate performance from four different perspectives, including financial, customer, internal business process and learning and growth, to measure performance. BSC suggests that systems for appraising business performance should not only consider financial lagging indicators, but also customer, internal business process and learning and growth leading indicators [3]. Kaplan & Norton (1996) [16] indicated that BSC objectives and indicators based on vision and strategies can maintain a balance between short and long term objectives, financial and non-financial measures, lagging and leading indicators, and internal and external performance perspectives. Kaplan & Norton (2004) [17] suggested strategy maps as a main means of describing and communicating strategies. Strategy maps clarify how to convert intangible assets, such as intangible activities contributing to learning and growth, to improve organizational processes, increase customer satisfaction, and obtain better tangible outcomes, such as return on investment.. III. RESEARCH METHOD Step 1: Construct hierarchical framework of the BSC performance evaluation criteria From the four BSC perspectives, the hierarchical framework of the BSC performance evaluation criteria is constructed. Step 2: Using AHP method to calculate the weight If get the weightmatrix W in pairwise comparison matrix A, standardize geometrical mean of row vectors, multiply element in every row, get geometrical mean and normalize it. Step 3: Construct Positive Reciprocal Matrix Every evaluation member use fuzzy AHP evaluation scale to express relative weight between each dimensions
and criteria, and construct fuzzy Positive Reciprocal Matrix. Step 4: Consistency Check The check methods are as follows: 4.1 Consistency Index (C.I.) According to Consistency Index (C.I.), C.I=0 indicate that evaluation has perfect consistency; C.I>0 indicate that evaluation has consistency; C.I. <0.1 indicate that evaluation has evaluation has tolerant bias. 4.2 Consistency Rate (C.R.) Saaty (1980) [4] supposed that Consistency Rate (C.R.) to evaluate the consistency of pairwise comparisons in a matrix among criterions. Under the condition of different rank of matrix, it produce different random index (R.I.). Under the condition of the same rank of matrix, the ratio of C.I. to R.I. is called C.R.. When C.R 0.1, the consistency level is acceptable. Step5: Calculate fuzzy weight value Utilize the Lambda-Max method which Csutora and Buckley (2001) [18] proposed, calculate the fuzzy weight of evaluation criterions. The steps of calculation are as follows: 5.1 When α=1, use α-cut to get median Positive Reciprocal Matrix. Then, calculate the weight use AHP method to get the weight matrix. 5.2 When α=0, use α-cut to get minimum positive reciprocal matrix and maximum positive reciprocal matrix. Then, calculate the weight use AHP method to obtain the weight matrix. 5.3 In order to make sure that calculated weight value is fuzzy number, therefore, adjusted the coefficient. 5.4 After obtained adjusted coefficient, calculate minimum positive reciprocal weight matrix and maximum positive reciprocal weight matrix of every measurement dimension. 5.5 Combing adjusted minimum, maximum and median values to get the fuzzy weight in kth evaluation member and kth measurement dimension. 5.6 Utilize average method to integrate the fuzzy weight of evaluation members and measurement dimensions. IV. RESULTS Based on previous studies on applying the BSC approach to information systems, this study used the 17 indicators as performance evaluation indicators to construct the research model and develop the questionnaire items based the model. The 17 indicators are showed in Figure 1. Figure 1. The hierarchical framework of the BSC performance evaluation criteria The 25 related managers of the enterprises in Taiwan s information and electronic industries which located in the Technology Park, Industrial Park, and Export Processing Zone in Taiwan were selected, the propose framework was explained to these experts and each expert was separately asked to do pairwise comparisons through the use of personal interviews for the final data analysis. Next, calculate the weights of model dimensions and indicators using Fuzzy AHP method. The results showed in Figure 2. The results demonstrated that the importance weights of all dimensions were ordered as follows when managers measure IOIS performance: the financial, the internal processes, the learning and growth and the customer dimension. IOIS focus on the financial and internal processes dimensions and related performance indicators. The managers of Taiwan s information and electronic industries pay more attentions on the traditional financial and internal processes dimension, and less on the learning and growth and the customer dimension. However, academic researchers recently stress the importance of
learning and customers for business administration for future development [3]. Additionally, the importance weights of the top 5 performance indicators were ordered as follows: increase the inventory turnover rate, reduce information exchange costs, reduce transaction costs, reduce IT repetition investment expenses, and accelerate the process speed related to working with suppliers and customers. All performance indicators are larger than 0.0202. For performance indicators, consistent with the suggestions of Lu et al. (2006) [19], the managers still emphasized operational efficiency and capability on IOIS performance measurement. followings. First, this study contributes to the IOIS theory, as it offers the model for assessing IOIS performance. The proposed model offers a strong linkage between strategic plans and performance measures based BSC, and also considering inner and outer organizational perspectives. The model provides a holistic viewpoint of the firms by simultaneously considering its performance from four perspectives: the learning and growth, internal business processes, customer, and financial perspectives. Furthermore, the enterprises then use the model to develop strategies and blueprints for self-evaluation. Second, this study uses the fuzzy AHP method to assess IOIS performance. The fuzzy AHP method can conquer this problem which decision makers are often uncertain in assigning the evaluation score in using the AHP method. The managers in Taiwan s information and electronic industries use the fuzzy AHP method is liable lead to better balanced scorecard consequences. For Taiwan s information and electronic industries, the proposed model can serve as a useful modeling instrument in analyzing coupled improvement problems. Identifying the dimensions and performance indicators offers an important piece of information which would assist managers in Taiwan s information and electronic industries to better understand the main facets of IOIS performance and adopt the right action to enhance the total performance. The model not only aid the companies in Taiwan s information and electronic industries to respond to the changes much faster, but also supplies the management teams to move forward with confidence. This study has some limitations. First, drastically changing environments influences dependency relationships of dimensions and indicators and the accuracy of the proposed model. Second, the proposed model is used in enterprises in which IOIS have already been deployed. If IOIS have not been deployed, one might not be able to apply the model effectively. Figure 2. The result of this study V. CONCLUSION The results of this study supply two contributions as
REFERENCES Cisco and Xiao Tong in China, Information Management, vol. 43, pp. 395 408,, 2006. & [1] K. Lyytinen, and J. Damsgaard, Inter-organizational information systems adoption - A configuration analysis approach, European Journal of Information Systems. vol. 20, pp. 496 509, 2011. [2] C. Craighead, J. Patterson, P. Roth, and A. Segars, Enabling the benefits of supply chain management systems: An empirical study of electronic data interchange (EDI) in manufacturing, International Journal of Production Research, vol. 44, no. 1, pp. 135 157, 2006. [3] R. S. Kaplan, and D. P. Norton, The Balanced Scorecard - Measures that Drive Performance, Harvard Business Review, pp. 71 79, 1992. [4] T. L. Saaty, The Analytic Hierarchy Process, New York: McGraw-Hill, 1980. [5] İ. Yüksel, and M. Dagdeviren, Using the fuzzy analytic network process (ANP) for Balanced Scorecard (BSC): A case study for a manufacturing firm, Expert systems with application, vol. 37, no. 2, pp. 72 79, 2010. [6] D. B. Clinton, S. A. Webber, and J. M. Hassell, Implementing the balanced scorecard using the analytic hierarchy process, Management Accounting Quarterly, vol. 3, no. 3, pp. 1 11, 2002. [7] D. L. Searcy, Alignment the balanced scorecard a firm s strategy using the analytic hierarchy process, Mangement Accounting Quarterly, vol. 5. no. 4, pp. 1-10, 2004. [8] S. I. Liedtka, Analytic hierarchy process and multi criteria performance management systems, Cost Management, vol. 19. no. 6, pp. 30 38, 2005. [9] L. A. Zadeh, Fuzzy Sets, Information and Control, vol. 6, pp. 338 353, 1965. [10] A. H. I. Lee, W. C. Chen, and C. J. Chang, A fuzzy AHP and BSC approach for evaluating performance of IT department in the manufacturing industry in Taiwan, Expert systems with application. vol. 34, no. 1, pp. 96 107, 2010. [11] S. Barrett, and B. Konsynski, Inter-Organization Information Sharing Systems, MIS Quarterly. vol. 6, no. 4, pp. 93 105, 1982. [12] J. I. Cash, B. R. Konsynski, IS Redraws Competitive Boundaries, Harvard Business Review, vol. 63, no. 2, pp. 134 142, 1985. [13] K. S. Soliman, Internet-based Business-to-Business electronic commerce: A CIO s perspective, Information Systems Management, vol. 20, no. 1, pp. 35 41, 2003. [14] P. F. Drucker, Management: Task, responsibilities, practices, New York: Harper and Row, 1973. [15] L. Kloot, and J. Martin, Strategic performance management: A balanced approach to performance management issues in local government, Harvard Business School Press, pp. 231-251, June 2000. [16] R. S. Kaplan, and D. P. Norton, The Balanced Scorecard :Translating Strategy into Action, Boston, MA: Harvard Business School Press, 1996. [17] R. S. Kaplan, and D. P. Norton, Strategy Maps: Converting Intangible Assets into Tangible Outcomes, Boston, MA: Harvard Business School Press, 2004. [18] R. Csutora, and J. J. Buckley, Fuzzy Hierarchical Analysis: The Lambda-Max Method, Fuzzy Sets and Systems, vol. 120, no. 2, pp. 181 195,, April 2001. [19] R. Csutora, and J. J. Buckley, Critical success factors of inter-organizational information systems-a case study of