A comprehensive framework for selecting an ERP system

International Journal of Project Management 22 (2004) 161 169 www.elsevier.com/locate/ijproman A comprehensive framework for selecting an ERP system Chun-Chin Wei, Mao-Jiun J. Wang* Department of Industrial Engineering and Engineering Management, National Tsing Hua University, Hsin Chu, Taiwan, 300, ROC Received 8 May 2002; received in revised form 28 May 2002; accepted 25 October 2002 Abstract This paper presents a comprehensive framework for combining objective data obtained from external professional reports and subjective data obtained from internal interviews with vendors to select a suitable Enterprise Resource Planning (ERP) project. A hierarchical attribute structure is proposed to evaluate ERP projects systematically. In addition, fuzzy set theory is used to aggregate the linguistic evaluation descriptions and weights. An actual example in Taiwan demonstrates the feasibility of applying the proposed framework. # 2003 Elsevier Ltd and IPMA. All rights reserved. Keywords: Enterprise Resource Planning; Decision making; Fuzzy set theory 1. Introduction An Enterprise Resource Planning (ERP) system is an integrated enterprise computing system to automate the flow of material, information, and financial resources among all functions within an enterprise on a common database [1]. A successful ERP project involves selecting an ERP software system and vendor, implementing this system, managing business processes change (BPC), and examining the practicality of the system. However, a wrong ERP project selection would either fail the project or weaken the system to an adverse impact on company performance [2,3]. Due to limitations in available resources, the complexity of ERP systems, and the diversity of alternatives, selecting an ERP project is a time-consuming task. Several methods have been proposed for selecting a suitable ERP project or management information system [4 11]. The scoring method [5] is one of the most popular. Although it is intuitively simple, it does not ensure resource feasibility [9,10]. Teltumbde [4] suggested 10 criteria for evaluating ERP projects and constructed a framework based on the Nominal Group Technique (NGT) and the analytic hierarchy process (AHP) to make the final choice. Santhanam and *Corresponding author. Tel.: +886-3-5742655; fax: +886-3- 5722685. E-mail address: mjwang@ie.nthu.edu.tw (M.-J.J. Wang). Kyparisis [7,8] proposed a nonlinear programming model to optimize resource allocation and the interaction of factors; their model considered interdependencies of criteria in the information system selection process. Lee and Kim [9] combined the analytic network process (ANP) and a 0 1 goal-programming model to select an information system. However, these mathematical programming methods can not contain sufficient detailed attributes, above all, which are not easy to quantify, so that the attributes were restricted to some financial factors, such as costs and benefits. Furthermore, many of them involved only the consideration of internal managers, but do not offer a comprehensive process for combining evaluations of different data sources to select an ERP project objectively. Reports made by professional organizations and information collected from interviews with ERP suppliers should be considered in evaluating information of ERP projects. Professional organizations, such as research institutes and consulting companies, employ many experts to analyze information about ERP, including market share, vendor size, system performance, and other data. Their professional studies are very helpful to managers to have an overview of ERP systems and vendors. Furthermore, decision-makers can extract important attributes from these reports. However, the literature lacks studies on integrating the evaluation of objective external professional data sources and subjective internal interview data sources. This 0263-7863/03/$30.00 # 2003 Elsevier Ltd and IPMA. All rights reserved. doi:10.1016/s0263-7863(02)00064-9

162 C.-C. Wei, M.-J.J. Wang / International Journal of Project Management 22 (2004) 161 169 study aims to provide a new framework for integrating the two kinds of data evaluation for selecting a suitable ERP project. In reality, selecting a suitable ERP project involves multiple factors. Some of the measures, for example, the risk of the project, the functional fitness, and the ability of a vendor may not be precisely defined. Evaluation ratings under various attributes and the weights of the attributes are frequently assessed in linguistic terms, high, poor, among others. A fuzzy multiple-criteria decision-making method (FMCDM) is very useful in integrating various linguistic assessments and weights to evaluate ERP alternatives. This study proposes a comprehensive framework for selecting a suitable ERP project. Decision-makers can effectively integrate objective professional comments and subjective opinions of managers. A measure called, fuzzy ERP suitability index is used to account for the ambiguities involved in the evaluation of the appropriateness of ERP alternatives and the importance weights of attributes. An actual case in Taiwan is described to demonstrate the proposed method in practice. Step 9. Analyze the results of indices, l and k. Observe the change in the final ERP suitability and the final ranking value. Step 10. Select the ERP project with the maximum ranking value. Step 11. Implement the selected ERP project. Fig. 1 shows the comprehensive framework of the method. 2.1. Form a project team and conduct BPR The first step is to form a project team that consists of decision-makers, functional experts and senior representatives of user departments. In essence, an ERP project is not 2. Procedure for selecting an ERP project A systematic ERP selection algorithm, using twodimensional analysis and fuzzy set theory, is presented. The first dimension involves objective ratings of ERP project data in accordance with external professional reports. The second dimension requires assigning subjective ratings to ERP projects on the basis of data acquired in interviews. The objective and subjective evaluations are combined to obtain the final fuzzy ERP suitability. A stepwise procedure follows. Step 1. Form a project team and conduct the business process re-engineering (BPR). Step 2. Collect all possible information about ERP vendors and systems. Filter out unqualified vendors. Step 3. Establish the attribute hierarchy and assign weights to the attributes. Step 4. Interview vendors and collect detailed information. Step 5. Analyze the data obtained from the external professional reports to obtain the objective ERP suitability. Step 6. Assign subjective ratings to the ERP projects on the basis of data acquired in interviews to calculate the subjective ERP suitability. Step 7. Combine the evaluations of both data sources and aggregate the decision-making assessments to determine the final fuzzy ERP suitability. Step 8. Utilize the fuzzy integral value ranking method to obtain the rank of each ERP project. Fig. 1. Comprehensive ERP project selection procedure.

C.-C. Wei, M.-J.J. Wang / International Journal of Project Management 22 (2004) 161 169 163 only installing a new information technology system to replace the legacy system but also reshaping the business processes to overcome the challenges of dynamic market. BPR is necessary to be undertaken to rationalize and standardize the workflows of all business processes in advance. The functional characteristics of ERP are developed during the BPR. The project team can decompose the business objectives, for example costs reducing, quality and efficiency improving, and performance enhancing. Structuring the objectives involves organizing them, so that the project team can describe in detail what the company wants to achieve, and then incorporating these objectives appropriately into the decision model. 2.2. Collect information and eliminate unqualified alternatives Collect as wide a range of information as possible concerning ERP vendors and systems from professional magazines, exhibitions, yearbooks, the Internet, and other sources. Ensure the search includes less widely known vendors to make sure that some more feasible projects are not overlooked. The characteristics that reflect the system s requirements are transferred to a questionnaire or a checklist of the system specifications. The listed vendors are invited to provide information in response to these specific questions. Eliminate the clearly unqualified vendors and thereby reduce the number of candidates. 2.3. Establish attribute hierarchy Several researchers claim that both quantitative and qualitative attributes that can satisfy the routine operation under the strategies and goals of the company should be involved [5]. The aspects companies usually consider when selecting ERP project include: 1. The strategy of system to meet the business strategy and goals 2. The ability of system to support the business process 3. The technical requirements on which the system operates 4. The ability of vendor to support the system implementation and maintenance 5. The methodologies of business processes change and project management Thus, after organizing the factors addressed in prior studies [4 11], the attributes can be classified into three categories, as follows: 1. Project factors: attributes involved in project management, such as total cost, time of implementation, benefits, and risks; 2. Software system factors: features of the software and system, including strategic fitness and the function of ERP; 3. Vendor factors: attributes that pertain to vendors, such as ability and reputation. Fig. 2 depicts the attribute hierarchy for selecting the ERP projects. The main attributes x i (i=1, 2,..., 8) are summarized from the attributes used in professional reports. They are typical but not sufficiently detailed to evaluate ERP projects. Therefore, each main attribute is divided into sub-attributes, x ij (j=1, 2,..., n(i)), where n(i) denotes the number of sub-attributes of main attribute x i. The decision-makers evaluate only the professional data of the ERP projects under the main attributes. On the other hand, data are assessed from interviews under the sub-attributes and aggregate to the corresponding main attributes. Finally, the evaluations from both data sources are combined to obtain the conclusions. The weights of each attribute can be determined by direct assignment or pairwise comparisons. Decisionmakers use a set of five linguistic terms in weighting set, W, to describe the weights of each attribute, W={VL, L, M, H, VH}. In addition, assume that a set of linguistic terms, S={VP, P, F, G, VG}, is used to rate ERP projects to qualitative attributes. Table 1 specifies the triangular fuzzy numbers for these linguistic weights and values. 2.4. Hold interview meetings The vendors that remain on the list are asked to provide their proposals. A series of interviews with these vendors is scheduled. The project team arranges the schedule, agenda, scenarios, and questions for the vendors before the interviews are held. The scenarios describe how the ERP system exchanges data in a transaction and performs particular functions. The real company data, which were arranged in advance, were used to ask for the detailed demonstrations. The representatives of different user departments in the project team should provide the knowledge of their special processes to examine the vendor s demonstrations. Above all, decision-makers should direct the meetings and ensure that sufficient information about the ERP projects can be collected. 2.5. Aggregate external professional data Among the attributes, quantitative attributes are those that can be numerically evaluated. The values of these quantitative attributes are collected from the data by the ERP vendor provided or the data, which negotiated with the vendor. The crisp values must be converted into dimensionless values to ensure that these

164 C.-C. Wei, M.-J.J. Wang / International Journal of Project Management 22 (2004) 161 169 Fig. 2. ERP evaluation attribute hierarchy. values are compatible with the linguistic ratings of the qualitative attributes. Define B and C to be the set of benefit attributes and cost attributes, respectively. That is, in Fig. 2, B={benefit (x 3 )} and C={total cost (x 1 ), implementation time (x 2 )}. Let T ti (t=1, 2,..., m, i=1, 2, 3) represent the values assigned to ERP project P t under main attribute x i. Then, RT ti ¼ else, RT ti ¼ T ti P m t¼1 T ti ; if i 2 B ð1þ T ti 1 P m Tti 1 t¼1 ; if i 2 C ð2þ The advantage of using the above converting equation is that it can prevent any extreme attribute value of conscious or unconscious negligence after transformation. Assume that the value of a crisp rating is r, its triangular fuzzy number is (r, r, r). For different decision-makers the values of these quantitative data are the same. Then, let O~ ti ¼ RT ti, i =1, 2, 3, where O~ ti is the transferred fuzzy rating of ERP project P t under main attribute x i from the quantitative data. On the other hand, the attributes (from x 4 to x 8 ) that are difficult to quantify are reasonably treated as qualitative attributes. Decision-makers evaluate the professional data of the ERP projects under main qualitative Table 1 Linguistic variables describing weights of attributes and values of ratings Very low (VL) Very poor (VP) (0, 0, 0.3) Low (L) Poor (P) (0, 0.3, 0.5) Medium (M) Fair (F) (0.2, 0.5, 0.8) High (H) Good (G) (0.5, 0.7, 1.0) Very high (VH) Very good (VG) (0.7, 1.0, 1.0)

C.-C. Wei, M.-J.J. Wang / International Journal of Project Management 22 (2004) 161 169 165 attributes in linguistic terms set S. Assess O~ tih (t=1, 2,..., m; i=4, 5,...,8;h=1, 2,..., n), the linguistic rating of ERP project P t by decision-maker D h under main attribute x i from the professional data evaluation. Define W~ ih (i=1, 2,...,8;h=1, 2,..., n) as the linguistic weight assigned to main attribute x i by decision-maker D h. A mean operator is used to pool each rating by decision-makers, since the fuzzy average operation is a commonly used method of aggregation and is easy to understand [12]. Define, O~ ti ¼ ð1=nþ O~ ti1 O~ ti2... O~ tin ; ð3þ t ¼ 1; 2;...; m; i ¼ 4; 5;...; 8 and W~ i ¼ ð1=nþ W~ i1 W~ i2... W~ in ; i ¼ 1; 2;...; 8 where O~ ti is the average fuzzy rating of ERP project P t under main attribute x i from the professional data evaluation and W~ i is the average weight of main attribute x i. Then, combine the quantitative and qualitative evaluations to obtain the fuzzy objective suitability, O~ t, of ERP project P t by the following equation: O~ t ¼ h i ð1=8þ O~ t1 W~ 1 O~ t2 W~ 2... O~ t8 W~ 8 ; t ¼ 1; 2;...; m 2.6. Aggregate interview data Like the quantitative attributes transformation method we mentioned above, let S~ ti ¼ RT ti, i =1, 2, 3, where S~ ti is the transferred fuzzy rating of ERP project P t under main attribute x i from the quantitative data. After interviewing with ERP vendors, decision-makers assess the linguistic rating under the sub-attributes. S~ tijh (t=1, 2,..., m; i=4, 5,...,8;j=1, 2,..., n(i); h=1, 2,..., n) indicates the linguistic rating of ERP project P t by decision-maker D h for sub-attribute x ij from the evaluation of interview data. Let W~ ijh be the linguistic weight assigned to sub-attribute x ij by decision-maker D h. Define, S~ tij ¼ ð1=nþ S~ tij1 S~ tij2... S~ tijn ; t ¼ 1; 2;...; m; i ¼ 4; 5;...; 8; j ¼ 1; 2;...; nðiþ ð6þ and W~ ij ¼ ð1=nþ W~ ij1 W~ ij2... W~ ijn i ¼ 4; 5;...; 8; j ¼ 1; 2;...; nðiþ ð4þ ð5þ ð7þ where S~ tij is the average rating of ERP project P t under sub-attribute x ij from the evaluation of interview data, and W~ ij is the average weight of sub-attribute x ij. Then, aggregate these S~ tij to the corresponding main attributes. The aggregated rating, S~ ti, of ERP project P t under main qualitative attribute x i from the evaluation of interview data can be obtained, as in Eq. (8). h i S~ ti1 W~ i1 S~ ti2 W~ i2... S~ tinðiþ W~ inðiþ S~ ti ¼ ; W~ i1... W~ inðiþ t ¼ 1; 2;...; m; i ¼ 4; 5;...; 8 ð8þ Thus, integrate the quantitative and qualitative evaluations to obtain the fuzzy subjective suitability, S~ t,of ERP project P t by aggregating S~ ti with W~ i. Then, S~ t ¼ h ð1=8þ S~ t1 W~ 1 t ¼ 1; 2;...; m S~ t2 W~ 2... S~ t8 W~ 8 2.7. Combine objective and subjective suitabilities i ; Combine the evaluations of both data sources. The final fuzzy ERP suitability, R~ t, of ERP project P t can be obtained with an index l by Eq. (10). R~ t ¼ lo~ t ð1 lþs~ t ; 0 4l41; t ¼ 1; 2;...; m ð9þ ð10þ The value l can be manipulated to reflect the decision-makers attitude concerning the relative importance of both data sources. 2.8. Rank final fuzzy ERP suitability Selecting the appropriate ERP project depends on ranking the final fuzzy ERP suitability. Many fuzzy ranking methods have been proposed [13 15]. For simplicity and effectiveness in problem solving, the fuzzy ranking method with integral value proposed by Liou and Wang [13] is applied to rank the final fuzzy ERP suitability. According to the ranking method, index k indicates the degree of optimism of the decision-makers. A larger k represents a higher degree of optimism. Rank R~ t by the total integral value, respectively. Select the ERP project with the maximum total integral value. 2.9. Change indices, l and k, and make final decision A method that considers various trade-offs among projects is necessary for making the final decision, since

166 C.-C. Wei, M.-J.J. Wang / International Journal of Project Management 22 (2004) 161 169 the preferences of decision-makers and their environment are not always stable. Substitute various values of l and k into the model and analyze the changes in the final outcomes. Finally, the project team can choose the ERP project with the maximum total integral ranking value. 2.10. Implement the selected ERP project With the suitable ERP project selected, the project team should prepare to make a contract with the selected vendor. ERP project implementation is a complex exercise in technology innovation and business processes change management. A cautious, evolutionary, bureaucratic, and interactive implementation process based on change management and culture readiness can lead to successful ERP implementation. 3. An actual evaluation The proposed framework was used to select an ERP project at an electronics company in Science-Based Industrial Park in Taiwan. This medium-sized company designs and manufactures a variety of modular microwave systems. The company seeks to maintain its competitive advantage by improving the effectiveness of its global logistics and the efficiency of its response to customer demand. The stepwise procedure is presented in the following. Step 1. The top managers announced the launching of a series of E-business projects, including BPR, ERP, and an information communication system to increase the competitive advantage of the business and replace the legacy system. Executing BPR and analyzing workflow are essential in determining ERP system requirements. A steering committee of seven major managers was formed. It included the General Manager and managers of sales, manufacturing, R&D, MIS, finance, and purchasing to formulate the project plan and select an ERP system. Representatives of different user departments were also chosen to participate in the project team. Step 2. Information on 20 ERP vendors and systems was initially collected. Unfavorable alternatives were eliminated by asking a few questions, which were formulated by the specifications. Table 2 lists some of the questions. After preliminary screening, four local ERP vendors, P 1, P 2, P 3, and P 4, remained under consideration. Step 3. The system s requirements were translated into the corresponding attributes to formulate the hierarchical attribute structure. On the basis of the hierarchical attribute structure, the decision-makers assigned weights to main attributes, W~ i, and to subattributes, W~ ij, from the linguistic description set W. Table 2 Examples of screening questions Item Vendor size Complexity Cost vs. budget Domain knowledge Flexibility Covering requirements Fundamental Information technology Implementation methodology Service maintenance Consulting service Financial consideration Question 1. Does the vendor s size suit our company? 1. Is the ERP system too complex, or is it a good fit? 2. Does it fit our requirements, or is it overqualified? 1. What is the total cost of the project? 2. Can we accept the difference between the cost and budget? 1. What is the provider s target domain and market? 2. Does it match to our business needs? 1. Is the technology flexible and durable? 1. Does the system and its modules cover all our requirements? 1. What database and hardware can be supported by the system? 1. Does the vendor provide other information systems, such as SCM, MES, DW, CRM, and EC? 2. Does the vendor widely integrate its system with other partners information systems? 1. What is the implementation methodology? 2. Is it feasible and simple? 1. Who supports upgrades and maintenance? The software supplier or the reseller? 2. Does the vendor have any local service point or a branch company? 1. Does the vendor provide consulting services? 2. Does it cooperate with another consultant company? 1. How did the vendor perform financially over the last two years? 2. What is its current financial forecast? 3. Does it have any venture investment or warning signs?

C.-C. Wei, M.-J.J. Wang / International Journal of Project Management 22 (2004) 161 169 167 Table 3 Evaluation results ERP project P 1 P 2 P 3 P 4 O~ t (0.257, 0.515, 0.688) (0.216, 0.439, 0.681) (0.226, 0.455, 0.678) (0.167, 0.368, 0.640) S~ t (0.223, 0.466, 0.678) (0.196, 0.414, 0.672) (0.178, 0.380, 0.656) (0.138, 0.353, 0.617) R~ t (0.240, 0.491, 0.683) (0.206, 0.427, 0.677) (0.202, 0.418, 0.667) (0.153, 0.361, 0.628) IT 0:5ðR ~ T Þ 0.476 0.434 0.426 0.375 Step 4. Intensive interviews were scheduled with each of the four vendors. The team used a form to record data on the functionality, processes, local support, and finance. Most importantly, core processes and special operational features were assessed by considering demo scenarios and by examining each system s capacity to fulfill key demands. Step 5. The decision-makers evaluated the quantitative attributes (x 1, x 2,andx 3 ), using the information provided by the vendors. These attributes were rated for each ERP project by using Eqs. (1) and (2). On the other hand, the decision-makers evaluated the professional reports of ERP projects with respect to the main qualitative attributes (from x 4 to x 8 ) by using the linguistic ratings in scale set S. Then, aggregated the quantitative and qualitative evaluation with the corresponding weight to yield the objective ERP suitability O~ t by Eqs. (3), (4), and (5). Table 3 gives the fuzzy objective suitabilities of all ERP projects. Step 6. Let S~ ti ¼ RT ti, i=1,2,3. The linguistic rating of ERP project P t by decision-maker D h under subattribute x ij was S~ tijh (t=1, 2,..., m; i=4, 5,...,8;j=1, 2,..., n(i); h=1, 2,..., n), assessed by evaluating the interview data. Then, the aggregated rating S~ ti of ERP project P t under main attribute x i for evaluating the interview data can be obtained by conducting Eqs. (6), (7), and (8). Aggregate S~ ti and W~ i by averaging the corresponding products over all main attributes. The fuzzy subjective suitability S~ t of ERP project P t can be obtained by Eq. (9). Table 3 presents the fuzzy subjective suitabilities of all ERP projects. Step 7. Combine the results of both data sources, as in Eq. (10), with l=0.5. That is, the degrees of importance of both data source evaluations were equal. Table 3 summaries the fuzzy ERP suitability indices. Step 8. The total integral values R~ t of these final fuzzy ERP suitability indices were obtained by using the fuzzy integral value ranking method with k=0.5 (Table 3). Step 9. The rank order of the ERP projects was P 1, P 2, P 3, and P 4. The most suitable project was thus P 1. Fig. 3 shows the change in ranking when k=0.5 and l was varied from 0 to 1. Project P 1 was the best choice with any l. However, the order of P 2 and P 3 changed (P 3 >P 2 ) when l=0.8 was changed to 1.0. Therefore, P 3 Fig. 3. Total integral value change with 04l41 (k=0.5).

168 C.-C. Wei, M.-J.J. Wang / International Journal of Project Management 22 (2004) 161 169 Fig. 4. Total integral value change with 04k41 (l=0.5). was preferred over P 2 if the evaluation of the professional data source was considered to be more important than the evaluation of the interview data. Next, l=0.5 was fixed and k was varied. Fig. 4 shows P 1 always remaining the first preference. However, the values of P 2 and P 3 were very close to each other when k was varied. That is, P 2 and P 3 were evaluated as approximately equal. Step 10. The project team finally recommended ERP project P 1 as the most suitable selection for the company. The managers were satisfied with the framework that we presented. They can integrate the knowledge of external professional experts and their judgments to choose an ERP project. With a proper ERP project selected using the proposed framework, the selection time, as well as the cost of ERP project implementation and maintenance were reduced. After the ERP system was implemented, the managers were able to acquire information from the USA, mainland China, and Taiwan to respond to customers more efficiently and effectively. The mean lead-time was reduced by approximately 43% from 3 weeks to 12 days. 4. Conclusion This study has proposed a comprehensive framework for selecting an ERP project that combines data obtained from professional studies with that surveyed from interviews with vendors. A hierarchical attribute structure including project, software, and vendor factors has been provided for evaluating ERP projects. An integration model that uses the fuzzy average method and fuzzy integral ranking has been developed. The final decision is determined by the highest total integral value. The results of a real example indicate that the proposed framework is very useful for selecting a suitable ERP system selection. The proposed framework offers the following advantages in the ERP project. 1. It provides a comprehensive and systematic method. Decision-makers can easily select a suitable ERP project by following the stepwise procedure. 2. It provides a simple and intuitive procedure for integrating the subjective opinions of decisionmakers and the objective professional comments of external experts, thereby avoiding the use of a complex mathematical model. 3. The proposed algorithm considers not only quantitative data but also linguistic data. Managers can assess various attributes of a system, particularly in an ill-defined situation, by using linguistic or quantitative values. It can be refined since it flexibly accommodates additional considerations. 4. The values of l and k can be changed to determine related changes in the prioritization of projects, with regard to the current business situation, to solidify the final decision.

C.-C. Wei, M.-J.J. Wang / International Journal of Project Management 22 (2004) 161 169 169 References [1] Kumar V, Maheshwari B, Kumar U. Enterprise resource planning systems adoption process: a survey of Canadian organizations. International Journal of Production Research 2002;40: 509 23. [2] Wilson F, Desmond J, Roberts H. Success and failure of MRPII implementation. British Journal of Management 1994;5:221 40. [3] Hicks DA, Stecke KE. The ERP maze: enterprise resource planning and other production and inventory control software. IIE Solutions 1995;27(8):12 16. [4] Teltumbde A. A framework of evaluating ERP projects. International Journal of Production Research 2000;28:4507 20. [5] Ptak CA. ERP tools, techniques, and applications for integrating the supply chain. St. Lucie Press, 2000. [6] Chen K, Gorla N. Information system project selection using fuzzy logic. IEEE transactions on systems, man, and cybernetics part A: Systems and Humans 1998;28:849 55. [7] Santhanam R, Kyparisis GJ. A multiple criteria decision model for information system project selection. Computers & Operations Research 1995;22(8):807 18. [8] Santhanam R, Kyparisis GJ. A decision model for interdependent information system project selection. European Journal of Operational Research 1996;89:380 99. [9] Lee JW, Kim SH. An integrated approach for interdependent information system project selection. International Journal of Project Management 2001;19:111 8. [10] Badri MA, Davis D, Davis D. A comprehensive 0 1 goal programming model for project selection. International Journal of Project Management 2001;19:243 52. [11] Lai VS, Robert TP, Wong BK. Software selection: a case study of the application of the analytical hierarchical process to the selection of a multimedia authoring system. Information Management 1999;36:221 32. [12] Lee DH, Park D. An efficient algorithm for fuzzy weighted average. Fuzzy Sets and Systems 1997;87:39 45. [13] Liou TS, Wang MJ. Ranking fuzzy numbers with integral value. Fuzzy Sets and Systems 1992;50:247 55. [14] Chen SH. Ranking fuzzy numbers with maximizing set and minimizing set. Fuzzy Sets and Systems 1985;17:113 29. [15] Buckley JJ. Ranking alternatives using fuzzy numbers. Fuzzy Sets and Systems 1985;15:21 31.