Determining the relative e ciency of MBA programs using DEA

Transcription

1 European Journal of Operational Research 125 (2000) 656±669 Theory and Methodology Determining the relative e ciency of MBA programs using DEA Amy Colbert a, Reuven R. Levary a, *, Michael C. Shaner b a Department of Decision Sciences and MIS, School of Business and Administration, Saint Louis University, 3674 Lindell Blvd., St. Louis, MO 63108, USA b Department of Management, School of Business and Administration, Saint Louis University, 3674 Lindell Blvd., St. Louis, MO 63108, USA Received 1 February 1998; accepted 1 April 1999 Abstract Data envelopment analysis (DEA) is used here to determine the relative e ciency of 24 top ranked US MBA programs. E ciency scores were determined using three output sets of the MBA programs: output that measured student, output that measured recruiter and output that measured both. DEA is also used here to determine the relative e ciency of three foreign MBA programs as compared to several top ranking US MBA programs. The publicized ranking of MBA programs by national magazines has a signi cant impact on corporate recruiters, potential MBA students and on business schools themselves. A new ranking based on DEA will more completely and accurately represent MBA programs. Further, DEA will make it possible to more fairly compare speci c programs. Ó 2000 Elsevier Science B.V. All rights reserved. Keywords: MBA programs; Higher education; DEA 1. Introduction Masters of Business Administration (MBA) programs are charged with satisfying many customers with limited resources. Students who enroll in the top MBA programs expect to learn skills that will enable them to operate the world's most successful companies or to create such companies of their own. Students demand a curriculum that is varied but applicable to the business world. After * Corresponding author. Tel.: ; fax: address: [email protected] (R.R. Levary). completing the program, students expect to be placed in challenging positions with lucrative starting salaries. MBA programs must satisfy the needs of corporate recruiters as well as the needs of the students. The recruiters represent companies from around the world. They expect students from the top MBA programs to have analytical skills that will enable them to solve comprehensive business problems. They also expect students to have learned to perform well as part of a team. Additionally, they expect them to have developed a global view of the business world. If an MBA program has satis ed these needs, corporate /00/$ - see front matter Ó 2000 Elsevier Science B.V. All rights reserved. PII: S ( 9 9 )

2 A. Colbert et al. / European Journal of Operational Research 125 (2000) 656± recruiters will hire their graduates and will provide them with lucrative starting salaries and signing bonuses. The biennial ranking of business schools that is prepared by Business Week magazine (Byrne, 1997) is based on surveys of business school graduates and corporate recruiters. The individual programs are rst given both a corporate ranking and a graduate ranking and then the two rankings are combined into a single overall ranking. This widely reported ranking is monitored by MBA students, corporate recruiters and business schools alike. In this study, data from the Business Week survey (Byrne, 1997) are used to determine the relative e ciency of the top MBA programs in the United States. This measure of e ciency is based on the level of output produced for each unit of input. Because MBA programs must focus on satisfying the needs of both students and corporate recruiters, the relative e ciency of the programs is determined here using three output sets: outputs that measure student, outputs that measure recruiter, and output that combines the two. E ciency scores calculated using multiple output types for the model are compared with the e ciency scores based on a single type of output for that model. Data envelopment analysis (DEA) as a tool for determining the relative e ciency of operating units is overviewed in Section 2. Other methods of determining e ciency are described in Section 3. Section 4 discusses the utility of DEA in evaluating operating units in the higher education sector. The results and analysis of evaluating the relative e ciency of MBA programs within the United States is provided in Section 5. In Section 6, the relative e ciency of three foreign MBA programs is compared with the relative e ciency of several top ranking US MBA programs. A summary and conclusions are the subject of the nal section. 2. Using DEA to determine relative e ciency Using DEA, the relative e ciency of decision making units (DMUs) that use multiple inputs to produce multiple outputs may be calculated. The relative e ciency of a DMU is calculated using a ratio de nition of e ciency (Charnes et al., 1978). This ratio generalizes the single output to single input de nition to multiple outputs and inputs without the use of pre-assigned weights. The weights used for each DMU are those which maximize the ratio between the weighted output and weighted input. These weights are determined in such a way that no method of aggregating the inputs and outputs, such as value or market price, is necessary. DEA is an analytical procedure developed by Charnes et al. (1978) for measuring the relative e ciency of DMUs that perform the same type of functions and have identical goals and objectives. Decision making units include departments, sections, branches, and divisions of organizations belonging to the same business sector. If the relative e ciency of a set of DMUs performing the same type of function is to be evaluated, the DMUs must use the same type of input to produce the same type of output. Each DMU in a given set can then be ranked according to how e ciently it utilizes its inputs to produce its outputs. When the combined number of inputs and outputs approaches the total number of DMUs in a set, however, DEA may be problematic. Under such circumstances, one must be very cautious interpreting e ciency scores (Charnes et al., 1985a). Numerous re nements of DEA now enhance its analytical e ectiveness. The ``window analysis'' concept (Charnes and Cooper, 1985) was incorporated into DEA to enable it to trace the performance of each DMU over time. Tracing performance over time is done by evaluating the DMUs at di erent time periods. As ``window analysis'' requires that a DMU be de ned for each time period used in the analysis, it substantially increases the volume of calculations. Thanassoulis and Dyson (1992) developed several DEA based models that can be used to estimate alternative input-output target levels and are helpful in rendering relatively ine cient organizational units e cient. The DEA model applied in this study was developed by Banker et al. (1984) and has been used in many applications (e.g., Bessent and Bessent,

3 658 A. Colbert et al. / European Journal of Operational Research 125 (2000) 656± ; Seiford, 1996). Chang and Guh (1991) pointed out some problems of using this model. A comprehensive bibliography of DEA is given in Seiford (1996). The e ciency measure for each DMU ranges from 0 to 1. A DMU with an e ciency value of 1 is considered most e cient. An e ciency value smaller than 1 indicates the degree of relative e ciency. One possible explanation of a DMU's ine ciency is that some of its inputs are not utilized fully. E cient DMUs achieve greater output per unit of input when compared with ine cient DMUs. By identifying unutilized resources, DEA can provide a rm's management with some information regarding causes of ine ciency. In order to formulate the DEA model let us assume that n-mba programs are to be evaluated based on m inputs and s outputs. Let y rj be a known level of the rth output of program j r ˆ 1; 2;...; s; j ˆ 1; 2;...; n and x ij be a known level of the ith input to program j i ˆ 1; 2;...; m. Each MBA program is assigned a weight w j j ˆ 1; 2;...; n for its input and output. A hypothetical composite MBA program can then be de ned using weighted inputs and outputs of the programs being evaluated. The weights w j are the model decision variables. The e ciency of MBA program k relative to the composite MBA program can be determined by solving the following linear programming problem: min h k 1 subject to: X n jˆ1 X n jˆ1 X n jˆ1 w j ˆ 1; 2 w j y rj P y rk ; r ˆ 1; 2;...; s; 3 w j x ij 6 x ik h k ; i ˆ 1; 2;...; m; 4 h k ; w j j ˆ 1; 2;...; n P 0; 5 where h k is the relative e ciency of MBA program k. Minimizing the relative e ciency of MBA program k is equivalent to minimizing the inputs of the composite MBA program. Constraints (2) ensures that the sum of the weights is equal to 1. Constraints (3) ensure that each output level of the composite MBA program is at least as high as the output level of MBA program k. Constraints (4) ensure that each input level of the composite MBA program is at most as high as its input capacity. 3. Other methods of determining e ciency DEA is not the only method that can be used to determine e ciency. Ratio analysis is another common method. While the DEA is a ratio model (Charnes et al., 1978), we refer here to a ratio analysis method that is not a DEA based method. Using ratio analysis, a ratio comparing outputs to inputs is computed. A simple ratio compares one measure of input to one measure of output. For example, an educational unit might use total cost per student enrolled to measure e ciency. This measure treats all students as if they were identical. The di erences in the amount of knowledge gained or starting salaries earned are not considered. Multiple inputs and outputs may be incorporated into ratio analysis by calculating multiple ratios. However, this makes it di cult to determine overall e ciency. A measure of overall e ciency can be computed by aggregating all inputs and outputs. This requires assigning a weight to each input and output. While such weights may be determined according to the value or market price of each input and output, this information is not always available. When the market value of each input and output is missing, one may consider using the Cook and Kress (1990) approach to determine the set of weights. Cook and Kress developed a methodology for aggregating preference ranking and applied their approach to aggregate votes in a preferential election. Their model determines for each candidate i the best set of weights w j to apply to the jth place standing m ij for each candidate. Multiple regression is another method for determining e ciency. Using multiple regression, output level is modeled as a function of various input levels. Operating units that are relatively e cient lie above the modeled relationship and

4 A. Colbert et al. / European Journal of Operational Research 125 (2000) 656± have positive residuals. Operating units that are relatively ine cient lie below the modeled relationship and have negative residuals. This method has several drawbacks. First, because singleequation multiple regression can model only one output level, a single output measure must be determined or all outputs must be arti cially combined into a single indicator. Multiple-equation regression models can be used when an operating unit has multiple outputs. Like multiple ratios, however, this method does not produce an overall measure of e ciency. Multiple residuals provide di erent measures of the operating unit's e ciency. Another drawback of regression analysis is that it compares e ciency with average performance rather than with the best performance. Additionally, ``regression analysis requires the parametric speci cation of a production function, that is, an equation detailing how inputs are combined to produce outputs'' (Sexton, 1986, p. 9). This is often di cult because such a function may be unknown for the industry in question. Several studies combined DEA with regression analysis to evaluate operating units which have multiple inputs and outputs. Cooper and Tone (1997) used simulation to study a combined DEAregression model. Friedman and Sinuany-Stern (1997) developed a methodology using canonical correlation analysis to provide a full rank scaling for all the units. Their methodology closed the gap between the frontier approach of DEA with the average tendencies of statistics. Compared to the methods mentioned, DEA has several advantages. Multiple inputs and outputs can be used in the DEA model. The weights that will be used to aggregate inputs and outputs are determined using linear programming. No decisions need be made regarding the relative importance of each input and output. With DEA, each operating unit's e ciency is compared to an ``ideal'' operating unit rather than to average performance. DEA has some limitations as well, however. As with any other method of determining e ciency, all inputs and outputs must be speci ed and measured. Failure to include a valid input or output or inclusion of an invalid input or output will bias the results. Additionally, DEA can measure ``relative'' e ciency, but not ``absolute'' e ciency. It compares an operating unit to a subset of peers and not to a theoretical maximum performance. 4. Using DEA to evaluate higher education operating units DEA has been used to determine the relative e ciency of University departments in several studies (e.g., Ahn, 1987; Ball and Wilkinson, 1992; Beasley, 1995). DEA is particularly useful in evaluating educational units because inputs and outputs are combined using a priori weight. These weights are determined using linear programming and ``are not the values of inputs and outputs in any economic sense'' (Sexton, 1986, p. 10). Because the economic value of many of the inputs and outputs of educational units is di cult to determine, the DEA model is a good choice. With its multiple inputs and outputs, the relative e ciency of educational units can be calculated. Nonetheless, the speci cation of inputs and outputs is often di cult. Many of the outputs of educational units are not measurable. For example, it is di cult to measure a university's contribution to the surrounding community. Other outputs, such as the increase in a student's knowledge may be measured (e.g., using entrance and exit exams) but the accuracy of these measures is questionable. Additionally, this data may not be readily available. Because of the di culties inherent in the speci cation of inputs and outputs for educational units, some studies have examined the e ects of variation in inputs and outputs on e ciency scores. Sinuany-Stern et al. (1994) used DEA to determine the relative e ciency of 21 departments at Ben-Gurion University. Operational expenditures and faculty salaries were used as inputs. Grant money, number of publications, number of graduate students and number of credit hours o ered were used as outputs. Fourteen of the departments were found to be ine cient. Sinuany-Stern et al. (1994) also tested the e ects of variations in inputs and outputs on e ciency scores. In one trial, one output was deleted from the original model. The

5 660 A. Colbert et al. / European Journal of Operational Research 125 (2000) 656±669 output was chosen for deletion because no departments were relatively ine cient in that output. In this trial, two additional departments became ine cient. The DEA model was run again with the two inputs combined. Again, two additional departments became ine cient. Ahn and Seiford (1993) used DEA to determine the relative e ciency of 153 doctoral-degreegranting institutions of higher learning (IHLs). Of these, 104 were public and 49 were private. The purpose of the study was to determine the e ect of di erent sets of output variables on the relative e ciencies of public and private institutions. Public IHLs are often funded based on an enrollment-related output measure. For this reason, Ahn and Seiford predicted that public IHLs would be more e cient when enrollment-related outputs were considered and private IHLs would be more e cient when less closely monitored outputs were considered. This hypothesis was tested using multiple variable sets. In one trial, faculty salaries, physical investment, and overhead expenses were used as input variables. Undergraduate full-time equivalent students (FTEs) and graduate FTEs were used as output variables. Using these enrollment-related output variables, public IHLs were found to be more e cient than private IHLs. A second trial used faculty salaries, physical investment, overhead expenses, undergraduate FTEs and graduate FTEs as inputs. Undergraduate degrees, graduate degrees, and grants were used as output variables. Using these less closely monitored output variables, private universities were found to be more e cient. Both studies cited above show that the choice of output variables had an impact on the e ciency scores of the operating units in question. Sinuany- Stern et al. (1994) showed that a reduction in the number of input and output variables used, whether by deletion or combination, caused e ciency scores to decrease or remain the same. Ahn and Seiford (1993) showed that the type of output variables used had an impact on the e ciency scores of the operating units in question. Breu and Raab (1994) used data from a ranking of the top 25 national universities by US News and World Report to calculate relative e ciency. To determine the ranking of these 25 universities, US News and World Report used 12 performance indicators that measured reputation, student selectivity, faculty resources, nancial resources and student. Breu and Raab used four of the performance indicators from the student selectivity, faculty resources and nancial resources categories as input measures. Input measures included: percentage of faculty with doctorates, faculty to student ratio, educational and general expenditures per student and average or midpoint SAT/ ACT scores. A fth input, tuition charge per student, was also included as an input measure. The outputs used by Breu and Raab in their DEA model were graduation rate and freshman retention rate. The two measures were used to indicate student in the US News and World Report ranking. The US News and World Report poll based their ranking on the of only one customer group ± students. The operating units used in this study are 24 of the top 25 MBA programs in the Business Week ranking of MBA programs in the United States (Byrne, 1997). The data for the study were taken from the surveys and other data collected by Business Week to rank these MBA programs. Business Week, however, recognizes that MBA programs must satisfy numerous customer groups, the primary ones being students and recruiters. Recognizing the need to satisfy both groups, e ciency scores are determined in this study using three di erent output sets. Outputs in the rst trial measure both student and recruiter. Outputs in the second trial measure only student. In the third trial, e ciency is determined using outputs that measure only recruiter. 5. Results and analysis of evaluating the relative e ciency of MBA programs E ciency scores vary based on the type of output used. It is expected that the number of ef- cient MBA programs will be higher in trial one where both types of output are included in the same model. Sexton (1986, p. 10) explained that ``DMUs will place higher weights on the inputs

6 A. Colbert et al. / European Journal of Operational Research 125 (2000) 656± that they use least and on the outputs that they produce most''. Because of this, programs that more e ciently produce student outputs will place higher weights on these outputs while programs that more e ciently produced recruiter outputs will place higher weights on those outputs. As a result, more programs will be relatively e cient when both types of output are included in the same model. In trial one, two output measures showing student were used: y 1 ± percentage of alumni who donate money to the program, y 2 ± student with teaching, curriculum and placement. The value of output y 2 is based on the survey of graduates done by Business Week (Byrne, 1997). Additionally, two output measures were used to represent recruiter : y 3 ± average salary of graduates, y 4 ± recruiter with analytical skills, team work, and global view. The recruiter scores were taken from the Business Week survey of corporate recruiters. Because the resources available to MBA programs must be used to satisfy all customer groups, the same set of inputs was used in each test of relative e ciency. These included: x 1 ± faculty to student ratio, x 2 ± average GMAT score of students in the program, x 3 ± number of electives o ered. The relative e ciency of 24 programs was determined using the three inputs and four outputs described above. The inputs and outputs for each program are given in Tables 1 and 2. Stanford University was excluded from the sample because data on the percentage of alumni who donate money to the program was not available. The DEA model represented by relations (1)±(5) was utilized. E ciency scores are shown in Table 3. The MBA programs listed in Tables 1±3 are ordered identically to the way in which they are ordered in Business Week (i.e., University of Pennsylvania's Wharton is ranked at the top). As the above results show, only eight of the 24 programs were found to be ine cient. The e ciency scores ranged from to 1.0. In trial two, e ciency scores were calculated using only outputs that measure student. These outputs were as follows: y 1 ± percentage of alumni who donate money to the program, y 2 ± student with teaching, y 3 ± student with curriculum, y 4 ± student with placement. In trial three e ciency scores were calculated using only the outputs that measure recruiter. These outputs were as follows: y 1 ± average salary of graduates, y 2 ± recruiter with analytical skills, y 3 ± recruiter with team work skills, y 4 ± recruiter with graduates' global view. Satisfaction scores were included separately rather than as an average so that the number of outputs in trials two and three would be the same as the number of outputs in trial one. Additionally, the same inputs were used in these two trials as were used in the rst trial. The results from trials two and three are given in Table 3. The results show that 13 programs were ine cient in achieving student related outputs and nine programs were ine cient in achieving recruiter related outputs. This supports the hypothesis that more programs will be e cient when a combination of two types of output are used. Analysis of each program's relative e ciency scores provides further insight into the impact of combining two types of output into one model. The number of e cient programs increases when two types of output are used in the same model because programs that more e ciently produce one type of output will place a higher weight on that type of output. Based on this, it would seem that a program that e ciently produces either student outputs or recruiter outputs can choose weights that will produce an e ciency score of one when both types of outputs are considered in the same model. The results above show that 10 programs are e cient in achieving either student outputs or recruiter outputs, but not both. Of these, seven were able to choose weights in trial one that resulted in an e ciency score of one.

7 662 A. Colbert et al. / European Journal of Operational Research 125 (2000) 656±669 Table 1 Input data for the top 25 MBA Programs in the United States a MBA program Number of faculty Number of students Faculty to student ratio GMAT score University of Pennsylvania (Wharton) University of Michigan Northwestern University (Kellogg) Harvard University University of Virginia (Darden) Columbia University Stanford University University of Chicago Massachusetts Institute of Technology (Sloan) Dartmouth College (Tuck) Duke University (Fugua) University of California at Los Angeles (Anderson) University of California at Berkeley (Haas) New York University (Stern) Indiana University Washington University (John M. Olin) Carnegie Mellon University Cornell University (Johnson) University of North Carolina (Kenan-Flagler) University of Texas University of Rochester (Simon) Yale University Southern Methodist University (Cox) Vanderbilt University (Owen) American Graduate School of International Management (Thunderbird) a All data was taken from Byrne (1997) and from the companion web site ( Number of electives Two additional trials were run on the 24 programs tested in trials one through three. Sinuany- Stern et al. (1994) found that e ciency scores either decrease or remain the same when the number of input or output variables included in the DEA model is reduced. In trials four and ve, student scores and recruiter scores were combined into a single indicator as in trial one. This reduced output to two in both trials. The output for trial four was as follows: y 1 ± percentage of alumni who donate money to the program, y 2 ± student with teaching, curriculum, and placement. The output for trial ve was as follows: y 1 ± average salary of graduates, y 2 ± recruiter with analytical skills, team work, and global view. The e ciency scores from trials four and ve are included in Table 3. Comparing trials two and four, the e ciency scores for 12 programs decreased and the e ciency scores for the remaining 12 programs remained the same when the number of outputs was reduced. When trials three and ve were compared, the ef- ciency scores for nine programs decreased while the e ciency scores for the remaining fteen programs remained the same. Analysis of Table 3 and the solutions to the LP problems formulated by relations (1)±(5) suggests that MBA programs having an e ciency score of 1 also had zero slacks and therefore they are Pareto± Koopmans e cient (see Charnes et al., 1985b; Chang and Kao, 1992). Of those having an e ciency score of one, a single program may have used less of one or two resources but never less of

8 A. Colbert et al. / European Journal of Operational Research 125 (2000) 656± Table 2 Output data for the top 25 MBA programs in the United States a MBA program Measures of student Measures of recruiter University of Pennsylvania (Wharton) Percentage of alumni who donate Student with teaching Student with curriculum Student with placement score starting salary ($) Recruiter with analysis Recruiter with team players Recruiter with world view , University of Michigan , Northwestern University , (Kellogg) Harvard University , University of Virginia , (Darden) Columbia University , Stanford University N/A , University of Chicago , Massachusetts Institute of , Technology (Sloan) Dartmouth College (Tuck) , Duke University (Fugua) , University of California at , Los Angeles (Anderson) University of California at , Berkeley (Haas) New York University , (Stern) Indiana University , Washington University , (John M. Olin) Carnegie Mellon University , Cornell University , (Johnson) University of North , Carolina (Kenan-Flagler) University of Texas , score

9 664 A. Colbert et al. / European Journal of Operational Research 125 (2000) 656±669 Table 2 (Continued) MBA program Measures of student Measures of recruiter University of Rochester (Simon) Percentage of alumni who donate Student with teaching Student with curriculum Student with placement score starting salary ($) Recruiter with analysis Recruiter with team players Recruiter with world view , Yale University , Southern Methodist , University (Cox) Vanderbilt University (Owen) American Graduate School of International Management (Thunderbird) , , a All data was taken from Byrne (1997) and from the companion web site ( score

10 A. Colbert et al. / European Journal of Operational Research 125 (2000) 656± Table 3 E ciency scores of MBA programs for several combinations of outputs and inputs a MBA program E ciency score trial 1 E ciency score trial 2 E ciency score trial 3 E ciency score trial 4 E ciency score trial 5 University of Pennsylvania (Wharton) University of Michigan Northwestern University (Kellogg) Harvard University University of Virginia (Darden) Columbia University University of Chicago Massachusetts Institute of Technology (Sloan) Dartmouth College (Tuck) Duke University (Fugua) University of California at Los Angeles (Anderson) University of California at Berkeley (Haas) New York University (Stern) Indiana University Washington University (John M. Olin) Carnegie Mellon University Cornell University (Johnson) University of North Carolina (Kenan-Flagler) University of Texas University of Rochester (Simon) Yale University Southern Methodist University (Cox) Vanderbilt University (Owen) American Graduate School of International Management (Thunderbird) a The MBA programs are ordered identically to the way in which they are ordered in Business Week. all three resources when compared to the other programs. The results given in Table 3 indicate that Columbia University, Fugua, Kenan-Flagler, University of Texas, and Owen are ine cient when both student and recruiter measures of are considered. Wharton, Harvard University, Darden, Sloan, Stern, Carnegie Mellon and Johnson are found to be ine cient with regard only to measures of student. The University of California at Los Angeles, Haas and John M. Olin are found to be ine cient with regard to recruiter. Administrators at the e cient MBA programs (University of Michigan, Kellogg, University of Chicago, Tuck, Indiana University, Simon, Cox, and Thunderbird) can strive to raise to the level of individual input measures to approach the highest existing levels (i.e., faculty to student ratio of at Kenan- Flagler; average GMAT score of 685 at University of Chicago; and number of 286 electives at Columbia University). Administrators of ine cient programs can strive to simultaneously improve their level of e ciency and raise the level of individual input measures. 6. Determining the e ciency of MBA programs outside the united states using DEA As technology draws us into a global community, it is important to consider not only the relative e ciency of MBA programs in the United States, but also the relative e ciency of MBA programs throughout the world. Although DEA is an appropriate tool for this task, barriers do exist. First, data for MBA programs outside the United States is not readily available. As was

11 666 A. Colbert et al. / European Journal of Operational Research 125 (2000) 656±669 mentioned earlier, the data used in this study was taken form the Business Week ranking of MBA programs (Byrne, 1997). MBA programs outside the United States, however, are not included in the ranking. The data available for these programs was not as complete as the data provided regarding MBA programs within the United States. Further, MBA programs outside the United States may have di erent objectives than programs within the United States. For this reason, it may be di cult to specify a uniform set of outputs for all programs. It may be di cult to construct a uniform set of inputs as well. Many MBA programs outside the United States, for example, do not require students to take the GMAT admittance exams. Despite these barriers, an analysis of relative e ciency was completed using seven MBA programs within the United States and three MBA programs outside the United States. The seven programs within the United States are the top seven programs in the Business Week survey for which the input and output data was available. The MBA programs outside the United States were pro led in the Business Week Guide to the Best Business Schools (Byrne, 1997) but were not included in the ranking. The data used in the DEA analysis is given in Table 4. The inputs that were used in this model included: x 1 ± faculty to student ratio, x 2 ± average GMAT score of students in the program, x 3 ± average number of years of work experience for students in the program. The outputs that were used in this model included: y 1 ± percentage of alumni who donate money to the program, y 2 ± average starting salary of graduates. It should be noted that output y 1 was used as a measure of student in the analysis above and output y 2 was used as a measure of recruiter. Based on the conclusions drawn above, it was expected that programs ef- cient in satisfying either of these two mentioned customer groups would be e cient according to this model as well. These expectations were con rmed in the results. The e ciency scores from this analysis are included in Table 4. Only one of the 10 programs was found to be ine cient. It is possible that more programs would have been shown to be ine cient if less types of output had been considered in the model. It is also possible that a greater number of ine cient programs would have been identi ed had a wider range of MBA programs been considered (i.e., not only highly regarded programs). Analyzing the results in Table 4 indicate that the e cient MBA programs are also Pareto±Koopmans e cient. 7. Summary and conclusions In this study, the e ect of various types of output sets on e ciency scores was examined. The relative e ciency of 24 MBA programs from Business Week's top 25 programs in the United States was determined using three output sets. The rst trial included two outputs that measured student and two outputs that measured recruiter. The second and third trials segregated these output types. Trial two used only outputs that measured student while trial three used only outputs that measured recruiter. The hypothesis that more programs would have an e ciency score of one when two types of output were used in the same model was con rmed in this analysis. This result was expected given that each operating unit chooses the weights to be placed on inputs and outputs in the model. Units understandably place higher weights on outputs that are e ciently produced in an e ort to achieve e ciency scores of one. This study replicated the result found by Sinuany-Stern et al. (1994). They found that a reduction in the number of input or output variables used caused e ciency scores to decrease or remain the same. In trials two and three, three measures of student and three measures of recruiter were taken from the Business Week surveys and used as output measures. In trials four and ve, these survey results were combined into one measure of student

12 A. Colbert et al. / European Journal of Operational Research 125 (2000) 656± Table 4 Input and output data and e ciency scores for some US and foreign MBA programs a MBA program Number of faculty Number of students Faculty to student ratio GMAT score years work experience Percentage of alumni who donate money to the program Averge starting salary ($) University of Pennsylvania , (Wharton) University of Michigan , Northwestern University (Kellogg) , Harvard University , University of Virginia (Darden) , Columbia University , University of Chicago , INSEAD (France) , London Business School (England) , Western Business School (Canada) , a All data was taken from the Byrne (1997) and from the companion web site ( E ciency score

13 668 A. Colbert et al. / European Journal of Operational Research 125 (2000) 656±669 and one measure of recruiter. This reduced the number of outputs in trials four and ve to two. A decrease in e ciency scores was noted as a result. This study's analysis of relative e ciency of 24 MBA programs had limitations. Data availability was limited, for example. While the Business Week ranking (Byrne, 1997) provided a reliable source of data, the data was available only for the top MBA programs. In this study, e ciency scores were all above 0.9. A more balanced sample of MBA programs might have resulted in a wider range of e ciency scores. Because of limited data regarding MBA programs outside the United States, only three foreign MBA programs were included as part of this e ciency analysis. The foreign MBA programs were part of a set of ten programs that included seven of the top US programs. Analysis indicated that all three foreign programs and all but one US program were e cient. Barriers to calculating the relative e ciency of programs outside the United States were noted. Methods for overcoming these barriers and using DEA to determine the relative e ciency of MBA programs around the world warrant further research. The results of this study highlight the importance of the inputs and outputs used in determining relative e ciency. Varying the inputs and outputs used will a ect the calculated e ciency scores. It is advisable to carefully consider the objectives of the MBA programs in question when determining which input and output measures to use in DEA. The publicized ranking of MBA programs by national magazines has a signi cant impact on corporate recruiters, potential MBA students and the business schools themselves. New rankings based on DEA will result in a more complete, accurate representation of MBA programs. Further, DEA will provide better insight as to how the speci c programs compare with each other. References Ahn, T.S., E ciency and related issues in higher education: A data envelopment analysis approach. Ph.D. dissertation, Graduate School of Business, University of Texas, Austin, TX. Ahn, T., Seiford, L.M., Sensitivity of DEA to models and variable sets in a hypothesis testing setting: The e ciency of university operations. In: Yuji Ijiri (Ed.), Creative and Innovative Approaches to the Sciences of Management. Quorum Books, Westport. Ball, R., Wilkinson, R.H., Measuring the performance of higher education institutions: The application of data envelopment analysis. In: M. Wright (Ed.), Proceedings of the Thirteenth International Forum of the European Association for Institutional Research, pp. 219±234. Banker, R.D., Charnes, A., Cooper, W.W., Some models for estimating technical and scale ine ciencies in data envelopment analysis. Management Science 30 (9), 1078± Beasley, J.E., Determining teaching and research e ciencies. Journal of the Operational Research Society, 441±452. Bessent, A.M., Bessent, E., Determining the comparative e ciency of schools through data envelopment analysis. Educational Administration Quarterly 16, 57±75. Breu, T.M., Raab, R.L., E ciency and perceived quality of the nationõs `top 25' national universities and national liberal arts colleges: An application of data envelopment analysis to higher education. Socio-Economic Planning Science 28, 33±45. Byrne, J.A., Business Week Guide to the Best Business Schools, 5th ed. McGraw-Hill, New York. Chang, K.P., Guh, Y.Y., Linear production functions and the data envelopment analysis. European Journal of Operational Research 52, 215±223. Chang, K.P., Kao, P.H., The relative e ciency of public versus private municipal bus rms: An application of data envelopment analysis. The Journal of Productivity Analysis 3, 67±84. Charnes, A., Cooper, W.W., A preface to topics in data envelopment analysis. Annals of Operations Research 2, 59± 94. Charnes, A., Cooper, W.W., Rhodes, E., Measuring the e ciency of decision making units. European Journal of Operational Research 2, 429±444. Charnes, A., Cooper, W.W., Golany, B., Seiford, L., Stutz, J., 1985a. Foundations of data envelopment analysis for Pareto±Koopmans e cient empirical production functions. Journal of Econometrics 30, 91±107. Charnes, A., Cooper, W.W., Lewin, A.Y., Morey, R.C., Rousseau, J., 1985b. Sensitivity and stability analysis in DEA. Annals of Operations Research 2, 139±156. Cook, W.D., Kress, M., A data envelopment model for aggregating preference ranking. Management Science 36 (11), 1302±1310. Cooper, W.W., Tone, K., Measures of ine ciency in data envelopment analysis and stochastic frontier estimation. European Journal of Operational Research 99, 72±88. Friedman, L., Sinuany-Stern, Z., Scaling units via the canonical correlation analysis in the DEA context. European Journal of Operational Research 100, 629±637.

14 A. Colbert et al. / European Journal of Operational Research 125 (2000) 656± Seiford, L.M., Data envelopment analysis: the evolution of the state of the art 1978±1995. The Journal of Productivity Analysis 7, 99±137. Sexton, T.R., The methodology of data envelopment analysis. In: R.H. Silkman (Ed.), Measuring E ciency: An Assessment of Data Envelopment Analysis. Jossey-Bass, San Francisco. Sinuany-Stern, Z., Mehrez, A., Barboy, A., Academic departments e ciency via DEA. Computers and Operations Research 21, 543±556. Thanassoulis, E., Dyson, R.G., Estimating preferred target input±output levels using data envelopment analysis. European Journal of Operational Research 56, 80±97.