Ahti Salo Systems Analysis Laboratory Aalto University School of Science and Technology P.O.Box 11100, Aalto FINLAND

Transcription

1 Data Envelopment Analysis Ahti Salo Systems Analysis Laboratory Aalto University School of Science and Technology P.O.Box 11100, Aalto FINLAND These slides build extensively on the teaching materials of Prof. Sri Talluri who gave a DEA course in Helsinki in 2007 (used with permission). 1

2 Data Envelopment Analysis Ahti Salo 2

3 Which Decision Making Unit (DMU) is most productive? Data Envelopment Analysis Ahti Salo 3

4 DEA (Charnes, Coopers & Rhodes 78) DMU = Decision Making Unit A method for measuring the productivity of DMUs which consume multiple inputs and produce multiple outputs #cust x x x x DMU labor hrs. #cust. #cust/hr DMU s 1,3,4,5 are dominated by DMU 2. x labor hrs. Data Envelopment Analysis Ahti Salo 4

5 Extending to multiple outputs.. 8 M.D.s works at a Hospital for the same 160 hrs in a month. Each performs exams and surgeries Which ones are most productive? D o cto r # E x a m s # S u rg e rie s Note: There is some efficient trade-off between the number of surgeries and exams that any one M.D. can do in a month, but what is it? Data Envelopment Analysis Ahti Salo 5

6 Scatter plot of ouputs 120 #6 Efficient M.D. s: These two M.D. s (#1 and #6) define the most efficient trade-off between the two outputs. 100 #Surg geries # These points are dominated by #1 and # #Exams Pareto-Koopman efficiency along the efficient frontier: It is impossible to increase an output (or to decrease an input) without a compensating decrease (increase) in other outputs (inputs). Data Envelopment Analysis Ahti Salo 6

7 Performance gaps How bad are inefficient M.D.s relative to the efficient ones? Where are the gaps? How bad are the gaps? Data Envelopment Analysis Ahti Salo 7

8 Reference set Nearest efficient DMUs define ❶ a reference set and ❷ linear combination of the reference set inputs and outputs of a hypothetical composite unit (HCU) Data Envelopment Analysis Ahti Salo 8

9 Summary of DEA thus far Input/output productivity is defined relative to the efficient frontier This frontier characterizes observed efficient trade-offs among inputs and outputs for a given set of DMUs Efficiency is defined as the relative distance to the frontier Nearest point on the frontier is the efficient comparison unit (hypothetical comparison unit, HCU) Differences in inputs and outputs between DMU and HCU correspond to productivity gaps (improvement potential) But how can we do this analysis systematically? Data Envelopment Analysis Ahti Salo 9

10 A real example on NY Area Sporting Goods Stores Data Envelopment Analysis Ahti Salo 10

11 Productivity Conceptually, productivity (efficiency) is the ratio between outputs and inputs Productivity = Yet reality is rather more complex Inputs Outputs Inputs Outputs equipment facility space server labor mgmt. labor Technology + Decision Making #type A cust. #type B cust. quality index $ oper. profit Data Envelopment Analysis Ahti Salo 11

12 Differences among Operating Units (DMUs) Mix of customers served Availability and cost of inputs Configuration of production facilities Processes and practices used Examples Bank branches, retail stores, clinics, schools, etc Questions: How to compare the productivity of diverse operating units that serve diverse markets? What are the best practice and under-performing units? What are the trade-offs among inputs and outputs? Where are the improvement opportunities and how big are they? Data Envelopment Analysis Ahti Salo 12

13 Some approaches Operating ratios Examples: Labor hours per transaction, sales per square meter Appropriate for highly standardized operations But these do not reflect the varying mix of inputs/outputs of diverse operations Financial approach: Convert everything to monetary terms Concerns Inputs in Outputs in Some inputs/outputs cannot be valued in (non-profit) Profitability is not the same as operating efficiency (e.g. variances in margins and local costs matter as well) Data Envelopment Analysis Ahti Salo 13

14 Profitability vs. efficiency Profitability is a function of three elements Input prices (costs) Output prices Technical efficiency: How much input is required to generate the output Improving operations calls for an understanding of technical efficiency, not just overall profitability. Data Envelopment Analysis Ahti Salo 14

15 Variants of DEA Models CCR Model Charnes, Cooper, and Rhodes (1978) Assumes constant returns to scale in production possibilities: an increase in the amount of inputs leads to a proportional increase in outputs BCC Model Banker, Charnes and Cooper (1984) Constant returns to scale not assumed, efficiency depends on the scale of operations Super efficiency model DEA models with weight information Cross-efficiency models in DEA Ratio-based Efficiency Analysis (REA) Data Envelopment Analysis Ahti Salo 15

16 Notation Data K #operating units (DMUs) k = 1,...,K M # inputs m = 1,...,M N #outputs n = 1,...,N y observed level of output n from DMU k nk x observed level of input m from DMU k mk Model variables v m weight of input i u weight on output n n E efficiency of DMU k (0-100%) k E = k Data Envelopment Analysis Ahti Salo 16 N n=1 M m=1 u y n v x m nk mk

17 Evaluating the CCR efficiency of DMU k Choose nonnegative I/O weights to This is equivalent to max n n subject to n m u,v 0 n u y n v x m m nl ml m u y v x m nk mk 1, l = 1,...,K max max k subject to E 1, k = 1,...,K Data Envelopment Analysis Ahti Salo 17 k subject to m m mk n E Weighted input of DMU k is normalized to one un ynl vmxml 0, l = 1,...,K n u,v 0 n m u y n v x = 1 m nk

18 An example with 4 DMUs Four DMUs, one input, one output The efficiency ratio is highest for DMU A Max.efficiency = 1 Input weight twice as high as output weight Efficiencies of other DMUs E B = 6/8 = 0.75 E C = 9/12 = 0.75 E D = 10/16 = Output needed to reach efficiency = 2x4 = 8 Actual output = 6 Data Envelopment Analysis Ahti Salo 18

19 Input-Oriented CCR Ratio Model How much less inputs should an inefficient DMU use in order to become efficient? min K i=1 λ K i=1 i θ subject to λ x θ x, m = 1,...,M λ i mi mk y y, n = 1,...,N i ni nk 0, i = 1,...,K Optimal θ is the same efficiency as from the primal model Data Envelopment Analysis Ahti Salo 19

20 Dual formulation Dual variable associated with DMU i λ > 0 DMU i is in the reference set of DMU k i λ i These variables can be used to construct an efficient hypothetical composite unit (HCU) with y ˆ = K n i ni i=1 K x ˆ = λ x, m = 1,...,M Input n of HCU m i mi i such that λ y, n = 1,...,N yˆ y, n = 1,...,N n nk xˆ x, m = 1,...,M m mk Output n of HCU Data Envelopment Analysis Ahti Salo 20

21 Uses of the HCU HCU can be used to measure how much more the DMU should produce or how much less it should consume inputs in order to become efficient Output = y ˆ - y 0, n = 1,...,N Input n nk = x - xˆ 0, m = 1,...,M mk m Cf. spreadsheet examples Data Envelopment Analysis Ahti Salo 21

22 Excessive uses of inputs by inefficient DMUs Examples B should produce its current output (6) with one unit less of inputs in order to reach the efficient frontier The gap is therefore one unit Input = 4-3 = 1 Data Envelopment Analysis Ahti Salo 22

23 Output-oriented CCR model Seeks to answer how much more DMU k should produce in order to become efficient max K i=1 λ K i=1 i λ θ subject to x x, m = 1,...,M λ y θ y, n = 1,...,N i mi mk i ni nk 0, i = 1,...,K 1 Efficiency is the reciprocal of optimum θ (i.e. ) θ Data Envelopment Analysis Ahti Salo 23

24 Output gaps for inefficient DMUs Examples The optimal θ for is 4/3 Thus B should produce (4/3)*6 6 = 2 units more using its current inputs to reach the efficient frontier The CCR efficiency of B is 1 over 4/3 = 0.75 Data Envelopment Analysis Ahti Salo 24

25 An illustrative CCR model DMU Input 1 Input 2 Output 1 Output 2 Output max 5v + 14v = u,u,u,v,v 0 7v + 12v u + 4v + 16v 9u + 4u + 16u 5u +7u + 10u 4u + 9u + 13u 5v + 14v 8v + 15v subject to Data Envelopment Analysis Ahti Salo 25

26 Results for the illustrative example DMU 1 and DMU 3 are efficient Efficiency of 1.00 with no slacks DMU 2 is inefficient Efficiency < 1.00 DMUs 1 and 3 can be employed as benchmarks for improvement See Excel example Data Envelopment Analysis Ahti Salo 26

27 BCC Model CCR model assumes constant returns to scale (CRS) whereas the BCC model considers variable returns to scale (VRS) New constraint (convexity) min K i=1 K i=1 K i=1 θ subject to λ x θx, m = 1,...,M i mi mk λ y y, n = 1,...,N i ni nk λ = 1, λ 0 i i Data Envelopment Analysis Ahti Salo 27

28 Change in the set of production possibilities C is BCC efficient B is BCC inefficient A 50%-50% combination of DMUs A and C uses 6 input units and produces 6,5 output units This is more than the 6 units that B produces The resulting BCC output efficiency becomes 1 over (6.5/6) = Similar analyses for input can be made The resulting CCR efficient frontier BCC efficient frontier Data Envelopment Analysis Ahti Salo 28

29 Super efficiency model Helps determine how much more efficient an efficient DMU is relative to other DMUs max m m n n mk u y nk subject to un ynl vm x ml, l = 1,...,K, l k n u,v 0 m v x = 1 n m DMU k under evaluation is removed from the constraint set thereby allowing its efficiency score to exceed a value of 1.00 The model does help rank inefficient DMUs Data Envelopment Analysis Ahti Salo 29

30 Super efficiency illustrated D evaluated relative to the frontier defined by C-E-F Superefficiency defined by the distance OD/OD Similarly E evaluated in comparison with the frontier O 1 /I C-D-F and its superefficiency defined by the distance OE-OE By visual inspection, D is slightly more superefficient than D O Data Envelopment Analysis Ahti Salo 30 C A D D Efficient frontier E E F O 2 /I

31 DEA models with weight information DMUs may attain their efficiency scores for extreme weights in conventional DEA models Preference information can be captured through preference statements about the relative values of ❶ input units ❷ output units Statements impose constraints on the input/output weights The introduction of weight information often leads to lower (but never higher) efficiency scores Data Envelopment Analysis Ahti Salo 31

32 Sets of feasible weights (assurance regions) Preference statements constrain feasible weights A Dissertation is as at least as valuable as 2 Master s Theses, but not more valuable than 7 master s theses» u doctoral 2u master s, u doctoral 7u master s An article in a refereed journal is at least as valuable as a Master s Thesis» u article u master s Only relative weights matter Several elicitation methods can be employed Feasible sets defined by corresponding constraints { ( 1,..., )' 0 0, 0} { (,..., )' 0 0, 0} S = u = u u u A u u N u S = v = v v v A v v 1 M v Data Envelopment Analysis Ahti Salo 32

33 Example of a DEA model with weight restrictions max n u y n nk subject to m v x = 1 m mk u n ynl vm x ml, l = 1,...,K n α v v β v, m = 1,,M a u u b u, n = 1,,N u,v 0 m m m 1 m m n n 1 n n m n Data Envelopment Analysis Ahti Salo 33

34 Weight constraints illustrated (1 input, 2 outputs) Without any weight information, F is efficient O 1 /I Assume that the 1st output on the vertical axis is has more weight than the 2nd output on the horizontal axis Now F becomes dominated by D and E (i.e., for all weights in the revised weight set, D and E will have a higher efficiency) O C A D E F O 2 /I Data Envelopment Analysis Ahti Salo 34

35 Cross-efficiencies in DEA CCR efficiencies are based on the weights which are most favourable to the DMU being evaluated Yet it may be of interest to know how the DMU performs when using other weights as well. The cross efficiency score represents how the DMU performs when evaluated with the optimal weights for all DMUs A DMU with a high cross efficiency score can be considered to be a good overall performer; others are more niche DMUs Data Envelopment Analysis Ahti Salo 35

36 Cross efficiency matrix Efficiency score of DMU 2 when evaluated with the optimal weights of DMU 1 Cross-efficiency score for DMU k is the average of these scores 1 K CR k K i = 1 = Θ Multiple optima are possible, selections either based on aggressive formulation or benevolent formulation Data Envelopment Analysis Ahti Salo 36 ik

37 Selecting inputs and outputs Examples of inputs in operations management Workers, machines, operating expenses, budget, etc. Examples of outputs Number of actual products produced Performance and activity measures such as quality levels, throughput rates, lead-times, etc. If there are M inputs and N outputs then potentially MN DMUs can be efficient To achieve discrimination the number of DMUs should be high enough Data Envelopment Analysis Ahti Salo 37

38 Designing DEA Studies Enough DMUs in relation to inputs/outputs for building an efficient frontier K > 2(N + M) Ambivalence about inputs/outputs - all should matter! Approximate similarity (comparability) of DMUs Objectives Technology DEA provides relative efficiency only Choice of DMUs does matter Inclusion of global leader unit may be desirable Experiments with different I/O combinations may be necessary Data Envelopment Analysis Ahti Salo 38

39 Using the results: Efficiency Profit matrix High Profit Under-performing potential leaders Best practice comparison group Low Eff. High Eff. Under-performing possibly profitable Candidates for closure Low Profit Data Envelopment Analysis Ahti Salo 39

40 Information provided by DEA Objective measures of efficiency A reference set of comparable units Indicators of excess use of inputs Shortfalls in the production of outputs Returns to scale measure Data Envelopment Analysis Ahti Salo 40

41 DEA Summary Uses of DEA Benchmarking to identify best practice units Data mining to generate hypotheses about the drivers of efficiency Performance evaluation and measurement Caveats Essentially a black box approach - gives no information about the causes of inefficiency Strong assumptions (linearity, set of production possibilities) Should not be employed for resource allocation in any straightforward manner Results can be sensitive to selection of inputs/outputs and introduction of outlier DMUs For further reading, see, e.g., W.D. Cook, L.M. Seiford (2009) Data envelopment analysis (DEA) Data Envelopment Analysis Ahti Salo Thirty years on, European Journal of Operational Research 192/1,

42 Ratio-based Efficiency Analysis (REA) 1 DEA measures efficiencies relative to the efficient frontier that is defined by production possibilties This set may not be easy to characterize Introduction of an outlier DMUs may disrupt efficiency scores DEA scores reflect DMUs performance only for weights that are most fabourable to it (cf. motivation for cross-efficiencies) REA Offers efficiency results without making assumptions about production possibilities beyond the set of DMUs that are under comparison Considers the relative efficiencies of DMUs for all feasible weights Offers several efficiency measures 1 Ahti Salo and Antti Punkka (2010). Ranking Intervals and Dominance Relations for Ratio-Based Efficiency Analysis, submitted manuscript, downloadable at Data Envelopment Analysis Ahti Salo 42

43 Efficiency measures in REA Key questions What are the best and worst rankings that a given DMU can attain in comparison with other DMUs, based on the comparison of DMUs' efficiency ratios for all feasible weights? Given a pair of DMUs, does the first DMU dominate the second one? (in the sense that the efficiency ratio of the first DMU is higher than or equal to that of the second for all feasible weights and strictly higher for some weights) How much more/less efficient can a given DMU be relative to some other DMU? Or relative to the most and least efficient DMU in some subset of DMUs? Ranking intervals, dominance relations, efficiency bounds Data Envelopment Analysis Ahti Salo 43

44 Efficiency ratios in CCR-DEA Efficiency score of DMU k is computed with weights u k *,v k * to maximize min l=1,...,k E k /E l Does not provide information about the efficiencies for other weights These weights depend on what DMUs E are considered changing the set of DMUs can influence the order of two DMUs scores E 3 / E * =1 E 5 E 1 / E * =1 E 3 / E * =0.98 E 1 E 2 E * DMU 1 and DMU 3 are efficient If DMU 5 is included, then DMU 2 becomes more efficient than DMU 3 in terms of its DEA score E 3 E 4 E 4 / E * =0.82 Data Envelopment Analysis Ahti Salo 44 u 1

45 Efficiency dominance (1/2) DMU k dominates DMU l iff (i) its efficiency ratio is at least as high as that of DMU l for all feasible weights (ii) higher for some feasible weights E E ( u, v) E ( u, v) for all ( u, v) ( S, S ) k l u v E ( u, v) > E ( u, v) for some ( u, v) ( S, S ) k l u v Example, 2 outputs, 1 input Feasible weights such that 2u 1 u 2 u 1 E 1 E 2 E 3 E 4 E * DMU 3 and DMU 2 dominate DMU 4 Also the inefficient DMU 2 is nondominated u 1 =1/3 u 2 =2/3 u 1 =1/2 u 2 =1/2 Data Envelopment Analysis Ahti Salo 45

46 Efficiency dominance (2/2) A graph shows dominance relations among several DMU Transitive: If A dominates B, and B dominates C, then A dominates C E Asymmetric: (i) If A dom. B, then B does not dom. A and (ii) no DMU dominates itself E 1 E 2 E * Additional preference information helps establish additional relations» An exception: if A dom. B and E A = E B for some feasible weights, then it is possible that E A = E B throughout the smaller feasible region Statement 5u 1 4u 2 leads to new dominance relations u 1 =1/3 u 2 =2/3 Data Envelopment Analysis Ahti Salo 46 E 3 E 4 5u 1 = 4u 2 u 1 =1/2 u 2 =1/2

47 Ranking intervals (1/2) For any feasible weights (u,v), the DMUs can be ranked based on their Efficiency Ratios The minimum ranking of DMU k, r min k, is obtained for weights such that the number of DMUs with strictly higher Efficiency Ratio is minimized The maximum ranking of DMU k, r max k, is obtained for weights such that the number of DMUs with higher or equal Efficiency Ratio is maximized E E 1 E 2 E 3 E 4 u 1 =1/3 u 2 =2/3 ranking 1 ranking 2 ranking 3 ranking 4 E * u 1 =1/2 u 2 =1/2 DMU 1 DMU 2 DMU 3 DMU 4 Data Envelopment Analysis Ahti Salo 47

48 Ranking intervals (2/2) Properties Can be readily compared Provides a holistic view of efficiency ratios at a glance Show also how bad DMUs can be Are insentitive to the introduction of outlier DMUS Additional weight information can narrow (but not widen) ranking intervals CCR-DEA-efficient DMUs have the highest efficiency ratio for some weights their minimum ranking is 1 Data Envelopment Analysis Ahti Salo 48

49 Computation of dominance relations (1/2) How to determine whether DMU k dominates DMU l E ( u, v) E ( u, v) ( u, v) ( S, S ) and k l u v E ( u, v) > E ( u, v) for some ( u, v) ( S, S )? k l u v E ( u, v) E ( u, v) ( u, v) ( S, S ) if k l u v min [ E ( u, v) E ( u, v)] 0 ( u, v) ( S, S ) u v Ek ( u, v) min 1... u v E ( u, v) ( u, v) ( S, S ) k l l (S u,s v ) is open, not bounded, and the objective function non-linear... How to solve the optimization problem? Data Envelopment Analysis Ahti Salo 49

50 Computation of dominance relations (2/2) Normalize weights so that The value of inputs of DMU k =1 The value of outputs of DMU l is equal to its value of inputs Feasible weights are now bounded, closed by linear constraints, objective function linear If the minimum is exactly 1, maximize the same objective function to see whether there N u y n nk n nl n= 1 n= 1 min / 1 A u 0 M M u Av v 0 vm x mk vm xml m= 1 m= 1 N min u y 1 n= 1 Au u 0 Av v 0 vm x v x mk n = 1 = nk u y m ml n nl N u y exists weights such that E k > E l Data Envelopment Analysis Ahti Salo 50

51 Computation of ranking intervals and efficiency bounds Minimum (best) rankings for DMU k 1. For all other DMUs, define binary variables z l so that z l = 1 if E l > E k E ( u, v) E ( u, v) + Cz, C >> 0 l k l 2. Choose a suitable normalization to come up with a MILP model 3. The minimum is 1 + the minimum of z l over (S u,s v ) Maximum rankings with a corresponding model Efficiency bounds compared to the most efficient DMU Maximum with LP similar to the computation of DEA scores Minimum 1. Minimize the linear model used for the computation of dominance relations against all DMUs in the benchmark group 2. The smallest of these is the minimum Comparisons to the least efficient DMU with corresponding models Data Envelopment Analysis Ahti Salo 51

52 Example: Efficiency analysis of TKK s departments Departments consume inputs in order to produce outputs Data from TKK s reporting system 2 inputs, 44 outputs x 1 (Budget funding) x 2 (Project funding) Department y 1 (Master s Theses) y 2 (Dissertations) y 3 (Int l publications) Preferences from 7 members of the Resources Committee Ex: What is the value of a Master s thesis relative to a dissertation.? Each member responded to elicitation questions, which yielded crisp weights The feasible weights were then modeled as all possible convex combinations of these weightings Data Envelopment Analysis Ahti Salo 52

53 A B C D E F G H I J K L A B C D E F G H I J K L Efficiency bounds compared to the most efficient department A B I J G D, F, H L K C, E Dominance relations Ranking intervals Departments A, J and L are efficient But A can attain ranking 7 > 4, the worst ranking of the inefficient department K There are feasible weights so that the Efficiency Ratio of A is only 57 % of that of the most efficient department» For K, the corresponding ratio is 71% The efficiency intervals of D, F and H overlap with those of B and G Yet, for all feasible weights the Efficiency Ratios of D, F and H are smaller than those of B and G Data Envelopment Analysis Ahti Salo 53

54 Conclusion REA results use all feasible weights to evaluate DMUs Dominance relations compare DMUs pairwise Ranking intervals show which rankings can be attained by DMUs Efficiency bounds show how efficient a DMU can be compared to the DMUs in a benchmark group Computed with LP and MILP models Admits preference information Helps exclude the use of extreme weights in efficiency determination: 100 dissertations is less valuable than an article Additional preference information makes REA results more conclusive Introduction of new DMUs do not affect results for other DMUs Data Envelopment Analysis Ahti Salo 54