An Improved Dynamic Programming Decomposition Approach for Network Revenue Management

Transcription

1 An Improved Dynamic Programming Decomposition Approach for Network Revenue Management Dan Zhang Leeds School of Business University of Colorado at Boulder May 21, 2012

2 Outline Background Network revenue management formulation Classical dynamic programming decomposition An improved dynamic programming decomposition Numerical results Summary

3 Network RM Formulation (Gallego and van Ryzin, 1997; Gallego et. al., 2004; Liu and van Ryzin, 2008) m resources with capacity c (an m-vector) Capacity for resource i is c i. n products N = {1,..., n} Fare for product j is f j Product consumption matrix A = [a ij ] Finite time horizon with length τ In each period, there is one customer arrival with probability λ, and no customer arrival with probability 1 λ. Given a set of products S N, a customer chooses product j with probability P j (S). No-purchase probability P 0 (S) = 1 j S P j (S). Objective: Maximize expected total revenue

4 Applications Industry Resources Products Airlines Scheduled flights O-D itineraries at certain fare levels Hotels Room-days Single(multi)-day stays at certain rates Car rentals Car-days Single(multi)-day rentals at certain rates Air Cargo Scheduled flights (weight) O-D shipments at certain rates Scheduled flights (volume)

5 Dynamic Programming Formulation DP optimality equations: v t (x) = v τ+1 (x) = 0, max S N(x) { λ j S P j (S)(f j + v t+1 (x A j )) } + (λp 0 (S) + 1 λ)v t+1 (x), t, x, x. Notations v t (x): DP value function A j : resource incidence vector of product j N(x): {j N : x A j }

6 Dynamic Programming Formulation DP optimality equations: v t (x) = v τ+1 (x) = 0, max S N(x) { λ j S P j (S)(f j + v t+1 (x A j )) } + (λp 0 (S) + 1 λ)v t+1 (x), t, x, x. Notations v t (x): DP value function A j : resource incidence vector of product j N(x): {j N : x A j } Curse of dimensionality: state space grows exponentially with the number of resources

7 Choice-based Deterministic Linear Program (CDLP) z CDLP = max h S N λr(s)h(s) S N λq(s)h(s) c, h(s) τ, S N (Resource constraint) (Time constraint) h(s) 0, S N. (Non-negativity) Replace stochastic demand with deterministic fluid with rate λ Given offer set S N Total time S is offered: h(s) Revenue from unit demand: R(S) = j S f jp j (S) Consumption of resource i from unit demand: Q i (S) = j S a ijp j (S)

8 CDLP (Gallego et. al, 2004; Liu and van Ryzin, 2008) CDLP can by efficiently solved for certain class of choice models. The vector of dual values π associated with resource constraints can be used as bid-prices for resources z CDLP provides an upper bound on revenue Some recent references: Talluri (2010): Concave programming formulation Gallego, Ratliff, Shebalov (2011): Efficient reformulation

9 Classical Dynamic Programming Decomposition For each i, approximate the DP value function with v t(x) v t,i (x i ) + πk x k, } {{ } k i Value of the } {{ } i-th resource Value of all other resources t, x.

10 Classical Dynamic Programming Decomposition For each i, approximate the DP value function with v t(x) v t,i (x i ) + πk x k, } {{ } k i Value of the } {{ } i-th resource Value of all other resources t, x. Using the approximation in DP recursion leads to ( v t,i (x i ) = max λp j (S) f j ) a kj πk +v t+1,i (x i a ij ) S N(x i,c i ) j S k i } {{ } Fare proration + (λp 0(S) + 1 λ)v t+1(x i ), t, x i.

11 Classical Dynamic Programming Decomposition For each i, approximate the DP value function with v t(x) v t,i (x i ) + πk x k, } {{ } k i Value of the } {{ } i-th resource Value of all other resources t, x. Using the approximation in DP recursion leads to ( v t,i (x i ) = max λp j (S) f j ) a kj πk +v t+1,i (x i a ij ) S N(x i,c i ) j S k i } {{ } Fare proration + (λp 0(S) + 1 λ)v t+1(x i ), t, x i. Compute offer sets dynamically using the approximate value function v t(x) i v t,i (x i ), t, i.

12 Classical Dynamic Programming Decomposition A DP with m-dimensional state space is reduced to m one-dimensional DPs, one for each resource states (Assume 100 seats per flight) states

13 Classical Dynamic Programming Decomposition A DP with m-dimensional state space is reduced to m one-dimensional DPs, one for each resource states (Assume 100 seats per flight) states Variants of the approach are widely used in practice. Review: Talluri and van Ryzin (2004a)

14 DP Decomposition Bounds Proposition (Zhang and Adelman, 2009) The following relationships hold: (i) v t (x) min l=1,...,m {v t,l (x l ) + } k l π k x k v t,i (x i ) + k i π k x k, i, t, x; (ii) v 1 (c) v 1,i (c i ) + k i π k c k z CDLP, i. Decomposition value for each leg provides an upper bound on revenue Decomposition bounds are tighter than the bound from CDLP

15 Linear Programming Formulation of DP (Adelman, 2007) min v 1 (c) {v t( )} t v t (x) j S λp j (S)(f j + v t+1 (x A j )) + (λp 0 (S) + 1 λ)v t+1 (x), t, x, S N(x).

16 Linear Programming Formulation of DP (Adelman, 2007) min v 1 (c) {v t( )} t v t (x) j S λp j (S)(f j + v t+1 (x A j )) + (λp 0 (S) + 1 λ)v t+1 (x), t, x, S N(x). Huge number of decision variables and constraints

17 Linear Programming Formulation of DP (Adelman, 2007) min v 1 (c) {v t( )} t v t (x) j S λp j (S)(f j + v t+1 (x A j )) + (λp 0 (S) + 1 λ)v t+1 (x), t, x, S N(x). Huge number of decision variables and constraints Functional approximation idea: use a parameterized representation of the value function to reduce the number of decision variables

18 The Affine Functional Approximation (Zhang and Adelman, 2009) Affine approximation is given by v t (x) θ t + i V t,i x i, t, x. (1)

19 The Affine Functional Approximation (Zhang and Adelman, 2009) Affine approximation is given by v t (x) θ t + i V t,i x i, t, x. (1) Using (1) in the linear programming formulation leads to min θ,v θ1 + i V 1,i c i ( θ t + V t,i x i λp j (S) f j + θ t+1 + V t+1,i (x i a ij ) i j S i ( + (λp 0(S) + 1 λ) θ t+1 + ) V t+1,i x i, t, x, S N(x). i )

20 The Affine Functional Approximation The dual program is given by z P1 = max λp j (S)f j Y t,x,s Y t,x,s N(x) j S { c i, if t = 1, x i Y t,x,s = x,s N(x) (x i ) j S λp j (S)a ij Y t 1,x,S, t = 2,..., τ x,s N(x) { Y t,x,s = 1, if t = 1, x,s N(x) Y t 1,x,S, t = 2,..., τ. x,s N(x) Y 0. i, t,

21 The Affine Functional Approximation The dual program is given by z P1 = max λp j (S)f j Y t,x,s Y t,x,s N(x) j S { c i, if t = 1, x i Y t,x,s = x,s N(x) (x i ) j S λp j (S)a ij Y t 1,x,S, t = 2,..., τ x,s N(x) { Y t,x,s = 1, if t = 1, x,s N(x) Y t 1,x,S, t = 2,..., τ. x,s N(x) Y 0. i, t, Due to the large number of columns, solving the linear program above still requires considerable computational effort.

22 Functional Approximation Approaches for Network RM Citation Choice Model Functional approximation Solution strategy Adelman (2007) Independent demand Affine Column generation Zhang and Adelman (2009) MNLD Affine Column generation Zhang (2011) MNLD Nonlinear non-separable CDLP+Simultaneous DP Liu and van Ryzin (2008) MNLD Separable (fare proration) CDLP+DP Decomposition Miranda Bront et. al. (2009) MNLO Separable (fare proration) CDLP+DP Decomposition Farias and Van Roy (2008) Independent demand Separable concave Constraint sampling Meissner and Strauss (2012) MNLD Separable concave Column generation Kunnumkal and MNLD Separable (fare proration) Convex programming Topaloglu (2011) +DP Decomposition Tong and Topaloglu (2011) Independent demand Affine Reduction + Constraint generation Vossen and Zhang Independent demand Affine Reduction + MNLD + Dynamic disaggregation MNLD: Multinomial logit model with disjoint consideration sets MNLO: Multinomial logit model with overlapping consideration sets

23 Research Questions Computational cost: ADP (affine or separable concave approximation) classical DP decomposition

24 Research Questions Computational cost: ADP (affine or separable concave approximation) classical DP decomposition How can we balance solution quality with solution time? Can we improve the classical DP decomposition?

25 A Strong Functional Approximation (Zhang, 2011) v t (x) min ˆv t,i(x i ) + πk x k, t, x. i k i Nonlinear and non-separable functional approximation Each value v t (x) is approximated by a single value across legs Motivated by the decomposition bounds (Zhang and Adelman, 2009)

26 A Nonlinear Optimization Problem Using the new functional approximation leads to z NLP = min min ˆv 1,i(c i ) + πk c k ˆv t,i ( ) t,i i k i min ˆv t,i(x i ) + πk x k i k i λp j (S) f j + min ˆv t+1,l(x l a lj ) + πk (x k a kj ) l j S k l + (λp 0(S) + 1 λ)min ˆv t+1,l(x l ) + πk x k l, t, x, S N(x). k l The problem is a nonlinear optimization problem with a huge number of nonlinear constraints.

27 A Restricted Optimization Problem Step 1: Writing each constraint as m equivalent constraints Step 2: Restricting the constraints so that each constraint only involves one resource The restricted problem provides a relaxed bound: Proposition The objective value of the restricted program, z NLP, is bigger than z NLP.

28 An Equivalent Simultaneous Dynamic Program ˆv t,i(x i ) = max S N(x i,c i ) { min l i ( { λp j (S) f j + min ˆv t+1,i(x i a ij ) πk a kj, j S k i max [ˆv t+1,l(y l ) y l πl ] } }) a kj πk + πi x i 0 y l c l a lj k + (λp 0(S) + 1 λ) min i, t, x i. { ˆv t+1,i(x i ), min l i { max 0 y l c l [ˆv t+1,l(y l ) π l y l ] + π i x i DP recursion for resource i involves values from all other resources }}

29 An Equivalent Simultaneous Dynamic Program ˆv t,i(x i ) = max S N(x i,c i ) { min l i ( { λp j (S) f j + min ˆv t+1,i(x i a ij ) πk a kj, j S k i max [ˆv t+1,l(y l ) y l πl ] } }) a kj πk + πi x i 0 y l c l a lj k + (λp 0(S) + 1 λ) min i, t, x i. { ˆv t+1,i(x i ), min l i { max 0 y l c l [ˆv t+1,l(y l ) π l y l ] + π i x i DP recursion for resource i involves values from all other resources The dynamic program is equivalent to the restricted nonlinear program can be solved efficiently via a simultaneous dynamic programming algorithm leads to tighter revenue bounds }}

30 New Bounds Proposition (Zhang, 2011) Let {ˆv t,i ( )} t,i,x i be the optimal solution from the simultaneous dynamic program. The following results hold: (i) ˆv t,i (x i) v t,i (x i ), i, x i ; (ii) v 1 (c) z NLP z NLP = min i {ˆv 1,i (c i) + } k i π k c k min i {v 1,i (c i ) + } k i π k c k z CDLP. The simultaneous dynamic program provides tighter bounds on revenue than the classical decomposition.

31 Recap High dimensional dynamic program

32 Recap High dimensional dynamic program Large scale linear program

33 Recap High dimensional dynamic program Large scale linear program Large scale nonlinear program with nonlinear constraints

34 Recap High dimensional dynamic program Large scale linear program Large scale nonlinear program with nonlinear constraints Restricted nonlinear program with nonlinear constraints

35 Recap High dimensional dynamic program Large scale linear program Large scale nonlinear program with nonlinear constraints Restricted nonlinear program with nonlinear constraints Simultaneous dynamic program

36 Comparison: Classical vs. Improved Approaches Classical dynamic programming decomposition: Solve m single-leg DPs Prorated fares Fare proration Static bid-prices Solve CDLP

37 Comparison: Classical vs. Improved Approaches Classical dynamic programming decomposition: Solve m single-leg DPs Prorated fares Fare proration Static bid-prices Solve CDLP Network effects only captured through fare proration

38 Comparison: Classical vs. Improved Approaches Classical dynamic programming decomposition: Solve m single-leg DPs Prorated fares Fare proration Static bid-prices Solve CDLP Network effects only captured through fare proration Improved dynamic programming decomposition: Solve one simultaneous DP Static bid-prices Solve CDLP

39 Comparison: Classical vs. Improved Approaches Classical dynamic programming decomposition: Solve m single-leg DPs Prorated fares Fare proration Static bid-prices Solve CDLP Network effects only captured through fare proration Improved dynamic programming decomposition: Solve one simultaneous DP Static bid-prices Solve CDLP Network effects captured during DP recursion!

40 Computational Study: Problem Instances Randomly generated hub-and-spoke instances Number of non-hub locations (flights) in the set {4, 8, 16, 24} Number of periods in the set {100, 200, 400, 800} Two products for each possible itinerary Multinomial Logit Choice Model with Disjoint Consideration Sets (MNLD) Largest problem instance: 24 non-hub locations (flights), 336 products, 800 periods

41 Numerical Study: Policies DCOMP1: the new decomposition approach where the approximation m v t (x) ˆv t,i(x i ), t, x i=1 is used to compute control policies. DCOMP: the classical dynamic programming decomposition CDLP: static bid-price policy based on the dual values of resource constraints in CDLP CDLP10: A version of CDLP that resolves 10 times with equally spaced resolving intervals Each policy is simulated times

42 Computational Time Case # Parameters Capacity Load CPU seconds DCOMP1 DCOMP per leg factor CDLP DCOMP DCOMP1 DCOMP A1 (100,4,4,16) % A2 (200,4,4,16) % A3 (400,4,4,16) % A4 (800,4,4,16) % A5 (100,8,8,48) % A6 (200,8,8,48) % A7 (400,8,8,48) % A8 (800,8,8,48) % A9 (100,16,16,160) % A10 (200,16,16,160) % A11 (400,16,16,160) % A12 (800,16,16,160) % A13 (100,24,24,336) % A14 (200,24,24,336) % A15 (400,24,24,336) % A16 (800,24,24,336) %

43 Bound Performance Case # CDLP DCOMP DCOMP1 Bound improvement %-difference across legs bound bound bound %-CDLP %-DCOMP DCOMP DCOMP1 A % 4.74% 4.46% 0.00% A % 1.55% 1.87% 0.36% A % 0.98% 2.36% 0.00% A % 0.36% 0.58% 0.00% A % 0.99% 3.18% 0.05% A % 0.21% 2.18% 0.22% A % 0.18% 1.09% 0.00% A % 0.03% 1.49% 0.00% A % 0.76% 4.69% 0.00% A % 0.64% 2.95% 0.03% A % 0.30% 1.52% 0.01% A % 0.15% 0.94% 0.00% A % 14.05% 10.37% 0.00% A % 2.27% 5.50% 0.00% A % 0.66% 2.80% 0.03% A % 0.24% 1.44% 0.00%

44 Bounds from Individual Legs Bounds from individual Legs DCOMP DCOMP Leg

45 Bounds from Individual Legs Bounds from individual Legs DCOMP DCOMP Leg DCOMP1 bounds are more homogeneous across legs

46 A Hub-and-spoke Network with 2 Non-Hub Locations Case # τ Load Capacity DCOMP1 DCOMP1 Revenue Gains OPT-GAP factor per leg REV %-CDLP %-CDLP10 %-DCOMP B % % 8.56% 2.74% B % % 5.74% 4.11% B % -1.00% -0.33% 0.03% B % % 4.41% 7.17% B % 48.88% 3.00% 0.09% B % 13.42% 3.62% 5.77% B % % 4.78% 7.97% B % 42.16% -4.49% 0.00% B % 13.27% 1.33% 1.94% B % 15.05% 0.98% 4.65% B % 15.46% 1.70% 8.14% B % 15.47% 2.45% 5.52% B % 2.89% 0.47% 0.34% B % 4.84% 1.14% 0.11% B % 22.11% 2.22% 0.04% B % 7.92% 2.14% 0.01% B % 23.82% 1.46% 0.18% B % 2.61% 1.24% 0.04% B % 30.16% 2.48% 0.01% B % 31.46% 2.60% 0.00%

47 DCOMP1 Percentage Revenue Gain vs. Load Factor DCOMP1 percentage revenue gain % CDLP10 % DCOMP Load factor

48 DCOMP1 Percentage Revenue Gain vs. Load Factor DCOMP1 percentage revenue gain % CDLP10 % DCOMP Load factor Higher load factor Higher revenue gains

49 DCOMP1 Percentage Revenue Gain vs. Number of Periods DCOMP1 percentage revenue gain % CDLP10 % DCOMP Number of periods

50 DCOMP1 Percentage Revenue Gain vs. Number of Periods DCOMP1 percentage revenue gain % CDLP10 % DCOMP Number of periods Significant revenue gains for problems with long selling horizons!

51 A Hub-and-spoke Network with 4 Non-Hub Locations Case # τ Load Capacity DCOMP1 DCOMP1 Revenue Gains OPT-GAP factor per leg REV %-CDLP %-CDLP10 %-DCOMP C % 14.60% 0.46% 0.10% C % 52.79% 1.95% 1.32% C % 12.35% -0.11% 0.03% C % 52.31% 1.36% 0.66% C % 44.57% 1.71% 1.96% C % 14.18% 1.72% 2.67% C % 29.01% -2.27% -0.81% C % 1.02% 0.49% 0.18% C % 4.98% 1.46% 2.67% C % 4.18% 1.61% 4.11% C % 3.44% 1.51% 4.43% C % 57.07% 1.92% 3.08% C % 1.30% -0.90% 0.21% C % 2.18% 0.21% 0.08% C % 6.38% 0.74% 0.00% C % 2.66% 0.89% -0.06% C % 2.50% -0.81% 0.44% C % 3.05% -0.03% 0.20% C % 3.24% 0.26% 0.05% C % 3.21% 0.41% 0.02%

52 DCOMP1 Percentage Revenue Gain vs. Load Factor DCOMP1 percentage revenue gain % CDLP10 % DCOMP Load factor

53 DCOMP1 Percentage Revenue Gain vs. Number of Periods DCOMP1 percentage revenue gain % CDLP10 % DCOMP Number of periods

54 Summary and Future Directions Functional approximation approach is promising for solving large scale stochastic dynamic programs. However, implementations of the approach often require very high computational cost. The first nonlinear non-separable functional approximation for network RM problem Novel approximation architecture Better revenue bounds Improved heuristic policies Moderate computational cost Current work Exploiting special structures of the LP formulations of dynamic programs in value function approximation (Vossen and Zhang, 2012) Applications with real data (Zhang and Weatherford, 2012)

55 Thank you! Questions? Comments?