Randomization Approaches for Network Revenue Management with Customer Choice Behavior

Transcription

1 Randomization Approaches for Network Revenue Management with Customer Choice Behavior Sumit Kunnumkal Indian School of Business, Gachibowli, Hyderabad, , India sumit March 9, 2011 Abstract In this paper, we present new approximation methods for the network revenue management problem with customer choice behavior. Our methods are sampling-based and so we require only minimal assumptions regarding the underlying customer choice model. The starting point for our methods is a dynamic program that allows randomization. An attractive feature of this dynamic program is that the size of its action space is linear in the number of itineraries, as opposed to exponential. It turns out that this dynamic program has a structure that is similar to the dynamic program for the network revenue management problem under the so called independent demand setting. Our approximation methods exploit this similarity and build on ideas developed for the independent demand setting. We present two approximation methods. The first one is based on relaxing the flight leg capacity constraints using Lagrange multipliers, whereas the second method involves solving a perfect hindsight relaxation problem. We show that both methods yield upper bounds on the optimal expected total revenue. Computational experiments indicate that our methods can generate tighter upper bounds and higher expected revenues when compared with the standard deterministic linear program that appears in the literature.

2 Network revenue management with customer choice behavior is well-studied and has many applications in the airline, hotel and car rental industries. In the context of airlines, a representative example, it involves controlling the sale of itineraries over a flight network. Customers arrive over the booking period to purchase itineraries. The airline has to decide which itineraries to make available for sale at each point in time taking into account the remaining capacities on the flight legs. This is a crucial decision to make since the customer s purchasing decision is influenced by the set of itineraries that are offered. Depending on the offer set, the customer may purchase one of the offered itineraries, or may not purchase anything and simply leave. The airline s goal is to determine the set of itineraries to offer at each point in time that maximizes the expected total revenues over the booking period. The airline s decision problem can be formulated as a dynamic program. However, computing the value functions and the optimal policy quickly become intractable and one has to resort to approximation methods. Many of the approximation methods for the network revenue management problem with customer choice build on methods developed for network revenue management under the assumption that the customer s purchasing decision is not influenced by the set of offered itineraries. This is the so called independent demand setting, where we assume that customers arrive with the intention of purchasing a fixed itinerary. If the itinerary is available, they make the purchase. Otherwise, they leave without making any purchase. Even with the independent demand assumption, the network revenue management problem becomes intractable as the size of the state space increases exponentially with the number of flight legs. Consequently, the approximation methods for the network revenue management problem with independent demand have mainly been concerned with reducing the dimensionality of the state space. Incorporating customer choice behavior adds another layer of complexity since the size of the action space also increases exponentially with the number of itineraries. This is because of the combinatorial nature of the problem of deciding which subset of itineraries to offer for sale from the set of all possible itineraries. So, while many of the approximation methods for the network revenue management problem with customer choice are able to handle the dimensionality of the state space quite well, they are less effective in dealing with the complexity of the action space. As a result, the tractability of many of the existing methods depends on the underlying model of customer choice. It is usually assumed that the customer choices are governed by the multinomial logit model and that the consideration sets, the sets of itineraries of interest to the different customer segments, are disjoint. In this paper, we propose new approximation methods that remain tractable for a large class of choice models. We assume that a customer s choice decision is governed by a simple utility maximization principle. That is, a customer has a utility for purchasing each of the itineraries and to not purchasing anything. Of the available alternatives, the customer chooses the one with the highest utility. The starting point for our methods is a dynamic program that allows randomization. We generate a sample path of customer arrivals along with their utilities for the different itineraries and formulate a dynamic program in order to compute the optimal offer sets. We show that it is possible to reformulate this problem as a dynamic program where the number of decision variables is linear in the number of itineraries. As a result, the size of the action space becomes manageable. In fact, the resulting formulation is similar to the dynamic programming formulation of the network revenue management problem with independent demand. Consequently, we use ideas from the independent demand setting to reduce the 2

3 size of the state space. We particularly focus on two approximation methods. One is based on the Lagrangian relaxation idea developed in Kunnumkal and Topaloglu (2010a) and the second is based on the randomized linear programming approach developed in Talluri and van Ryzin (1999). The methods that we propose have a number of appealing features. Since they are sampling-based, they can handle a much broader class of choice models. In particular, we do not require the assumption that customer choices come from a multinomial logit choice model with disjoint consideration sets. Our methods yield upper bounds on the optimal expected revenue and estimates of the expected marginal values of capacity on the flight legs. The marginal value of capacity on a flight leg, referred to as its bid price, is useful in constructing control policies. On the other hand, upper bounds are useful when assessing the suboptimality of heuristic control policies. Another useful feature of our approach is that the randomized dynamic program we propose has a similar structure to the dynamic program for the network revenue management problem with independent demand. This allows us to draw upon the rich literature around the network revenue management problem with independent demand. The two approximation methods that we propose require solving only linear programs, which most commercial optimization packages are capable of. Moreover, since the linear programs we solve have only a polynomial number of variables and constraints, it minimizes the need for customized coding in the way of column generation techniques. This may further enhance the practical appeal of our methods. Our work builds on previous research. Liu and van Ryzin (2008) propose a deterministic linear program for the network revenue management problem with customer choice. Zhang and Adelman (2009), Meissner and Strauss (2008) and Zhang (2011) use the linear programming approach to approximate dynamic programming to come up with different value function approximations, where as Kunnumkal and Topaloglu (2008) and Kunnumkal and Topaloglu (2010b) use Lagrangian relaxation ideas. Kunnumkal and Topaloglu (2010c) propose a linear integer program that allows randomization and show how it can be used to compute bid prices. The tractability of the above mentioned methods depends on the assumptions that the customers choices are governed by the multinomial logit model and that the consideration sets of the different customer segments are disjoint. Bront, Mendez-Diaz and Vulcano (2009) analyze the case where the consideration sets overlap and show that the column generation subproblem in the deterministic linear program of Liu and van Ryzin (2008) is NP-hard. Bront et al. (2009) and Meissner and Strauss (2010) propose heuristic methods for column generation. Talluri (2010) proposes a concave program for general choice models and describes a way to randomize it. Meissner, Strauss and Talluri (2011) build on this concave program and show how it can be strengthened by adding additional constraints. van Ryzin and Vulcano (2008) and Chaneton and Vulcano (2009) use stochastic approximation to respectively compute protection levels and bid prices for general choice models. The utility maximization criterion to model customer choice behavior has appeared in the literature. For example, van Ryzin and Vulcano (2008) and Chaneton and Vulcano (2009) use it to model customer choice in network revenue management while Mahajan and van Ryzin (2001) use it in the context of optimizing retail assortments. Our work also builds on approximation methods for the network 3

4 revenue management problem with independent demand. The papers closest to ours are Kunnumkal and Topaloglu (2010a) and Talluri and van Ryzin (1999). Kunnumkal and Topaloglu (2010a) propose a linear program that yields time dependent bid prices. Talluri and van Ryzin (1999) propose a randomized linear program that works with samples of the demand random variables. We refer the reader to Talluri and van Ryzin (2004) for a comprehensive review of the revenue management literature. We make the following research contributions in this paper. 1) We present a new dynamic programming approximation for the network revenue management problem with customer choice behavior. This dynamic programming formulation is attractive because it allows randomization and the size of its action space is linear in the number of itineraries. 2) We further build on this randomized dynamic program to obtain tractable approximation methods. As our methods are sampling-based, we are not constrained by the underlying customer choice model. We are able to handle a variety of choice models; all we require is the ability to generates samples of the customers utilities for the different alternatives. 3) We show that our approximation methods generate upper bounds on the optimal expected revenues. Upper bounds are useful when assessing the suboptimality of heuristic control policies. We also show how our methods can be used to obtain bid prices. 4) Computational experiments indicate that our methods can yield significantly tighter upper bounds and higher revenues than the standard deterministic linear program. The rest of the paper is organized as follows. Section 1 describes the network revenue management problem with customer choice behavior and formulates it as a dynamic program. In Section 2, we describe the linear program proposed by Liu and van Ryzin (2008). In Section 3, we present the randomized dynamic program and in Section 4 we describe two tractable approximation methods based on it. The first method is based on relaxing the flight leg capacity constraints whereas the second method solves a perfect hindsight relaxation. Section 5 presents our computational experiments. The proofs of all the propositions and lemmas are deferred to the Appendix. 1 Problem Formulation We have an airline network consisting of a set of flight legs that we can use to serve the customers that arrive over time with the intention of purchasing itineraries. We use L to denote the set of flight legs in the airline network. The initial capacity on flight leg i is c i. We use J to denote the set of all itineraries. An itinerary j has a revenue associated with it, which we denote by r j. If we accept a request for itinerary j, then we consume capacity on one or more flight legs. We use a ij to denote the number of units of capacity consumed by itinerary j on flight leg i. Naturally, we have a ij = 0 if itinerary j does not include flight leg i. We discretize the planning horizon into a finite number of time periods T = {1,..., τ} and assume that the discretization is fine enough so that there is at most one customer arrival at each time period. The probability of a customer arrival at time period t is λ. The fact that the arrival probability is constant over time is only for ease of exposition and it is straightforward to allow the arrival probability to depend on the time period t. We assume that customer choice is governed by a simple utility maximization principle. That is, 4

5 the customer s utilities for the different alternatives are random variables and the customer chooses the alternative with the highest utility. We note that the utility maximization principle is essentially equivalent to a choice model where customers have an ordered list of preferences and pick the most preferred alternative from the ones available. We let U jt be the random variable which denotes the utility for purchasing itinerary j at time period t and let U φt be the random variable which denotes the utility for not purchasing any itinerary at time period t. We let U J = {U jt : j J, t T } and U φ = {U φt : t T }. We allow the random variables {U jt : j J {φ}} to be dependent within each time period, but assume that they are independent across time periods. In other words, the purchasing decisions of the different customers are assumed to be independent of each other. Given an offer set S, the customer chooses the alternative j t = argmax j S {φ} {U jt } with the highest utility. We assume that there are no ties with probability 1. The probability that the customer chooses itinerary j at time period t given the offer set S is Pr{j t = j S} = Pr{U jt = max {U kt}} for j S. k S {φ} We have Pr{j t = j S} = 0 for j / S and the probability of purchasing nothing is Pr{j t = φ S} = Pr{U φt = max {U kt}} = 1 Pr{j t = j S}. (1) k S {φ} Here we emphasize that the customer s utilities for the different alternatives do not depend on the set of itineraries made available for sale. While there are choice models that do not satisfy this assumption, it covers many of the commonly used choice models in the literature; see Mahajan and van Ryzin (2001) and Zhang and Cooper (2005). At each time period, we have to decide which itineraries to make available for sale taking into account the state of the remaining leg capacities. Using x it to denote the remaining capacity on flight leg i at time period t, x t = {x it : i L} captures the state of the remaining leg capacities. We let Q(x t ) = {j J : a ij x it i L}, (2) denote the itineraries that can be potentially offered given the remaining leg capacities. The decision problem is to determine the set of itineraries to offer to the customers at each time period so as to maximize the expected total revenue over the planning horizon. Under the assumption that the customer arrivals in the different time periods and the purchasing decisions of the different customers are independent of each other, we can obtain the value functions {V t ( ) : t T } through the optimality equation V t (x t ) = max S Q(x t) = max S Q(x t) { { λ Pr{j t = j S} r j + V t+1 (x t } ] ] i L a ij e i ) + 1 λ + λ Pr{j t = φ S} V t+1 (x t ) λ Pr{j t = j S} r j + V t+1 (x t i L a ij e i ) V t+1 (x t )] } + V t+1 (x t ), (3) where e i is the L -dimensional unit vector with a one in the element corresponding to i L and the second equality follows from (1). The boundary condition for the optimality equation above is V τ+1 ( ) = 5

6 0. Throughout the rest of the paper, we assume that λ = 1 for notational brevity. We note that this is equivalent to letting Pr{j t = j S} = λ Pr{j t = j S} and Pr{j t = φ S} = 1 λ + λ Pr{j t = j S} and working with the probabilities { Pr{j t = j S} : j J {φ}}. Solving the above dynamic program for practical problem instances becomes difficult for two reasons. One is that the size of the state space increases exponentially with the number of flight legs in the airline network. For, if we let C i = {0,..., c i }, then the state space of the above dynamic program is i L C i, which is exponential in the number of flight legs. Secondly, the size of the action space also increases exponentially with the number of itineraries in the flight network since the number of potential offer sets is of the order of 2 J. In the following sections, we look at relaxations of problem (3) that are computationally tractable. 2 Choice Based Deterministic Linear Program The choice based deterministic linear program, proposed by Liu and van Ryzin (2008), is an approximation that replaces all random quantities by their expected values. If set S is offered at time period t, then the expected revenue obtained is r j Pr{j t = j S}, while the expected capacity consumed on flight leg i is a ij Pr{j t = j S}. The choice based deterministic linear program assumes that the revenue generated and the capacities consumed by offering set S take on their expected values. It determines the optimal choice of offer sets at each time period by solving z CDLP = max r j Pr{j t = j S}h t (S) (4) t T subject to a ij Pr{j t = j S}h t (S) c i i L (5) t T h t (S) 1 t T (6) S J h t (S) 0 t T. (7) In the above linear program, the decision variable h t (S) denotes the frequency with which set S is offered at time period t. The first set of constraints ensure that the expected capacity consumed on each flight leg does not exceed the available capacity. The second set of constraints ensure that the total frequency with which we offer the sets at each time period is at most one. Note that the number of decision variables in the above linear program is exponential in the number of itineraries. So in general, one has to resort to column generation to solve the problem (4)-(7). Liu and van Ryzin (2008) show that column generation can be efficiently carried out provided the choice probabilities come from the multinomial logit model. Gallego, Ratliff and Shebalov (2010) show that problem (4)-(7) can be reformulated as a linear program with only a polynomial number of variables provided the choice probabilities come from a general attraction model, of which the multinomial logit is a special case. There are two main uses of the choice based deterministic linear program. First, Liu and van Ryzin (2008) show that its optimal objective value gives an upper bound on the optimal expected total 6

7 revenue. That is, we have V 1 (c) z CDLP. Second, we can use the dual solution of the choice based deterministic linear program to construct heuristic control policies. Let ˆπ = {ˆπ i : i L} denote the optimal values of the dual variables associated with constraints (5). Noting that ˆπ i approximates the marginal value of capacity on flight leg i, we use ˆπ i as its bid price. We can use these bid prices to come up with different control policies. Zhang and Adelman (2009) propose approximating the value function V t (x t ) by i L ˆπ ix it and solving problem (3) using this value function approximation to decide on the offer set. That is, we solve the problem max S Q(x t) Pr{j t = j S} r j ] a ij ˆπ i i L to decide on the set of itineraries to offer at time period t. (8) We note that the above maximization problem is combinatorial in nature and can be potentially difficult to solve for a general choice model. Bront et al. (2009) and Meissner and Strauss (2010) propose heuristic methods for solving problem (8). Chaneton and Vulcano (2009) propose a simpler alternative, where we make an itinerary available for sale provided its fare exceeds the sum of the bid prices on the flight legs it uses and there is sufficient capacity. 3 Randomized Dynamic Program In this section, we present a randomized dynamic program for the network revenue management problem with customer choice behavior. Letting U J = {U jt : j J, t T } be a sample of the customers utilities for the different itineraries at the different time periods, we solve the optimization problem { V t (x t U J ) = max Pr{j t = j S, U J } r j + V t+1 (x t ] } i L a ij e i U J ) V t+1 (x t U J ) S Q(x t ) +V t+1 (x t U J ), (9) with the boundary condition that V τ+1 ( U J ) = 0. We use the argument U J to emphasize that the solution to the above optimality equation depends on the sampled utilities U J and therefore is a random variable. We also note that U J only specifies the utilities for purchasing the itineraries; the utilities for not purchasing anything U φ = {U φt : t T } are still random. The following proposition shows that E{V t (x t U J )} is an upper bound on V t (x t ), where the expectation is with respect to U J. Proposition 1 We have V t (x t ) E{V t (x t U J )} for all t T. Note that Proposition 1 implies that V 1 (c) E{V 1 (c U J )} and so we get an upper bound on the optimal expected revenue by solving problem (9). Besides giving an upper bound on the value function, the randomized dynamic program also simplifies the optimization problem by reducing the size of the action space. We show below that instead of optimizing over subsets of itineraries, it is sufficient to optimize over the individual itineraries. We introduce some notation first. We let p jt (U J ) = Pr{j t = j {j}, U J } = Pr{U jt > U φt U J } 7

8 be the probability that the customer purchases itinerary j when it is the only itinerary that is offered at time period t. Note that the last equality follows from the fact that the customer will purchase the itinerary only if its utility exceeds the utility of not purchasing anything. We use the argument U J to emphasize that this probability is conditional on the sampled utilities. The following lemma shows that at each time period, we can solve an optimization problem involving J decision variables as opposed to 2 J decision variables. Lemma 2 Consider the optimization problem { V t (x t U J ) = max p jt (U J )y jt r j + V t+1 (x t i L a ij e i U J ) V t+1 (x t U J )] } + V t+1 (x t U J ), subject to a ij y jt x it i L (10) (11) y jt 1 (12) y jt {0, 1} j J, (13) with the boundary condition V τ+1 ( U J ) = 0. We have V t (x t U J ) = V t (x t U J ) for all x t, t T. Although the number of decision variables in problem (10)-(13) is manageable, the size of the state space is still exponential in the capacities of the flight legs. On the other hand, noting that the decision variables in problem (10)-(13) are only over the itineraries, this problem has a similar structure to the network revenue management problem with independent demand. This allows us to use approximation ideas developed for the independent demand setting to reduce the complexity of the state space. We present two approximation methods in the following section. 4 Relaxations of the Randomized Dynamic Program In this section, we describe two tractable relaxations of problem (10)-(13). The first method is based on relaxing the flight leg capacity constraints using Lagrange multipliers. This yields an upper bound on the value function of the randomized dynamic program. We find the set of Lagrange multipliers which yields the tightest upper bound by solving a linear program. This idea is similar to that pursued in Kunnumkal and Topaloglu (2010a). The second method we propose is based on solving a perfect hindsight relaxation, where we have access to the customers utilities for not purchasing anything also. This method is similar to the randomized linear programming method of Talluri and van Ryzin (1999). We note that other approximation methods developed for the network revenue management problem with independent demand can also be applied to problem (10)-(13). In this paper, we particularly focus on the above mentioned two methods because they involve solving linear programs, which can be done quickly and efficiently. Speed is an important factor since we have to resolve the problems for many different samples. 8

9 4.1 Capacity Relaxation Letting Y = {y {0, 1} J : y j 1} and y t = {y jt : j J }, we consider relaxing constraints (11) by introducing Lagrange multipliers λ it and solve the optimization problem { V t (x t U J, λ) = max p jt (U J )y jt r j + V t+1 (x t ] i L a ij e i U J, λ) V t+1 (x t U J, λ) y t Y } + i L λ it (x it a ij y jt ) + V t+1 (x t U J, λ) (14) with the boundary condition that V τ+1 ( U J, λ) = 0. The following proposition shows that as long as the Lagrange multipliers are nonnegative, V t (x t U J, λ) is an upper bound on V t (x t U J ). Proposition 3 If λ = {λ it : i L, t T } 0, then we have V t (x t U J ) V t (x t U J, λ). Note that Propositions 1 and 3 together imply that as long as the Lagrange multipliers are nonnegative, we have V 1 (c) E{V 1 (c U J, λ)}. So we are naturally interested in finding the set of Lagrange multipliers that gives the tightest upper bound. solving the problem That is, for each sample U J, we are interested in min V 1(c U J, λ). λ 0 We next show that the above minimization problem reduces to solving a linear program and therefore is tractable. We begin with the following result, which gives a closed form expression for V t (x t U J, λ). Lemma 4 We have V t (x t U J, λ) = τ s=t Λ s + i L ( τ λ is )x it, where Λ t = max { p jt (U J ) r j i L a ij( τ s=t+1 λ is) ] i L a ijλ it } + and we use { } + = max{0, }. s=t Using the result in Lemma 4, we have that the problem min λ 0 V 1 (c U J, λ) can be solved as the linear program z CR (U J ) = min t T Λ t + ( i L t T λ it )c i subject to Λ t + ] a ij λ it + p jt (U J ) a ij (λ i,t λ iτ ) r j p jt (U J ) j J, t T i L Λ t 0 t T λ it 0 i L, t T, i L 9

10 with the understanding that λ i,τ+1 = 0. Taking the dual of this linear program, we get z CR (U J ) = max t T subject to r j p jt (U J )y jt (15) a ij y jt + ] a ij pj1 (U J )y j p j,t 1 (U J )y j,t 1 ci i L, t T(16) y jt 1 t T (17) y jt 0 j J, t T, (18) with the understanding that y j0 = 0. In the above linear program, we can interpret the decision variable y jt as the frequency with which we offer itinerary j for sale at time period t. Since p jt (U J )y jt represents the expected sales of itinerary j at time period t, we can interpret the first set of constraints as saying that the capacity consumed by the itineraries offered at time period t should not exceed the expected capacity consumed up to time period t, which is c i a ij pj1 (U J )y j p j,t 1 (U J )y j,t 1 ]. We emphasize that the expectations are conditional on the sampled utilities U J. The second set of constraints ensure that the total frequency with which we offer the individual itineraries at each time period is at most one. Letting ˆλ(U J ) = argmin λ 0 V 1 (c U J, λ), we have that E{V 1 (c U J, ˆλ(U J ))} = E{z CR (U J )} is an upper bound on the optimal expected revenue. Letting ˆλ it = E{ˆλ it (U J )}, we use τ s=t ˆλ is as the bid price of flight leg i at time period t. We approximate V t (x t ) by i L τ s=t ˆλ is x it and solve the problem max S Q(x t) Pr{j t = j S} r j a ij i L τ ] ˆλ is s=t (19) to decide on the set of itineraries to offer at time period t. As it becomes difficult to analytically compute the expectations E{z CR (U J )} and E{ˆλ it (U J )}, we resort to Monte Carlo simulation to estimate these quantities. In particular, we generate K samples of the customers utilities for the different itineraries UJ 1,..., U J K where U J k = {U jt k : j J, t T } are the utilities generated in the kth sample. We solve linear program (15)-(18) for each sample. Letting z CR (UJ k ) denote the optimal objective value and {ˆλ k it : i L, t T } denote optimal values of the dual variables corresponding to constraints (16), we use K k=1 z CR(UJ k )/K and K ˆλ k=1 k it /K as the sample estimates of E{z CR(U J )} and ˆλ it = E{ˆλ it (U J )}, respectively. 4.2 Perfect Hindsight Relaxation We consider another relaxation of problem (9) where we allow access to the customers utilities for not purchasing anything as well. In particular, letting U J = {U jt : j J, t T } be a sample of the customers utilities for the different itineraries at the different time periods and U φ = {U φt : t T } be a sample of the customers utilities for not purchasing anything and U = U J U φ, we solve the 10

11 optimization problem V t (x t U) = max S Q(x t ) { Pr{j t = j S, U} r j + V t+1 (x t ] } i L a ij e i U) V t+1 (x t U) +V t+1 (x t U). (20) Note that we can interpret problem (20) as determining the set of itineraries to offer at each time period after knowing the entire sample path: the customers utilities for the different itineraries as well as for not purchasing anything. Not surprisingly, it also gives an upper bound on problem (9). Proposition 5 We have V t (x t U J ) E{V t (x t U) U J } for all t T. Proposition 5 together with Proposition 1 imply that V 1 (c) E{V 1 (c U)}. Therefore, we obtain another upper bound on the optimal expected revenue by solving problem (20). Noting that Pr{j t = j S, U} = 1(U jt > U φt ) if j = argmax k S {U kt } and is zero otherwise, we have { V t (x t U) = 1(U jt > U φt ) r j + V t+1 (x t i L a ij e i U) V t+1 (x t U)] } + V t+1 (x t U). max j Q(x t) It follows that we can solve problem (20) as the following linear binary integer program: z P H (U) = max t T subject to r j 1(U jt > U φt )y jt (21) a ij 1(U jt > U φt )y jt c i i L (22) t T y jt 1 t T (23) y jt {0, 1} j J, t T. (24) In the above problem, the decision variable y jt indicates whether we offer itinerary j at time period t. The first set of constraints ensure that the total capacity consumed by the itinerary requests on each flight leg does not exceed its available capacity. The second set of constraints ensure that we offer at most one itinerary at each time period. We use E{V 1 (c U)} = E{z P H (U)} as an upper bound on the optimal expected revenue. In order to obtain a control policy, we solve the linear programming relaxation of problem (21)-(24). Letting ˆρ(U) = {ˆρ i (U) : i L} denote the optimal values of the dual variables corresponding to constraints (22), we use ˆρ i = E{ˆρ i (U)} as the bid price of flight leg i. We approximate V t (x t ) by i L ˆρ ix it and solve the problem max S Q(x t ) Pr{j t = j S} r j ] a ij ˆρ i i L (25) 11

12 to decide on the set of itineraries to offer at time period t. It again becomes difficult to analytically compute E{z P H (U)} and E{ˆρ i (U)} and so we resort to Monte Carlo simulation. In particular, we generate K samples of the customers utilities for the different itineraries as well as not purchasing anything U 1,..., U K where U k = {Ujt k : j J {φ}, t T } are the utilities generated in the kth sample. We solve problem (21)-(24) for each sample. Letting z P H (U k ) denote the optimal objective value, we use K k=1 z P H(U k )/K as the sample estimate of E{z P H (U)}. Letting {ˆρ k i : i L} denote the optimal values of the dual variables corresponding to constraints (22) in the linear programming relaxation of problem (21)-(24), we use K k=1 ˆρk i /K as an estimate of ˆρ i = E{ˆρ i (U)}. We close this section with a comment on the upper bounds obtained by problems (4)-(7), (15)-(18) and (21)-(24). It turns out that none of the upper bounds uniformly dominates the other. For example, consider a revenue management problem on a single flight leg for a single time period where we have two itineraries with r 1 = 11 and r 2 = 5. We have a single unit of capacity on the flight leg and each itinerary consumes one unit of capacity. Furthermore, we have U φ1 = 0 with probability 1. On the other hand, we have (1, 1) with probability 1/3 (U 11, U 21 ) = ( 1, 1) with probability 1/3 (1, 2) with probability 1/3. It is easy to verify that we have V 1 (c) = z CDLP = 22/3, while we have E{z P H (U)} = E{z CR (U J )} = 9. So, in this case we have that z CDLP < E{z P H (U)} = E{z CR (U J )}. It is also possible to come up with examples where the direction of the inequalities is reversed. On the other hand, in our computational experiments that we present next, we find that the capacity relaxation method consistently generates tighter upper bounds than the perfect hindsight relaxation method which in turn is tighter than the choice based deterministic linear program. 5 Computational Experiments In this section, we numerically compare the performance of the choice based deterministic linear program, the capacity relaxation method and the perfect hindsight relaxation method. We first describe the benchmark solution methods. After that we present our experimental setup and the results of the numerical study. Choice Based Deterministic Linear Program (CDLP): This is the solution method that we describe in Section 2. In our practical implementation, we divide the booking horizon into five equal segments. At the beginning of each segment, we solve problem (4)-(7) after replacing the right hand side of equation (5) with the remaining capacities on the flight legs and the set of time periods T with the current set of remaining time periods. We get a fresh set of optimal dual values {ˆπ i : i L} and we plug them into decision rule (8) to decide on the set of itineraries to offer. We continue to use this decision rule until the beginning of the next segment, where we resolve problem (4)-(7). Capacity Relaxation (CR): This is the solution method that we describe in Section 4.1. In our practical implementation, we divide the booking horizon into five equal segments. At the beginning of each 12

13 segment, we solve problem (15)-(18) after replacing the right hand side of equation (16) with the remaining capacities on the flight legs and the set of time periods T with the current set of remaining time periods. We repeat this for K samples to get a fresh set of dual values {ˆλ k it : i L, t T, k K} and use these in decision rule (19) to decide on the set of itineraries to offer. We continue to use this decision rule until the beginning of the next segment, where we resolve problem (15)-(18). We use K = 100 in our computational experiments. Increasing the value of K further did not result in any noticeable changes in performance. Perfect Hindsight Relaxation (PH): This is the solution method that we describe in Section 4.2. As with CDLP and CR, in our practical implementation, we divide the booking horizon into five equal segments. At the start of each segment, we refresh our bid prices by solving the linear programming relaxation of problem (21)-(24) after replacing the right hand side of equation (22) with the remaining capacities on the flight legs and the set of time periods T with the current set of remaining time periods. We repeat this for K samples and use the fresh set of optimal dual values {ˆρ k i : i L, k K} in decision rule (25) to decide on the set of itineraries to offer. We continue to use this decision rule until the beginning of the next segment, where we again resolve problem (21)-(24). As in CR, we use K = 100 in our computational experiments. We note that all of the above mentioned benchmark methods obtain bid prices that are capacity independent, in that they do not naturally change with the capacities on the flight legs. It is possible to obtain capacity dependent bid prices by using the optimal dual values obtained by the benchmark methods in a dynamic programming decomposition scheme as suggested by Liu and van Ryzin (2008) or Zhang (2011). We do not pursue that here for a number of reasons. Capacity dependent bid prices typically come with a higher overhead, both in terms of computation and implementation. Numerical studies also indicate that the performance gap between capacity independent bid prices remains intact when we use them in a dynamic programming decomposition scheme; see for example Kunnumkal and Topaloglu (2010c). We test the performance of the benchmark solution methods on two groups of test problems. The first group involves an airline network with a single hub serving multiple spokes, while the second group of test problems have an airline network with two hubs serving multiple spokes. Our test problems closely parallel those in Kunnumkal and Topaloglu (2010b). 5.1 Airline Network with a Single Hub We consider an airline network with a single hub that serves N spokes. Half of the spokes have two flights to the hub, while the remaining half have two flights from the hub. The total number of flights is 2N. Figure 1 shows the structure of the airline network with N = 8. There are four itineraries between each spoke-to-hub and hub-to-spoke origin destination pair. On the other hand, we have eight itineraries between each spoke-to-spoke origin destination pair, so that the total number of itineraries is 2N(N + 2). Half of these itineraries are high fare itineraries while the other half are low fare itineraries. We let γ denote the ratio between the high fare and the low fare. 13

14 Each origin destination pair is associated with a customer segment. We let K denote the set of customer segments. At each time period a customer from segment l K arrives with probability λ l. An arriving customer is interested only in the set of itineraries connecting the origin destination pair that it is associated with. Therefore, the consideration sets of the different customer segments are disjoint. Customer choice is governed by the multinomial logit model. In the multinomial logit model, the utility for purchasing itinerary j that is in the consideration set of customer segment l is given by U ljt = u ljt + ξ ljt, where u ljt is a constant called the nominal utility and ξ ljt is a Gumbel random variable with mean zero and scale parameter one. The utility for not purchasing anything for customer segment l is U lφt = u lφt + ξ lφt, where u lφt is the nominal utility for not purchasing anything and ξ lφt is a Gumbel random variable with mean zero and scale parameter one. The random variables {ξ ljt : j J {φ}, t T } are independent; see Ben-Akiva and Lerman (1994). We measure the tightness of the leg capacities in the same manner as Zhang and Adelman (2009). Letting S t = argmax S J r j Pr{j t = j S} be the offer set that maximizes expected revenue at time period t when there is ample capacity on all the flight legs, we use l K α = λ l t T i L a ij Pr{j t = j St } i L c, i to measure the tightness of the leg capacities. We have T = 200 time periods in all of our test problems. We vary N, γ and α to obtain different test problems. We label our test problems by the triplet (N, γ, α) {8, 10, 12} {1.5, 3} {1.3, 1.6}, where N is the number of spokes, γ is the ratio between the high and low fare itineraries and α measures the tightness of the leg capacities. This gives us a total of twelve test problems. Table 1 compares the upper bounds obtained by CR, PH and CDLP. The first column in this table gives the characteristics of the problem by using (N, γ, α). The second, third and fourth columns, respectively, give the upper bounds obtained by CR, PH and CDLP. The fifth column gives the percentage gap between the upper bounds obtained by PH and CR, while the last column gives the percentage gap between the upper bounds obtained by CDLP and CR. CR performs consistently well in our computational experiments and we use CR as a benchmark. In the last two columns, a indicates that the gap is significant at the 95% level, while a indicates that the gap is not significant at the 95% level. We observe that CR generates significantly tighter upper bounds than PH and CDLP. On average, the upper bounds obtained by CR are about 2% tighter than PH and 8% tighter than CDLP. Table 2 compares the total expected revenues obtained by CR, PH and CDLP. We evaluate the expected revenues by simulation and use common random numbers in our simulations. The columns have a similar interpretation as in Table 1 except that they give the expected revenues obtained by the three methods. The last two columns include a if CR does better than the respective solution method at the 95% level, a otherwise and a if there does not exist a statistically significant difference between the two. The average gap between the total expected revenues obtained by CR and CDLP is around 2%. The performance gaps are statistically significant in ten out of the twelve test problems. The performance gap between CR and CDLP seems to increase with the fare ratio and the tightness of the leg capacities. The performance gaps between CR and PH are small in most 14

15 cases, although we observe one instance where PH performs about 1% better than CR. PH performs significantly better than CDLP. The average gap between the total expected revenues obtained by PH and CDLP is around 2%. 5.2 Airline Network with Two Hubs We consider an airline network with two hubs that serve N spokes in total. Half of the spokes have two flights to the first hub, while the other half have two flights from the second hub. In addition, there are four flights from the first to the second hub. The total number of flights is 2N + 4. Figure 2 shows the structure of the airline network with N = 8. We randomly sample from the set of all the possible itineraries so that the total number of itineraries is around 4N 2. Half of these itineraries are high fare itineraries while the other half are low fare itineraries. Similar to the test problems with a single hub, each origin destination pair is associated with a customer segment. An arriving customer belongs to one of the segments and is interested only in the set of itineraries connecting the origin destination pair that it is associated with. We continue to assume that customer choice is governed by the multinomial logit model with disjoint consideration sets. We label the test problems by the triplet (N, γ, α) {4, 6, 8} {1.5, 3} {1.3, 1.6}, which gives us a total of twelve test problems. Table 3 compares the upper bounds obtained by CR, PH and CDLP. The columns have the same interpretation as in Table 1. The results display the same trends that we observed for the airline network with a single hub. CR consistently generates the tightest upper bounds, followed by PH and CDLP. On average, the upper bounds obtained by CR are about 2% tighter than PH and 9% tighter than CDLP. Table 4 compares the total expected revenues obtained by CR, PH and CDLP. CR generates significantly higher revenues than CDLP. The average gap between the total expected revenues obtained by CR and CDLP is around 2%, although we observe test problems where the gap is as high as 5%. We find one test problem where the performance gap between PH and CR is around 2.5%, but the gaps are quite small and insignificant in the remaining cases. The average performance gap between PH and CDLP is around 2%. The ratio between the high and low fares and the tightness of the leg capacities seem to be two factors which contribute to increasing the performance gaps between CDLP and the other two solution methods. Problems with large differences between the high and low fares and tight leg capacities tend to be more difficult to solve, because the consequences of offering the wrong set of itineraries tend to be more severe. It is therefore encouraging that CR and PH provide good performance for such test problems. All of the computational experiments are carried out on a Pentium Core 2 Duo desktop with 3 GHz CPU and 3 GB RAM running Windows XP. The running time of CDLP is of the order of seconds. For K = 100 samples, the running time of PH is of the order of seconds, while that of CR is in minutes. CR takes about a minute and a half to solve the largest test problem. 15

16 6 Conclusions We presented new methods to obtain upper bounds and bid prices for the network revenue management problem with customer choice behavior. The starting point for our methods is a dynamic programming approximation that we solve for a sample of the customers utilities for the different itineraries. An attractive feature of this randomized dynamic program is that the number of decision variables is linear in the number of itineraries. As a result, we are able to reduce the complexity of the action space. We build on this randomized dynamic program to obtain two tractable approximation methods. The first method that we propose involves relaxing the flight leg capacity constraints using Lagrange multipliers. The second method involves solving a perfect hindsight relaxation. We showed that both methods give upper bounds on the optimal expected total revenue. Our methods may also be appealing from a practical standpoint as they involve solving only linear programs. Computational experiments indicate that our methods can significantly improve upon the upper bounds and expected revenues obtained by the choice based deterministic linear program. Appendix Proof of Proposition 1 We show the result by induction over the time periods. It is easy to show that the result holds at time period τ. Assuming the result holds at time period t+1, we show that it holds at time period t. Letting ˆS be an optimal solution for problem (3), we note that ˆS is feasible for problem (9). We also note that for a given offer set S, Pr{j t = j S, U J } is a function of {U jt : j J }, while V t+1 (x t U J ) is a function of {U js : j J, s {t + 1,..., T }}. Since the random variables {U jt : j J } are independent across time, it follows that { } E Pr{j t = j S, U J }V t+1 (x t U J ) = E { Pr{j t = j S, U J } } E { V t+1 (x t U J ) }, (26) where the expectation is with respect to U J. Therefore, we have E { V t (x t U J ) } E { Pr{j t = j ˆS, U J } } r j + E { V t+1 (x t i L a ij e i U J ) }] + 1 E { Pr{j t = j ˆS, U J } }] E { V t+1 (x t U J ) } Pr{j t = j ˆS} r j + V t+1 (x t ] i L a ij e i ) + 1 Pr{j t = j ˆS} ] V t+1 (x t ) = V t (x t ), where the first inequality uses (26) and the fact that ˆS is a feasible but not necessarily optimal solution to problem (9) and the second inequality follows from the induction assumption and the fact that { } { Pr{j t = j S} = E 1(U jt = max {U kt}) = E E { 1(U jt = max {U } } kt}) U J = E { Pr{j t = j S, U J } }. k S {φ} k S {φ} 16

17 Proof of Lemma 2 We show the result by induction over the time periods. It is easy to show that the result holds at time period τ. Assuming the result holds at time period t + 1, we show that it holds at time period t. We first show that V t (x t U J ) V t (x t U J ). Let y t = { y jt : j J } be an optimal solution to problem (10)-(13) and let S = {j J : y jt = 1}. Note that since y t satisfies (11)-(13), the offer set S is feasible for problem (9). We have V t (x t U J ) = p jt (U J ) y jt r j + V t+1 (x t i L a ij e i U J ) V ] t+1 (x t U J ) + V t+1 (x t U J ) = Pr{j t = j S, U J } r j + V t+1 (x t ] i L a ij e i U J ) V t+1 (x t U J ) + V t+1 (x t U J ) V t (x t U J ), where the first equality follows from the optimality of y t, the second equality follows from the fact that S 1 and Pr{j t = j S, U J } = 0 for j / S and the induction assumption. The inequality holds since S is feasible for problem (9). This implies V t (x t ) V t (x t ). To show the reverse inequality, let ˆS be the optimal solution to problem (9), ˆȷ = argmax k ˆS{U kt } and ˆy t = {ˆy jt : j J } with ˆy jt = 1 for j = ˆȷ and ˆy jt = 0 for j J {ˆȷ}. Note that since ˆS Q(x t ), we have that ˆy t satisfies constraints (11)-(13). We have Pr{j t = j ˆS, U J } = Pr{U jt > U φt U J } = p jt (U J ) for j = ˆȷ. On the other hand, since U jt < Uˆȷt for j ˆS {ˆȷ}, we have Pr{j t = j ˆS, U J } = 0 for j ˆS {ˆȷ}. Also, note that Pr{j t = j ˆS, U J } = 0 for j / ˆS. Using the above facts and the optimality of ˆS, we have V t (x t U J ) = Pr{j t = j ˆS, U J } r j + V t+1 (x t ] i L a ij e i U J ) V t+1 (x t U J ) + V t+1 (x t U J ) = pˆȷt (U J ) rˆȷ + V t+1 (x t ] i L a iˆȷ e i U J ) V t+1 (x t U J ) + V t+1 (x t U J ) = p jt (U J )ˆy jt r j + V t+1 (x t i L a ij e i U J ) V ] t+1 (x t U J ) + V t+1 (x t U J ) V t (x t U J ), where the last equality follows from the induction assumption and the fact that ˆy jt = 0 for j J {ˆȷ} and the inequality holds since ˆy t is feasible for problem (10)-(13). Proof of Proposition 3 We show the result by induction over the time periods. It is easy to show that the result holds at time period τ. Assuming the result holds at time period t + 1, we show that it holds at time period t. Using the equivalent representation of V t (x t U J ) in Lemma 2 and letting y t = { y jt : j J } be an optimal 17

18 solution to problem (10)-(13), we have V t (x t U J ) = p jt (U J ) y jt r j + V t+1 (x t ] i L a ij e i U J ) + i L + 1 ] p jt(u J ) y jt V t+1 (x t U J ) p jt (U J ) y jt r j + V t+1 (x t ] i L a ij e i U J, λ) + 1 ] p jt(u J ) y jt V t+1 (x t U J, λ) λ it (x it a ij y jt ) V t (x t U J, λ), where the first inequality uses the induction assumption and the facts that since y t satisfies constraints (11)-(13) and λ 0, we have 1 p jt(u J ) y jt ] 0 and λ it (x it a ij y jt ) 0. The last inequality holds since y t is a feasible solution to problem (14). Proof of Lemma 4 We show the result by induction over the time periods. It is easy to show that the result holds at time period τ. Assuming the result holds at time period t + 1, we show that it holds at time period t. We have V t (x t U J, λ) = max y t Y + = max y t Y = { p jt (U J )y jt rj a ij ( i L τ Λ s + s=t+1 i L { τ s=t Λ s + i L ( τ s=t+1 λ is )x it y jt ( p jt (U J ) r j i L a ij ( ( τ λ is )x it, s=t τ s=t+1 λ is ) ] + i L λ it (x it τ s=t+1 λ is ) ] i L a ij λ it )} + } a ij y jt ) τ s=t+1 Λ s + i L ( τ λ is )x it where the first { equality follows from the induction assumption and the last equality uses the fact ( that max yt Y y jt p jt (U J ) r j i L a ij( τ s=t+1 λ is) ] )} i L a ijλ it = max {p jt (U J ) r j i L a ij( τ s=t+1 λ is) ] } + i L a ijλ it = Λt. s=t Proof of Proposition 5 We show the result by induction over the time periods. It is easy to show that the result holds at time period τ. Assuming the result holds at time period t+1, we show that it holds at time period t. Letting ˆS be an optimal solution for problem (9), we note that ˆS is feasible for problem (20). We also note that for a given offer set S, Pr{j t = j S, U} is a function of {U jt : j J {φ}}, while V t+1 (x t U) is a function of {U js : j J {φ}, s {t + 1,..., T }}. Since the random variables {U jt : j J {φ}} are independent across time, it follows that { } E Pr{j t = j S, U}V t+1 (x t U) U J = E { } { } Pr{j t = j S, U} U J E Vt+1 (x t U) U J, (27) 18

19 where the expectation is with respect to U φ and we recall that U = U J U φ. Therefore, we have E { } V t (x t U) U J E { Pr{j t = j ˆS, } U} U J r j + E { V t+1 (x t i L a } ] ij e i U) U J + 1 E { Pr{j t = j ˆS, } ] U} U J E { } V t+1 (x t U) U J Pr{j t = j ˆS, U J } r j + V t+1 (x t ] i L a ij e i U J ) + 1 Pr{j t = j ˆS, ] U J } V t+1 (x t U J ) = V t (x t U J ), where the first inequality follows from (27) and the fact that ˆS is a feasible but not necessarily optimal solution to problem (20) and the second inequality follows from the induction assumption and the fact that { Pr{j t = j S, U J } = E 1(U jt > } { max {U kt}) U J = E E { 1(U jt > k S {φ} max {U kt}) U }} k S {φ} = E { Pr{j t = j S, U} }. References Ben-Akiva, M. and Lerman, S. (1994), Discrete Choice Analysis: Theory and Applications to Travel Demand, The MIT Press, Cambridge, MA. Bront, J. J. M., Mendez-Diaz, I. and Vulcano, G. (2009), A column generation algorithm for choicebased network revenue management, Operations Research 57, Chaneton, J. and Vulcano, G. (2009), Computing bid-prices for revenue management under customer choice behavior, Working paper, New York University, New York City, NY. Gallego, G., Ratliff, R. and Shebalov, S. (2010), A general attraction model and an efficient formulation for the network revenue management problem, Working paper, Columbia University, New York. Kunnumkal, S. and Topaloglu, H. (2008), A refined deterministic linear program for the network revenue management problem with customer choice behavior, Naval Research Logistics Quarterly 55, Kunnumkal, S. and Topaloglu, H. (2010a), Computing time-dependent bid prices in network revenue management problems, Transportation Science 44(1), Kunnumkal, S. and Topaloglu, H. (2010b), A new dynamic programming decomposition method for the network revenue management problem with customer choice behavior, Production and Operations Management 19(5), Kunnumkal, S. and Topaloglu, H. (2010c), A randomized linear program for the network revenue management problem with customer choice behavior, Journal of Revenue and Pricing Management. (to appear). Liu, Q. and van Ryzin, G. (2008), On the choice-based linear programming model for network revenue management, M&SOM 10(2),

20 Mahajan, S. and van Ryzin, G. (2001), Stocking retail assortments under dynamic consumer substitution, Operations Research 49(3), Meissner, J. and Strauss, A. K. (2008), Network revenue management with inventory sensitive bid prices and customer choice, Working paper, Department of Management Science, Lancaster University. Meissner, J. and Strauss, A. K. (2010), Choice-based network revenue management under weak market segmentation, Working paper, Department of Management Science, Lancaster University. Meissner, J., Strauss, A. K. and Talluri, K. (2011), An enhanced concave program relaxation for choice network revenue management, Working paper, Universitat Pompeu Fabra, Barcelona, Spain. Talluri, K. (2010), A randomized concave programming method for choice network revenue management, Working paper, Universitat Pompeu Fabra, Barcelona, Spain. Talluri, K. and van Ryzin, G. (1999), A randomized linear programming method for computing network bid prices, Transportation Science 33(2), Talluri, K. and van Ryzin, G. (2004), The Theory and Practice of Revenue Management, Kluwer Academic Press. van Ryzin, G. and Vulcano, G. (2008), Computing virtual nesting controls for network revenue management under customer choice behavior, M&SOM 10, Zhang, D. (2011), An improved dynamic programming decomposition approach for network revenue management, M&SOM 13, Zhang, D. and Adelman, D. (2009), An approximate dynamic programming approach to network revenue management with customer choice, Transportation Science 43, Zhang, D. and Cooper, W. L. (2005), Revenue management for parallel flights with customer-choice behavior, Operations Research 53,