Simulation-Based Booking Limits for Airline Revenue Management

Transcription

1 OPERATIONS RESEARCH Vo. 53, No. 1, January February 2005, pp issn X eissn informs doi /opre INFORMS Simuation-Based Booking Limits for Airine Revenue Management Dimitris Bertsimas, Sanne de Boer Operations Research Center, Massachusetts Institute of Technoogy, 77 Massachusetts Avenue, E40-130, Cambridge, Massachusetts Deterministic mathematica programming modes that capture network effects pay a predominant roe in the theory and practice of airine revenue management. These modes do not address important issues ike demand uncertainty, nesting, and the dynamic nature of the booking process. Aternativey, the network probem can be broken down into eg-based probems for which there are satisfactory soution methods, but this approach cannot be expected to capture a reevant network aspects. In this paper, we propose a new agorithm that addresses these issues. Starting with any nested bookingimit poicy, we combine a stochastic gradient agorithm and approximate dynamic programming ideas to improve the initia booking imits. Preiminary simuation experiments suggest that the proposed agorithm can ead to practicay significant revenue enhancements. Subject cassifications: simuation: appications; inventory: perishabe items; transportation: airines. Area of review: Manufacturing, Service, and Suppy Chain Operations. History: Received January 2001; revisions received November 2001, December 2003; accepted December Introduction After the dereguation of the airine industry in the 1970s, airines started offering a variety of fares for seats in the same cabin. The question of how many seats shoud be offered at each different rate is commony referred to as the airine revenue management (RM) probem. The economic importance of RM is iustrated by Deta Airines estimate that seing ony one seat per fight at fu rather than at discount rate adds over $50 miion to its annua revenues (Cross 1997). The probem of optimizing the passenger mix on a singe-eg fight has received a ot of attention in the academic iterature. The eary mode by Littewood (1972) for the basic case with ony two fare casses is based on the concept of the expected margina seat revenue (EMSR). This is the expected margina revenue of hoding an additiona seat for a certain fare cass. Beobaba (1987, 1989) generaized this approach to a heuristic booking poicy for mutipe fare casses, which was extended by Beobaba and Weatherford (1996) to incorporate se-ups. Brumee et a. (1990) examine the impact of demand correation. More recent modes use stochastic dynamic programming (SDP) to determine an optima poicy, e.g., Womer (1992), Brumee and McGi (1993), Robinson (1995), and Lee and Hersh (1993). Lautenbacher and Stidham (1999) provide a unified mode of this and reated work. These modes have been extended to incorporate important practica issues such as overbooking, canceations, and no-shows (e.g., Chatwin 1996, 1998; Subramanian et a. 1999). See Weatherford (1998) for a discussion of the issues reated to RM and McGi and van Ryzin (1999) for a comprehensive overview of the iterature in this fied. Whie booking poicies such as Beobaba s EMSR have proven to be very profitabe, it was recognized in the eary 1990s that an airine shoud aim to optimize its bookings over its network as a whoe, rather than on each fight eg in isoation. Wiiamson (1992) used simuation to show that expicity addressing the network aspect of the revenue management probem eads to a significant increase of expected revenue over eg-based methods. The impementation of such origin-destination based poicies is sti in progress at many major airines (e.g., Saranathan et a of United Airines and Pagé 1999 of Air Canada). Athough in theory the network aspect can easiy be added to an SDP mode (e.g., Gaego and van Ryzin 1997), in practice this is infeasibe. Because the size of the state space is determined by the number of avaiabe seats on each fight of the network, the number of casses, and the number of possibe passenger itineraries, the probem size expodes even for a moderate-size airine. This curse of dimensionaity of the SDP method necessitates the deveopment of aternative methods. Mathematica programming (MP) modes are especiay we suited to incorporate network effects, i.e., to recognize which itineraries contribute most to the airine s revenue. Dimensionaity probems, however, necessitate the use of inear optimization modes, which we discuss in the next section. The soution of these modes can be used to impement two different forms of booking contro that 90

2 Operations Research 53(1), pp , 2005 INFORMS 91 are used in practice, one based on booking imits and the other based on bid prices. The focus of this research is on booking-imit contro. The MP modes that have been proposed, whie effective in addressing network effects, are deterministic, static, and partitioned, thus ignoring the stochastic and dynamic nature of the demand and the nested character of booking-imit contro in a network. As a resut, poicies based on the MP seat aocations may ony capture part of the potentia revenue gain. In practice, most RM systems ony use the dua soution of the MP to heuristicay account for dispacement costs. The network probem can then be broken down into eg-based probems for which there are satisfactory soutions. However, this approach cannot be expected to capture a reevant network aspects. Our overa objective in this research is to propose a new way to cacuate booking imits that takes into account the stochastic and dynamic nature of the demand and the nested character of booking-imit contro in a network, is practicay feasibe for reaisticay sized probems, and eads to revenue enhancements. The basic idea of our approach is to approximate the expected revenue of any nested booking-imit poicy by simuating the booking process unti the current booking imits are up for revision. From there, a vaue function estimate is used to approximate the revenue that woud have been generated in the remaining part of the booking period. This revenue approximation scheme is embedded in a stochastic gradient-type agorithm to iterativey improve any initia set of booking imits. In particuar, these coud have been determined by any of the methods referred to above. The contributions of this research are: 1. We propose a new approach to airine RM that, buiding upon RM research in the ast decade, combines inear optimization modes that successfuy address the network effects with the newer ideas in this fied of simuation-based optimization and approximate dynamic programming. To the best of our knowedge, our method is the first to simutaneousy address nesting as we as the stochastic and dynamic nature of demand in a network environment. 2. We provide computationa evidence that indicates that the proposed method can ead to revenue enhancements over current methodoogy and is computationay feasibe for reaisticay sized probems. This paper is structured as foows. In 2, we discuss the state of the art of origin-destination based seat inventory contro. We present the MP modes that have been proposed for this purpose and discuss booking-imit and bid-price contro. In particuar, this section contains the first step of our approach to the RM probem by determining an initia booking-imit poicy. In 3, we present the second part of our overa approach, which uses simuation-based optimization. In 4, we provide computationa evidence of the strength of our approach and its practica appicabiity. Section 5 summarizes our concusions. 2. Mathematica Programming-Based Approaches to Network Revenue Management In what foows, we use the term booking cass to refer to the typica combination of origin, destination, and fare cass, which we index by odf. Gover et a. (1982) proposed an integer programming mode to determine the number of seats that shoud be avaiabe to each booking cass, with the foowing inear programming reaxation: maximize f odf x odf odf subject to x odf E D odf odf x odf C odf S = 1 L (1) x odf 0 Here and in the remainder of this paper, f odf, E D odf, and x odf are the fare, expected demand, and the number of seats aocated to booking cass odf, respectivey; C is the seat capacity of eg, L is the tota number of egs, and S is the set of booking casses that trave through eg. Note that Mode (1) ony encompasses a singe fight compex in a singe time window. Many airines operate severa fights per day between city pairs and offer mutipe connections between the same origin and destination. The indexing of fights and odfs can easiy be extended to incorporate this aspect. It is we known that if the itinerary between any origin and destination is not fixed and the airine can route its passengers over its network, Mode (1) can be reformuated as a network fow probem. In practice, however, this is not the case, and even if the demand forecasts are integer, this mode does not necessariy have an integer soution. Given that it is computationay unattractive to sove an integer program, we focus on soving the reaxation. Its soution can be used to impement two different types of booking poicies: one based on booking imits and another based on bid prices Booking-Limit Contro Let xodf be an optima soution of Mode (1). The poicy represented by the mode aocates up to xodf seats to cass odf. A disadvantage of such a poicy is that it partitions the seat capacity of the airine s network. When seats for a booking cass are sod out, additiona booking requests for this cass wi be decined, which can ead to ost revenue. Intuitivey, seats aocated to the east profitabe booking casses shoud be made avaiabe to more profitabe casses as we, which is caed nesting. Smith and Penn (1988) of American Airines proposed the Dispacement Adjusted Virtua Nesting scheme to impement nested booking-imit contro in a network, which is outined beow. The operationa poicy was not specified, but we

3 92 Operations Research 53(1), pp , 2005 INFORMS incude both methods that are used in practice (steps 4 and 4 ). (We thank Dr. Peter Beobaba for insightfu comments on these methods.) Dispacement Adjusted Virtua Nesting (DAVN) Step 1 (Nesting order). Let = 1 L be an optima set of dua prices corresponding to the capacity constraints of Mode (1). Cacuate the dispacement adjusted eg revenues f odf = f odf odf S = 1 L This is a measure of profitabiity that determines a nesting order of booking casses on each eg. Step 2 (Custering). Custer booking casses of simiar adjusted eg revenue into a manageabe number (in practice about 5 10) of eg buckets on each eg. These buckets are ordered from high to ow, such that bucket 1 on eg contains the most profitabe booking casses on this eg, foowed by bucket 2, and so forth. In what foows, et Bi be the set of booking casses odf in bucket i on eg, et Iodf be the bucket of booking cass odf on eg, et N be the number of buckets defined on eg, and et N = L =1 N be the tota number of eg buckets in the network. For competeness, the custering agorithm we used in this research is presented in the appendix. Step 3 (Booking-imit cacuation). Define a booking imit bi for each eg bucket i on eg, such that bi b i+1 for i = 1 N 1 and b1 = C, because in the absence of overbooking, the sae of booking casses in the highest bucket shoud ony be restricted by capacity. In this research, we have considered two booking-imit cacuation methods, one based on the LP seat aocations (LP-BL) originay proposed by Wiiamson (1992), and the other (EMSR-BL) based on the we-known EMSRb method proposed by Beobaba (1992), which uses the dispacement adjusted eg revenues. For competeness, both methods are described in the appendix. Step 4 (Operationa poicy 1: Standard nesting). Interpret bi as the maximum number of seats we are wiing to se on eg for booking casses mapped into bucket i or ower. Let di N denote the accepted demand for buckets i through N (i.e., bucket i and ower) on eg since the most recent booking-imit cacuation. These variabes are updated continuousy between reoptimizations of the booking poicy. Consider a request for booking cass odf.ifon any of its egs odf S the booking imit for bucket i = Iodf or any higher bucket i<iodf has been reached (i.e., di N equas bi ), the booking request is decined; otherwise, it is accepted and the accepted demand variabes are updated accordingy (di N di N + 1 for a egs odf S and inventory buckets i Iodf ). Step 4 (Operationa poicy 2: Theft nesting). Stops seing seats on eg to booking casses mapped into bucket i after bi seats have been sod on this eg, regardess of to whom. Let d denote the accepted demand for seats on eg (i.e., for odf S ) since the most recent bookingimit cacuation. These variabes are updated continuousy between reoptimizations of the booking poicy. Consider a request for booking cass odf. If on any of its egs odf S the booking imit for bucket i = Iodf has been reached (i.e., d equas bi ), the booking request is decined; otherwise, it is accepted and the accepted demand variabes are updated accordingy (d d + 1 for a egs odf S ). Note that theft nesting is more restrictive than standard nesting because ow buckets wi cose earier in the booking process. This wi ead to a higher yied, but a ower oad factor. To our knowedge, industry experts differ in opinion about which poicy is better, and both may be found in practice. As far as we know, DAVN in combination with EMSR- BL is currenty being used by ony a handfu of airines wordwide. The term virtua nesting was chosen by Smith and Penn (1988) to refect that the avaiabiity of a booking cass is never stored in the system, which given the dimensions of the probem is impractica, but can be determined when needed from the eg-bucket avaiabiities. As they point out, it is not an optimization technique, but a contro framework that aows a reservation system to approximate market cass (odf ) contro. In 3, we propose a new method to cacuate booking imits within the framework of DAVN Bid-Price Poicy An aternative booking poicy is based on the genera idea of rejecting a request uness its fare exceeds the expected opportunity cost of not being abe to se the requested seats at a ater time, which can be seen as the bid price of a particuar itinerary. There are severa ways to approximate the true bid prices of itineraries on a fight network. Simpson (1989) proposed the foowing method based on the soution of Mode (1). Bid-Price Poicy (BP) Step 1. Sove Mode (1) and, using an optima set of dua variabes for each eg, cacuate the net contributions to network revenue f odf = f odf odf S. Step 2. If an incoming booking request has f odf 0, accept the request; otherwise, reject it. One obvious disadvantage of this poicy is that once a booking cass is open to bookings, there is no imit on the number of booking requests that are accepted for this cass. This induces the risk of fights fiing up with passengers from booking casses that ony marginay contribute to network revenue. To prevent this, it is essentia that the bid prices are updated frequenty during the booking period. An aternative soution, which we have not considered in this research, is to use dynamic bid prices that adjust as capacity is consumed. Van Ryzin (1998) argues that dynamic bid prices offer essentiay the same eve of contro over the booking process as booking imits do. Additiona issues

4 Operations Research 53(1), pp , 2005 INFORMS 93 reated to bid-price contro are discussed in Wiiamson (1992), Tauri and van Ryzin (1998), Chen et a. (1999), and Bertsimas and Popescu (2000) Modeing Issues The LP-based booking imits ceary need not be optima for a given virtua nesting scheme, because they are based on an inaccurate representation of the RM probem. Mode (1) is deterministic, static, and determines a seat partitioning, whie in reaity airine demand is stochastic, the booking process is dynamic, and seat inventory contro is nested. There have been some proposas to address the first issue and incude stochastic demand. Womer (1986) proposes an optimization mode invoving binary decision variabes x odf i that indicate whether i or more seats in the network are aocated to a particuar booking cass. The objective coefficient of x odf i is the expected margina revenue of aocating an additiona ith seat to the booking cass odf. The drawback of this mode is the arge number of decision variabes, which severey imits its practica appicabiity. De Boer et a. (2002) propose a scenario-based stochastic programming mode to overcome these dimensionaity probems, whie Tauri and van Ryzin (1999) propose a randomized inear programming approach. However, numerica resuts by Wiiamson (1992) and de Boer et a. (2002) show that, in fact, the deterministic mode gives the best input for the LP-BL heuristic. This suggests that demand uncertainty and nesting need to be addressed simutaneousy in the booking-imit cacuation step. Curry (1990) proposes a two-step agorithm in this direction. First, a inear programming mode with a piecewise inear approximation of the expected revenue as a function of the seat partitioning is used to determine seat aocations for each itinerary in the network. Then, given these aocations, nested booking imits are cacuated for each fare cass based on a singe-eg method that incorporates demand uncertainty. However, Curry s method is not suited to nest a booking casses on a fight eg simutaneousy, which, given the oftentimes arge fare differences between itineraries with common fight egs, is desirabe. The EMSR-BL method is another way to approach this probem. The booking imits are cacuated for each eg individuay, but the network aspect is to some extent captured by the dispacement adjusted eg revenues. However, by breaking up the probem we do ose some oweve characteristics that coud affect the optima set of booking imits for a given virtua nesting scheme. For instance, the optima mix of oca and connecting traffic shoud depend on the probabiity of an accepted connecting passenger dispacing oca passengers on a his fight egs simutaneousy, which cannot be captured propery by an average dispacement measure. In addition, the EMSR method is based on the assumption that there is a strict ow-before-high arriva order of booking requests for different fare casses. Athough this is generay beieved to be a workabe assumption for practica RM purposes, there is no reason that there woud be a particuar arriva order of booking requests for the different itineraries that make up the buckets in the virtua nesting scheme. Note that in that case, the choice between standard and theft nesting affects revenue, but this is not incuded in the mode. This may affect the performance of the EMSR-based booking imits in a network environment as we Other Methods Severa other methods have been proposed to impement booking contro in a network. The first such approach, somewhat simiar to DAVN, is to break up the network probem into eg-based probems by aocating the revenue of a mutieg fight over its egs, which is caed prorating. For these subprobems, we do have satisfactory stochastic and dynamic soution methods. Smith and Penn (1988) and Wiiamson (1992) consider prorating based on the mieage of the individua fight egs, whie Bratu (1999) studies an iterative prorating agorithm based on the expected margina seat revenue of the ast seat on each eg. However, despite some promising test resuts by Bratu, there is no intuitive expanation of why the prorating approach shoud correcty take into account network effects. The second approach is approximate dynamic programming (ADP). Given that exact SDP is not appicabe due to dimensionaity probems, Bertsimas and Popescu (2000) propose to approximate the SDP vaue function by Mode (1). This is a form of certainty equivaent contro, which is one of the heuristic soution techniques for compex dynamic programming probems discussed in Bertsekas (1995). Chen et a. (1998) show that the objective vaue of Mode (1) is in fact an upper bound on the optima expected revenue, whie any nonnested probabiistic mode such as Womer s (1986) provides a ower bound. They argue that the opportunity costs of seing a particuar itinerary are actuay underestimated by the stochastic mode, whereas they are overestimated by the deterministic mode. This insight eads to an agorithm in which the revenue of every booking request is compared to both estimates of the opportunity costs. The idea is to diminish the number of wrong decisions resuting from the biased estimates of the opportunity costs that are based on any such singe mode. Simuation experiments indicate that this approach indeed eads to higher revenues than the BP poicy, but further numerica testing is required to gather more concusive evidence. 3. Simuation-Based Optimization In this section, we show how to improve the booking imits of any particuar BL poicy (for instance, EMSR-BL or LP-BL) by taking into account the stochastic and dynamic nature of the demand. We use a combination of simuationbased optimization (a stochastic gradient agorithm) and approximate dynamic programming ideas.

5 94 Operations Research 53(1), pp , 2005 INFORMS The idea of simuation-based optimization in a revenuemanagement context was introduced by Robinson (1995). For the singe-eg probem under the assumption that demand for different fare casses arrives sequentiay, he derives optimaity conditions for the booking imits. He then proposes to sove these using Monte Caro integration, based on simuation of the demand. Van Ryzin and McGi (2000) propose an adaptive method to sove the same set of optimaity conditions, but using historica booking data instead of simuated demand. The convergence of this method is shown using stochastic approximation theory. Karaesmen and van Ryzin (1998) deveop a numerica agorithm to determine joint overbooking eves for party substitutabe inventory casses. This takes the use of simuation one step further, because no optimaity conditions are known for this probem. The reason that simuation-based optimization has not been appied more generay is most ikey the enormous size of the probem. The combination with mathematica programming modes that we propose here significanty reduces the probem size by efficienty deaing with the combinatoria aspects of the probem, i.e., the nesting order. As Wiiamson (1992, p. 127) nicey puts it: Without first knowing the correct nesting hierarchy of different odf s over a network, a network optimization which expicity accounts for nesting of ODF s is, to date, both theoreticay infeasibe and impractica Probem Definition Mode (1) is a static optimization mode in the sense that it determines a booking poicy that is impemented unti fight departure. However, as the booking process proceeds, the airine finds out the actua reaization of demand. This information can be used to improve the booking poicy, for instance, by hoding additiona seats for profitabe booking casses with higher than average demand. Hence, in practice the booking poicy is revised severa times, most often in overnight runs of the optimization agorithm. It is important that these future revisions are taken into account when determining a booking poicy. For instance, such revisions aow an airine to protect more seats for business traveers in the eary stages of the booking period because it wi be abe to offer additiona discount seats ater on if necessary. We have added a numerica exampe in 4.3 that iustrates this point. We assume that the booking period is divided into T time windows, not necessariy of the same ength. Whenever the booking process enters a new time window t, the booking poicy is reoptimized. During each time window, the nesting order and bucket mapping are fixed and based on Mode (1). In what foows, et b t be the set of booking imits used in time window t, et C t be the remaining capacity vector at the beginning of time window t, and et q t b t C t be the corresponding revenue generated during time window t. Let Q t b t C t be the tota revenue generated during time windows t to T if the airine uses the booking imits b t in time window t and the optima (maximizing expected revenue) booking-imit poicy thereafter. Note that, given the demand mode and the virtua nesting scheme, this notion is we defined, because the number of aowabe sets of booking imits is finite (cf. 2.1). Finay, et R t C t be the tota revenue generated from time window t to T if the airine impements the optima booking-imit poicy during each of these, given the intermediate state C t of the booking process. By definition, q t, Q t, and R t are a random functions whose vaues depend on the reaization of the demand process. At the beginning of time window t, the airine needs to sove E R t C t max E Q t b t C t (2) b t max E q t b t C t + R t+1 C t+1 (3) b t for the optima set of booking imits, where the expectation is taken with respect to the demand process. Note that (3) expicity formuates the booking-imit optimization probem as a dynamic program. For any reaistic mode of demand, E Q t b t C t cannot be expressed in cosed form and needs to be evauated numericay. We propose to simuate the booking process to determine the revenue q t generated during time window t, which directy depends on b t. We then use an estimate of the vaue function E R t+1 C t+1 to account for the revenue that woud have been generated in the remainder of the booking period. As we sha see, we can then attempt to sove probem (2) by a stochastic gradient agorithm. Aternativey, the booking process coud be simuated over a remaining time windows to evauate Q t b t C t directy. However, then the booking imits woud have to be recacuated mutipe times during each simuation run, which for a arge number of runs and reaisticay sized probems woud be intractabe. In 3.2, we define a computationay efficient stochastic gradient agorithm that approximates a soution to probem (2). In 3.3, we deveop a recursive agorithm to estimate the vaue functions E R t C t that are needed for this. In 3.4, we combine these agorithms and propose a simuation-based method for booking-imit cacuation The Stochastic Gradient Agorithm In this section, we propose a stochastic gradient agorithm to approximate a soution to probem (2), given an approximation of E R t+1 C t+1. EMSR-BL or LP-BL provide an initia soution. The agorithm iterativey improves the set of booking imits, using numerica estimates of the first finite differences of E Q t b t C t. First, we briefy review the principes of stochastic gradient agorithms to prepare the ground for what foows. Then, we adapt the stochastic gradient agorithm to probem (2) A Generic Stochastic Gradient Agorithm. Let be the set of reaizations of a random process. Let F x be a function of some variabe x m, whose vaue depends on the random outcome. Possiby,

6 Operations Research 53(1), pp , 2005 INFORMS 95 the function F can ony be evauated using simuation. In the RM context, x is a vector of parameters that characterize a poicy (either booking imits or bid prices), F is the revenue as a function of these parameters, whie is the particuar reaization of the stochastic and dynamic demand. We are interested in maximizing E F x over x X, where X m denotes the set of feasibe soutions. The stochastic gradient agorithm is as foows. Generic Stochastic Gradient Agorithm Step 0. Pick a starting vaue x 0 X and et k = 0. Step 1. Let x k+1 = X x k + k k, where k R m is defined by n k F x k + i = k e i kj F x k kj i = 1 m j=1 k and X x denotes the projection of a vector x on the feasibe region X. Step 2. Let k = k + 1. Return to Step 1. Here, k is a finite difference approximation of the gradient E F x k based on n independent sampes kj of the random process, k is sma positive scaar, and k is the stepsize that typicay is decreasing in k. Note that this agorithm is we defined, even if the objective function E F x is not differentiabe. Under certain reguarity conditions for the function F, the sequences k and k, and the feasibe region X, this agorithm can be shown to converge to an optima soution with probabiity 1 (e.g., Ermoiev 1988, Gaivoronski 1988). However, as discussed in 3.2.3, these conditions are not met for our particuar appication A Stochastic Gradient Agorithm for Booking- Limit Improvement. We now adapt the stochastic gradient agorithm to probem (2). To faciitate notation, we have suppressed the dependence of Q t b t C t on the time window t and the remaining capacity C t in this subsection. By our definition of booking-imit contro, even if the integraity of booking imits is reaxed, E Q b is a function of the integra part of b ony. As a resut, the objective function in probem (2) has discontinuities at the integer vaues of b and is not differentiabe. In our impementation of the stochastic gradient agorithm, we therefore have to work with numerica estimates of the first finite differences i E Q b =E Q b+e i E Q b for 1 i N (4) where ei is the unit vector corresponding to bi. These can be interpreted as the expected revenue change when a booking imit is increased by exacty one seat. Note that 1 E Q b = 0, because we assume that b 1 = C. Even if this booking imit is increased, this does not affect which booking requests are accepted due to the capacity constraint. The vector of first finite differences E Q b of the expected revenue function is defined by E Q b = i E Q b =1 L i=1 N To estimate E Q b, we coud simpy randomy generate n sequences of booking requests, evauate Q b and Q b+ei for each of these for a buckets i, and substitute the averages into (4). Cf. Step 2 of the stochastic gradient agorithm. However, accurate estimation of the first finite differences requires a arge number of simuation runs and the evauation of nn booking-imit poicies may be time consuming. We propose a more efficient approach here, based on the insight that the increase of a singe booking imit does not necessariy affect which booking requests are accepted. Demand might simpy be too ow to be constrained by any booking imit, an inventory bucket might be cosed for further bookings because the booking imit of a higher bucket is reached, or accepting a booking request might vioate the booking imits of severa inventory buckets at the same time, in which case increasing just one of them makes no difference. Thus, in many cases Q b equas Q b+ei. This insight motivated the foowing agorithm to estimate E Q b. Finite Differences Estimation Agorithm Step 1. FOR j = 1TOn (1a) Simuate the booking process in time window t; that is, generate a sequence of time-ordered booking requests according to some demand mode, such as the one defined in 4. Starting with the eariest one, these requests are then worked through one at a time, either accepting or rejecting them as dictated by some prespecified poicy in this case the booking imits b and the capacity constraints. (1b) Let Q b j be the revenue estimate of using the booking imits b in time window t and some periodicay revised booking-imit poicy thereafter, based on the revenue generated in simuation j and the approximation of E R t+1 C t+1. (1c) Whenever a booking request is decined because the booking imit of exacty one inventory bucket, say Bi, woud be vioated, start keeping track of what woud have happened if this booking imit woud have been one seat higher given the same future sequence of booking requests. Note that the booking request in that case woud have been accepted. Do this at most once for each inventory bucket. Let the resuting revenue estimate be denoted by Q b + ei j. (1d) If Q b + ei j has been defined, et iq b j = Q b + ei j Q b j. Otherwise, et iq b j = 0. END FOR Step 2. Let ie Q b = 1/n n j=1 iq b j be the fina estimate of ie Q b and et E Q b denote the corresponding estimate of E Q b. Note that we ony generate n sequences of booking requests, because Step (1c) is simpy a matter of parae bookkeeping. Parae simuations ony branch off the main simuation; hence, their number remains imited. The point is that for each reaization of demand, we ony

7 96 Operations Research 53(1), pp , 2005 INFORMS evauate the booking-imit poicy b + e i if the resut woud differ from b. This can be significanty more efficient than evauating each of these N poicies consecutivey, for instance, given ow demand. The sampe size n needs to be tuned beforehand. Larger vaues give more accurate estimates at the cost of more computation time. Using these estimates of the first finite differences, the booking imits wi now be iterativey adjusted unti no further improvement seems possibe or the maximum number of iterations k max is reached. Any feasibe set of booking imits b has to satisfy the constraint 0 b i+1 b i b 1 = C for 2 i N 1 Experimenta evidence has shown that booking imits shoud ony be moderatey increased during a singe iteration of the stochastic gradient agorithm. For this reason, we have introduced an upper bound max on the change of any particuar booking imit in each iteration. To address these issues, we have modified the stochastic gradient agorithm as foows. Numerica Booking-Limit Improvement Step Step 0. Let k = 1; et b 1 be an initia set of booking imits. Let b 0 = b 1 and E Q b 0 = 0. Step 1. Estimate E Q b k. Step 2. FOR = 1TOL FOR i = 2TON IF ie Q b k > 0 AND ie Q b k 1 0 (ceary increase the booking imit) i = min int ie Q b k k max ELSE IF ie Q b k < 0 AND ie Q b k 1 0 (decrease the booking imit) i = max int ie Q b k k 0 5 max ELSE (next booking imit between current and previous soution) i = ie Q b k / ie Q b k + ie Q b k 1 (with 0 0) 0 i prev = bi k bi k 1 i = int i i prev 0 5 sign i prev ENDIF bi k+1 = bi k + i END FOR END FOR Step 3. Make new soution feasibe: FOR = 1TOL FOR i = 2TON bi k+1 = min bi k+1 bi 1 k+1 bi k+1 = max bi k+1 0 END FOR END FOR Step 4. IF i = 0 for a eg-buckets OR k = k max TERMINATE ELSE k = k + 1 GOTO Step 1 ENDIF The starting point b 1 can be determined by EMSR-BL or LP-BL, whichever works best. The stepsize functions k, max, and k max are optimization parameters that need to be tuned, in conjunction with the number of sampes n for the first differences estimation agorithm. For instance, our experience with this agorithm suggests that because arger vaues of n give more accurate estimations of the first finite differences, this aows a arger stepsize k.given that there are no theoretica performance guarantees for this agorithm, extensive tuning is especiay important Theoretica Considerations. Given the discrete nature of booking imits, the expected revenue function E Q t b t C t is nonsmooth and nondifferentiabe. Attempting to sove this type of optimization probem with a stochastic gradient method has no theoretica justification, but is a heuristic based on the intuition behind gradient descent methods. The agorithm based on first finite differences is we defined, but does not necessariy converge (Ermoiev 1988). For this reason, we have introduced a maximum on the number of iterations. Even if the agorithm woud converge, this woud not necessariy be at a oca optimum. To see this, consider a two-eg fight that ony carries through-passengers at two different rates. Assume that the booking imit for the discount rate is the same on both egs. Then, given the BL poicy of 2.1, increasing ony one of these booking imits wi not affect ticket saes. Thus, both first finite differences are zero and the random search wi terminate, whie increasing both booking imits simutaneousy might have increased expected revenue. However, this is an unreaistic exampe given the absence of oca traffic, and we do not expect probematic cases ike this to occur in practice. Because the expected revenue function may not be convex, the fina set of booking imits may strongy depend on the starting point. We have found that the EMSRb heuristic generay gave a better starting point than the LP soution, especiay for arge-scae exampes (see 4.4). Summarizing, the agorithm cannot be guaranteed to terminate at an optima set of booking imits for a given virtua nesting scheme, but intuition and practica experience gained in the experiments of the next section suggest that they can at east be improved significanty Estimation of the Vaue Function In this section, we propose an agorithm to estimate the vaue functions E R t C t, which are used as input for the Numerica Booking-Limit Improvement Step. Because the dimension of the capacity vector can be arge, we propose to evauate the vaue function ony on a sma number

8 Operations Research 53(1), pp , 2005 INFORMS 97 of carefuy seected discretization points. This set of data is then used to estimate the vaue function as a whoe. We first define an agorithm to determine the discretization points and then propose an interpoation method Seection of the Discretization Points. The domain of the vaue function is the Cartesian product of C C C L where L, as before, denotes the number of egs. Hence, each dimension of the state-space corresponds to a specific eg in the network. A natura way of seecting the discretization points is picking a sma number, say q, of capacity eves on each eg, and forming a compete L-dimensiona grid. However, the number of such grid points woud be exponentia in L, which woud be intractabe for arge networks. In addition, it is questionabe whether the discretization points shoud reay be homogeneousy distributed over the state-space. For instance, at the end of the first time window, remaining capacity is ikey to be reativey cose to the origina fight capacities, whereas at the beginning of the fina time window, fights are more ikey to be sod out. Hence, if we coud somehow find the most ikey ocation of the remaining capacity vector in the state-space at any given time, we coud concentrate the discretization points in this area. This woud enabe us to get more accurate estimates of the vaue function at paces where it matters, with reativey few discretization points. We propose the foowing agorithm for this purpose. Discretization Points Seection Agorithm Step 1. Simuate the booking process. Whenever a simuation run enters a new time window, use a heuristic such as EMSR-BL to determine a reasonabe booking poicy, and save the remaining capacity of each eg of the network. Step 2. Using the empirica distribution of C t obtained in Step 1, cacuate the mean C t and standard deviation C t of the remaining capacity of eg at the beginning of time window t. Step 3. For the approximation of the vaue function at the beginning of time window t, use the discretization points { C t i = C t 1 C t 1 C t i C t +1 C t L i = 1 q = 1 L } where i C t i = C t min + q + 1 C t max C t min for C t min =max C t Ct 0 C t max =min C t + Ct C1 i = 1 q = 1 L i=1 q =1 L i=1 q =1 L Here is a predetermined constant defining a confidence interva centered around C t, in which the discretization points C t i in dimension are chosen. Mutipes of discretization points may occur for sma but are removed from the set. For each time window t, the agorithm creates ql discretization points on the axes of an artificia coordinate system in the state-space with origin C t = C 1 t C L t, that covers the area where the remaining capacity vector is most ikey to be. Working within this artificia coordinate system faciitates the definition of the piecewise inear and separabe approximation of the vaue function that we have used in this research. The number of discretization points q and the size of the confidence interva can ony be chosen by tria and error, given the usua trade-off between accuracy and computation time. In the numerica exampes of 4 we have used = 3, as this (roughy) corresponds to a 99% confidence interva Interpoation of the Vaue Function. In this section, we propose a method to estimate the vaue function E R t C t by approximating its vaue on the set of discretization points. For simpicity, we have used a piecewise inear and separabe approximation in this research. This is somewhat simiar to the bid-price approach, which can be seen as a inear approximation of the vaue function. Our approach utiizes first- and second-order information, which shoud ead to more accurate estimates. First, we deveop an efficient agorithm to estimate the first finite differences of the vaue function, which is again not differentiabe. Then, we propose a inear interpoation method that is motivated by the concavity of the expected revenue as a function of remaining capacity on a singe eg (given Poisson demand in the absence of group bookings, e.g., Lee and Hersh 1993). The proposed approach is recursive, in the sense that the estimation of E R t C t requires that an estimate of E R t+1 C t+1 is aready avaiabe. We propose the foowing agorithm to estimate the vaue function E R t C t and its first finite differences E R t C t for any given capacity vector C t. Vaue Function and First Finite Differences Estimation Agorithm Step 0. Use EMSR-BL or LP-BL (whichever works best) in combination with the Numerica Booking-Limit Improvement Step to determine a good set of booking imits b t. Step 1. FOR j = 1TOn (1a) Simuate the booking process in time window t. Let R t C t j be the estimate of the future revenue given capacity C t at the start of time window t, based on the revenue generated in the simuation and the approximation of E R t+1 C t+1. (1b) For each eg, determine what the revenue woud have been if the capacity on this eg woud have been decreased by one seat, without changing the booking imits. Let the revenue generated in that case be denoted by R t C t j.

9 98 Operations Research 53(1), pp , 2005 INFORMS (1c) Let the jth estimate of the th first finite difference be R t C t j = R t C t j R t C t j. END FOR Step 2. Let E R t C t = 1/n n j=1 Rt C t j and E R t C t = 1/n n j=1 R t C t j be the fina estimates of E R t C t and its th first finite difference, respectivey. Note that the reduction of capacity on a eg might aso affect the optima set of booking imits, given the virtua nesting scheme. However, because reoptimization woud be too time consuming, we have decided not to take this effect into account. The agorithm requires the generation of ony n sequences of demand, because Step (1b) is again just a matter of parae bookkeeping. The number of sampes n is determined by tuning, the trade-off again being accuracy and computation time. We now define the interpoation method. To simpify notation, we have suppressed the dependence on the time window t. Let C 1 < C 2 < < C q be the discretization points in dimension, and et C i be the th coordinate of C i. Use the estimation method outined above to approximate the vaue function E R C i and its th first finite difference E R C i for each discretization point C i. Approximate E R C as we. Let C = C 1 C L and et C = C + C C e be the projection of C on dimension of the artificia coordinate system that contains the discretization points. Then, the vaue function at point C can be approximated by Vaue Function Interpoation Agorithm Step 1. For a egs = 1 L: (1a) If C C 1, et E R C E R C 1 E R C 1 C 1 C. (1b) If C C q, et E R C E R C q + E R C q C C q. (1c) If C i C < C i+1 1 i<q, et E R C i + E R C i C C i E R C ub min E R C i+1 E R C i+1 C i+1 C E R C b E R C i + C C i ( ) E R C i+1 E R C i C i+1 C i E R C we R C ub + 1 w E R C b for 0 w 1 Step 2. Let E R C = E R C + L =1 E R C E R C. Figure 1. E R C E R C Iustration of the vaue function interpoation agorithm. True vaue function Vaue function approximation Bounds on approximation C i C i+1 C The interpoation agorithm is based on determining how much of the difference between the vaue function at C and C can be attributed to the difference in remaining capacity at each of the individua fight egs. For this purpose, we have introduced the auxiiary capacity vectors C that are equa to C except on eg. When C is between two discretization points in dimension (Case (1c)), we determine ower and upper bounds on its function vaue based on the presumed concavity of the vaue function for a singe-eg fight. These bounds are iustrated in Figure 1. The upper bound consists of the ower enveope of the tangent ines at the discretization points, whie the ower bound is given by the convex combination of the vaue function at these two points. We then combine these bounds by taking a weighted average. When C fas outside the range of discretization points (Cases (1a) and (1b)), inear extrapoation is used. Based on this approximation, we determine the difference between the vaue function at C and C, which can be seen as the change of the vaue function aong dimension. The difference between the vaue function at C and C is then estimated by the sum of these changes. We are now ready to define an agorithm that estimates a vaue functions recursivey. Let E R T Then, E R t C t t = 1 T can be estimated as foows. Recursive Vaue Function Estimation Agorithm Step 1. Use the Discretization Points Seection Agorithm to seect discretization points for each time window t = 1 T. Step 2. FOR t = T TO 1 (2a) Use the Vaue Function and First Finite Differences Estimation Agorithm to approximate E R t C t, E R t C t i, and E R t C t i for each discretization point C t i. (2b) Use the Vaue Function Interpoation Agorithm to approximate E R t C t for arbitrary C t in the state-space. END FOR The estimation of the vaue function might be computationay expensive, but it can be done offine. After C t,

10 Operations Research 53(1), pp , 2005 INFORMS 99 the discretization points C t i, and the estimates of E R t C t, E R t C t i, and E R t C t i have been cacuated and stored, the approximation of E R t C t reduces to a singe ca of the Vaue Function Interpoation Agorithm. It is important, however, that these estimates are updated reguary during the booking process, even for a given set of fight departures. As the airine earns more about the actua reaization of the demand process, it shoud update its demand forecasts for the remaining part of the booking period accordingy. For instance, if the number of advance booking requests for a certain destination was surprisingy high, this may indicate that there is a specia event taking pace that wi ead the number of ast-minute booking requests to be higher than usua as we. Ideay, the vaue function estimates shoud depend on the number of booking requests received for each ODF, but in that case the dimension of the state-space woud be too arge. Reguar updates of the vaue function estimates may at east hep to capture part of this effect. The current approximation based on a stochastic demand mode that is fixed throughout the booking process is ony intended to adjust the booking poicy for the effect of the statistica fuctuations of demand on the remaining capacity, which we beieve to be an important factor in effective inventory contro The Simuation-Based Booking-Limits Approach We now can define our proposa to determine a bookingimit poicy for an arbitrary time window t. Simuation-Based Booking-Limit Poicy (SBL) Step 1. (offine, say weeky) Run the Recursive Vaue Function Estimation Agorithm. Step 2. (offine, say overnight) Run the Numerica Booking-Limit Improvement Step on top of EMSR-BL or LP-BL (whichever works best) to determine the booking imits b t. Use the Vaue Function Interpoation Agorithm to evauate E R t C t when necessary. Step 3. (onine) Impement the booking-imit (BL) poicy with the set of booking imits b t. In the numerica experiments of 4, we have compared both ways to determine a starting point for the Numerica Booking-Limit Improvement Step, which we refer to as SBL EMSR and SBL LP, respectivey. For Step 0 of the Vaue Function and First Finite Differences Estimation Agorithm, we have aways used EMSR-BL. Note that the same agorithms can be used with different nesting poicies, both standard and theft, which ony affects the impementation of the simuation program. For this reason, we have tested the performance of the SBL approach for both methods. We fee that the most important ideas underying the SBL method are: 1. The use of Mode (1) to determine the nesting order, hence soving the combinatoria aspect of the probem efficienty. Because nesting can be combinatoriay exposive, Mode (1) provides a feasibe, and we think reasonabe, approximation to the optima nesting order. 2. The combination of simuation and approximate dynamic programming to estimate the expected revenue at the end of each time window. This aows capturing the effect of poicy updates for the Numerica Booking-Limit Improvement Step. 3. The stochastic gradient agorithm to improve a given set of booking imits. The SBL approach is ceary a heuristic, because the stochastic gradient agorithm wi most ikey not have converged to an optima soution at its termination. Moreover, much of its performance depends on the tuning of the parameters of the subroutines it invokes. However, the agorithm has been designed in such a way that it shoud aways ead to an improvement of the initia set of booking imits, given the simuation mode and the accuracy of the vaue function approximation. The numerica evidence presented in the next section supports this. In contrast, most of the methods that have been proposed in the iterature have a better theoretica justification, but they either oversimpify the probem or are directed at optimizing a different poicy than the one that is actuay impemented. 4. Computationa Resuts In this section, we conduct computationa experiments to determine how much the SBL approach improves over EMSR-BL and LP-BL and what factors affect the reative performance of these poicies. We consider some arge-scae exampes to show that the SBL approach is tractabe for reaisticay sized probems. In addition, we briefy compare the SBL approach with the BP poicy for competeness The Simuation Environment Foowing Weatherford et a. (1993), for our computationa experiments we mode the arriva process of booking requests for cass odf as a nonhomogeneous Poisson process (NHPP) with arriva intensity odf t = odf t A odf (5) where odf t is the standardized beta distribution odf t = 1 ( ) t 1 ( 1 t ) 1 + in which is the ength of the booking period, and and are parameters defining the arriva pattern and A odf is a random variabe that obeys the gamma distribution. The properties of this so-caed Póya process are studied in the monograph by Grande (1997) on the more genera cass of mixed Poisson processes. Modeing the rate by the beta distribution odf t aows for a wide range of unimoda arriva patterns. The random variabe A odf, which has the

11 100 Operations Research 53(1), pp , 2005 INFORMS interpretation of the tota demand for booking cass odf, adds an extra eve of randomness to the NHPP. In addition, it introduces a positive correation between the number of bookings in separate parts of the booking horizon. As a resut, the current number of bookings provides information about demand to come, which shoud be taken into account for the poicy updates. Because the gamma distribution is the conjugate prior of the Poisson distribution, it is easy to show that the tota demand D odf generated by the Póya process has the negative binomia distribution. With a itte extra work, it can be shown that the conditiona distribution of remaining demand after each update is negative binomia as we. This suggests that the arriva process of booking requests after each update can again be modeed by a Póya process with arriva intensity (5), where the parameters of the distribution of A odf now depend on the number of booking requests received up to that point. It is understood that, for each poicy update, Mode (1) and the EMSR method are based on the conditiona demand distributions, and that the Numerica Booking-Limit Improvement Step 2 of the SBL approach is based on a conditiona mode of the arriva process of booking requests. In contrast, as discussed in 3.3.2, the estimate of the vaue function is based on the unconditiona prior demand mode (5). Empirica studies by Lye (1970) and de Boer (1999) suggest that a Póya process with arriva intensity (5) is a reasonabe high-eve approximation of airine demand. Note, however, that this mode does not incorporate important practica factors such as overbooking, canceations, and no-shows. In addition, the mode assumes that each customer requests a particuar fare cass and that no other fare cass, ower or higher, woud do. Athough airines do try to fence off ow fares from price-insensitive customers for instance, by requiring a Saturday night stay this is unreaistic. More ikey, customers are ooking for the owest avaiabe fare that meets their restrictions; thus, the RM poicy affects demand. In this case, the same contro framework (virtua nesting) can be used to determine the seat avaiabiity for each booking cass, but the booking-imit cacuation step woud have to be adjusted to take this into account. Because it is hard to mode such a demand process in cosed form, independent demand for each booking cass is a common assumption in the RM iterature. However, for the SBL method, this behavior can easiy be captured, because ony the simuation mode woud need to be adjusted. For instance, each simuated customer coud consecutivey request the avaiabiity of a fare casses that meet his restrictions, starting with the cheapest one, and woud buy the first avaiabe. We have deiberatey eft out these factors in our numerica anaysis to concentrate on the effects of incuding nesting in a probabiistic and dynamic mode of demand. These simpifications imit the generaity of our resuts, because we cannot be sure how the SBL method woud perform given a more reaistic mode of the booking process. However, as far as we know, no other method has been proposed to date to address such a sophisticated representation of the probem against which we coud benchmark it. We have impemented a computer program in C++ that simuates the booking process according to the mode we outined. We have impemented the poicies EMSR-BL, LP-BL, SBL EMSR, SBL LP, and BP, under both standard and theft nesting, and aowing at most 10 buckets per eg in the custering step. Uness stated otherwise, different poicies are tested on the same simuated sequence of booking requests to get a more accurate estimate of the revenue differentia Sensitivity Anaysis We want to identify the factors that affect the reative performance of the considered poicies. In particuar, we are interested in under what conditions the SBL approach eads to the most significant revenue gains. We consider a singeeg fight and a sma network. To isoate these factors from the impact of poicy updates and the vaue function estimation, in the exampes beow the booking poicy is cacuated ony once at the beginning of the booking period T = Exampe 1: Singe-Leg Fight. Our first exampe is a singe-eg fight with five fare casses. The factors we consider here are the nomina oad factor, the demand variabiity, and the fare structure. The nomina oad factor (LF) is defined as the tota expected demand for seats divided by the tota number of avaiabe seats. Demand variabiity is measured by the coefficient of variation (CV) of the demand for each booking cass. The fare structure is determined by the reative difference (RDF) between two consecutive fares for the same itinerary. For exampe, an RDF of 50% impies that the difference between the owest and the highest fare in the market for the same itinerary is roughy a factor of five. In addition, we have examined the effect of the arriva order of booking requests, which is determined by the parameters of the beta distribution. We distinguish between time-homogeneous (HOM) arrivas for each booking cass, and the case that the ower-fare casses tend to book eary in the booking period, whie the higherfare casses tend to book coser to departure time (LBH). In Tabes 1 and 2 we report the resuts of 100,000 simuation runs using standard nesting. Here as we as in a other numerica exampes, the reported improvement of the SBL method over aternative booking-imit heuristics is using the atter as the starting point for the random search. For competeness, we aso report the improvement of EMSR-BL over LP-BL. The most reevant observations from these exampes are: The gain of SBL over LP-BL increases with demand variabiity and varies significanty with the fare structure, whie the gain of SBL over EMSR-BL is ess sensitive to these factors. The reason is that the EMSR method takes into account both demand uncertainty and the fare eves, whie the LP-based method does not.

12 Operations Research 53(1), pp , 2005 INFORMS 101 Tabe 1. Sensitivity gain SBL over EMSR and LP for RDF and CV (standard nesting). SBL EMSR EMSR SBL LP LP EMSR LP RDF \ CV 25% 50% 75% 25% 50% 75% 25% 50% 75% 25% 0 02% 0 05% 0 13% 0 67% 1 62% 2 94% 0 66% 1 56% 2 81% 50% 0 05% 0 11% 0 30% 0 12% 0 73% 1 20% 0 30% 0 61% 0 88% 75% 0 06% 0 11% 0 30% 0 43% 1 11% 1 60% 0 56% 0 99% 1 29% Tabe 2. Sensitivity gain SBL over EMSR and LP for LF and arriva process (standard nesting). SBL EMSR EMSR SBL EMSR LP EMSR LP LF \ Arrivas LBH HOM LBH HOM LBH HOM 75% 0 01% 0 06% 0 09% 0 12% 0 08% 0 04% 100% 0 08% 0 48% 0 92% 0 90% 0 85% 0 42% 125% 0 11% 0 68% 0 69% 1 14% 0 59% 0 47% The gain of SBL over EMSR-BL and LP-BL generay increases with the oad factor, refecting that a better RM method is more important given heavy demand. The gain of SBL over EMSR-BL strongy depends on the arriva process, whie the gain of SBL over LP- BL is ess sensitive to this factor. The reason is that the EMSR method impicity assumes that the owest casses book first, thus the protection eves are off when this is not the case. In most cases, the gain of EMSR-BL over LP-BL and the gain of SBL over EMSR-BL add up to roughy the gain of SBL over LP-BL, which suggests that the expected revenue of the SBL approach is reativey insensitive to the starting point of the random search. Note that the resuting poicies are not necessariy the same, which may ead to different combinations of oad factor and yied. To investigate the effect of the nesting poicy on the reative performance of the SBL approach, in Tabe 3 we report the resut of 100,000 simuation runs using theft nesting. For competeness, we aso compare the expected revenue of the SBL approach (EMSR starting point) under standard and theft nesting. The most reevant observations from this exampe are: The gains of the SBL approach over EMSR-BL and LP-BL are much more significant under theft nesting than they were under standard nesting. Unike the SBL approach, these methods do not take the operationa nesting poicy into account, whie this ceary shoud affect the booking imits. If the SBL approach is used to set the booking imits under both standard and theft nesting, then theft nesting performs better, and more so when the demand variance increases. Figure 2 shows the estimated objective vaues of the sequence of booking imits produced by the SBL agorithm with different starting points (EMSR-BL and LP-BL) for the case that CV = RDF = 50%, under both standard and theft nesting. The expected revenue estimates are again based on 100,000 simuation runs. In a cases, the stopping criterion of the Numerica Booking-Limit Improvement Step was met before the maximum number of iterations was reached. Note that the improvement obtained by the SBL Agorithm is not monotone, perhaps because of the discrete and Figure 2. Expected revenue 27,400 27,200 27,000 26,800 26,600 26,400 26,200 26,000 25,800 Convergence of the SBL Agorithm for Exampe 1 with CV = RDF = 50%. SBL stopping criterion met SBL_EMSR, standard nesting SBL_LP, standard nesting SBL_EMSR, theft nesting SBL_LP, theft nesting 25, Iteration Tabe 3. Sensitivity gain SBL over EMSR and LP for RDF and CV (theft nesting). SBL EMSR EMSR SBL LP LP SBL EMSR Standard Theft RDF \ CV 25% 50% 75% 25% 50% 75% 25% 50% 75% 25% 7 93% 3 20% 1 80% 11 79% 8 72% 8 00% 0 15% 0 24% 0 78% 50% 7 56% 3 78% 2 34% 8 80% 5 45% 4 03% 0 35% 0 40% 0 65% 75% 6 25% 3 36% 2 45% 7 09% 3 93% 3 06% 0 13% 0 45% 0 89%

13 102 Operations Research 53(1), pp , 2005 INFORMS Tabe 4. Sensitivity gain SBL over EMSR and LP for LFR and FLD (standard nesting). SBL EMSR EMSR SBL LP LP EMSR LP FLD \LFR 50% 62.5% 75% 50% 62.5% 75% 50% 62.5% 75% 25% 0 02% 0 50% 1 03% 0 79% 0 99% 1 17% 0 77% 0 49% 0 20% 50% 0 49% 0 70% 0 36% 0 76% 0 81% 1 09% 0 26% 0 14% 0 71% 75% 0 03% 0 07% 0 07% 0 83% 0 67% 0 66% 0 58% 0 60% 0 57% stochastic nature of the optimization probem, and that most progress is made in the first coupe of iterations Exampe 2: Singe-Hub Network. We continue our sensitivity anaysis with an exampe of a singe-hub network connecting five cities to investigate factors that may affect the performance of the considered poicies in a network setting. We ony consider one bank of inbound and one bank of outbound fights. The airine offers a 30 possibe itineraries, both oca and connecting, at a fu-fare and a discount rate. The factors that we have considered are the fraction oca demand on each eg (FLD) and the average ratio between oca and connecting fares (LFR). For exampe, an LFR of 50% indicates that on average traveing to or from the hub costs haf as much as a connecting fight from spoke to spoke. The resuts of 100,000 simuation runs for an LBH arriva process using standard nesting are reported in Tabe 4. The most important observation from this exampe is that when oca demand is reativey ow, the gain of SBL over EMSR-BL increases with the vaue of oca traffic, whie when oca demand makes up a arger fraction of tota demand, the effect is uncear. The gain of SBL over LP-BL is ess sensitive to these factors. A possibe expanation may be that EMSR-BL, by soving the probem for each eg separatey, may have a tendency to overprotect oca traffic. Note that a connecting passenger shoud ony be turned down when he is expected to dispace a oca passenger on both of his fight egs. However, egbased methods based on dispacement adjusted or prorated fares set protection eves to avoid dispacement on a singe eg, which has a higher probabiity of occurring. This tendency might be stronger when oca passengers are reativey more vauabe, but this woud matter ess the arger their share of tota traffic. Finay, note that the performance of SBL is generay independent of the starting point of the random search, because again in most cases the gain of EMSR-BL over LP-BL and the gain of SBL over EMSR- BL roughy add up to the gain of SBL over LP-BL Comparison with the Optima Poicy In our next exampe, we compare SBL, EMSR-BL, and LP-BL with the optima booking poicy determined by stochastic dynamic programming. To iustrate the importance of the vaue functions for the performance of the SBL approach, we test an aternative impementation of the Finite Differences Estimation Agorithm that simuates the booking process unti the end of the booking period, not accounting for future poicy updates (SBL NVV ). The comparisons between EMSR-BL and LP-BL and between standard nesting and theft nesting have been eft out deiberatey, because our primary interest is the reative performance of the SBL approach. Consider a singe-eg fight with capacity of 100 seats. The airine offers six different booking casses, with fares ranging from $100 to $800. Booking requests arrive according to a nonhomogeneous Poisson process of the LBH type. The gamma component of the arriva intensity in (5) is suppressed to aow modeing the probem as a onedimensiona Markov Chain A odf = E D odf. Average demand exceeds capacity by 30%. The resuts of 2,500 simuations runs using standard nesting for different numbers of poicy updates are reported in Tabe 5. The reported bounds on the optimaity gap are based on the SDP soution (optima expected revenue of $26,594.5) and a 95% confidence interva for the true expected revenue of the SBL approach based on the sampe average and its standard deviation. The most reevant observations from this tabe are that: The optimaity gap of the SBL method decreases with the number of poicy updates unti, with 95% confidence, it is ess than 0.17%. The gain of SBL over EMSR-BL and LP-BL increases with the number of poicy updates, and when SBL is impemented without accounting for the poicy updates, the revenue gains are generay ower. This suggests that it is important to take future updates into account when deter- Tabe 5. Comparison with optima booking contro (standard nesting). Number of updates SBL EMSR EMSR 0 11% 0 23% 0 37% 0 48% SBL LP LP 0 79% 0 89% 1 07% 1 18% SBL NVV EMSR EMSR 0 11% 0 16% 0 16% 0 27% SBL NVV LP LP 0 79% 0 93% 0 86% 1 13% opt. gap SBL EMSR 0.77% 1.55% 0.11% 0.90% <0 41% <0 17%

14 Operations Research 53(1), pp , 2005 INFORMS 103 Tabe 6. Accuracy of the vaue function estimates (standard nesting). Number of updates E R 1 C SBL EMSR revenue (53.3) (53.8) (53.3) (50.9) Tabe 7. Comparison with optima booking contro (theft nesting). Number of updates SBL EMSR EMSR 4 92% 0 91% 0 40% 0 20% SBL LP LP 4 97% 1 00% 0 68% 0 89% SBL NVV EMSR EMSR 4 92% 0 86% 0 24% 0 11% SBL NVV LP LP 4 97% 0 96% 0 48% 0 50% opt. gap SBL EMSR 0.79% 1.57% <0 79% <0 33% <0 09% mining the current booking poicy, which the atter two methods do not. To iustrate the performance of the Vaue Function Estimation Agorithm, in Tabe 6 we report both the average revenue generated by the SBL approach (standard deviation of the sampe average given between parenthesis) and the estimate of the vaue function at the beginning of the booking period. Note that the vaue function estimates are reativey accurate. In Tabe 7, we report the resuts for the same 2,500 reaizations of the demand process under theft nesting. The most reevant observations are that: The gain of SBL over EMSR-BL and LP-BL now decreases with the number of poicy updates. A possibe expanation is that theft nesting is more effective when the booking imits are updated more frequenty; thus, there is ess room for the SBL approach to improve. When SBL is impemented without accounting for the poicy updates, the revenue gains are ower and can even be negative. We have seen this in many other simuation experiments that we have not reported here as we, which shows that the vaue function estimation step is in fact essentia for the success of the SBL method. The revenue gains of the SBL approach are more significant under theft nesting than under standard nesting when the number of poicy updates is reativey sma, for reasons aready expained above Larger Networks We consider networks of 5, 10, and 15 cities connected by a singe hub. The airine offers a possibe itineraries, both oca and connecting, at five different rates. We consider the case of demand foowing a pure nonhomogeneous Poisson process (A odf = E D odf, with CV = 1/E D odf, generay ess than 30%), and the case of the arriva intensity itsef being a gamma-distributed random variabe (a Póya process, with CV = 35%), under both standard and theft nesting. The other simuation settings are RDF = 50%, LF = 125%, LFR = 75%, FLD = 25%, and an LBH arriva process. The booking poicy is updated 20 times during the booking period. The resuts of 1,000 simuation runs are reported in Tabe 8. Again, the comparison between standard and theft nesting has been eft out deiberatey. We appied sower SBL optimization settings for the 5-spokes network with demand modeed by a Póya process than for the other test cases (see 4.4.1), which may be part of the reason why the reative performance of the SBL approach is generay better in this case. Other important observations are: The gains of the SBL approach are practicay significant. Given the operating scae and cost structure of airines, seemingy sma revenue improvements might transate to miions of doars each year, added straight to the bottom ine. The gain of the SBL approach is arger under theft nesting than under standard nesting. In a cases, the combined gain of EMSR-BL over LP- BL and of SBL over EMSR-BL is much higher than the gain of SBL over LP-BL, which shows that the EMSR booking imits provide a better starting point for the SBL approach Computationa Tractabiity. When the scae of the probem aows it, we can use sower optimization Tabe 8. Large-scae simuation resuts (revenue performance). SBL EMSR EMSR SBL LP LP EMSR LP Test case \ # spokes Poisson, standard 0 16% 0 09% 0 08% 0 84% 0 24% 0 24% 1 67% 2 07% 2 17% Poisson, theft 0 18% 0 47% 0 61% 1 23% 0 60% 0 61% 1 50% 1 25% 1 08% Póya, standard 0 07% 0 05% 0 05% 1 06% 0 26% 0 25% 1 64% 1 84% 1 85% Póya, theft 0 32% 0 38% 0 48% 1 29% 0 84% 0 68% 1 07% 1 35% 1 01%

15 104 Operations Research 53(1), pp , 2005 INFORMS Tabe 9. Large-scae simuation resuts (computationa performance). VV-EST (hrs) SIM SBL EMSR (sec) SIM SBL LP (sec) Test case \ # spokes Poisson, standard Poisson, theft Póya, standard Póya, theft settings for the SBL approach, which may ead to a better soution. The number of iterations (n) for the Finite Differences Estimation Agorithm was 1,000 for the 5-spokes network with demand modeed by a Póya process, using a fixed stepsize k = 0 5, but ony 100 for the other test cases, with stepsize k = In a cases, we used max = 2 and k max = 10. The number of iterations for the Vaue Function and First Differences Estimation Agorithm was 10,000 for the 5- and 10-spokes networks, but ony 5,000 for the 15-spokes network. In a cases, we used q = 4 grid points in each dimension of the state-space, whie for the Vaue Function Interpoation Agorithm we used w = 0 5 without much tuning. The vaue function estimate based on standard nesting was used to impement the SBL approach under theft nesting as we, to save computation time. The run time of the vaue function estimation (VV-EST, in CPU) and of the booking-imit improvement step (approximated by the average simuation time per update of the SBL poicy (SIM, in CPU)) for these arge-scae exampes on a Pentium IV processor are reported in Tabe 9. The important observations are that: The estimation of the vaue function for the 15-spokes network took about 50 to 60 hours of CPU time, depending on the demand process. Tuning of the optimization parameters and a more efficient impementation of the program can speed up the agorithm, but the important point is that this part of the agorithm can be done offine. The onine part of the SBL approach takes ony seconds for a instances, which shows that the proposed SBL approach is computationay tractabe for reaisticay sized probems Comparison with Bid-Price Contro We now compare the SBL approach with the BP poicy defined in 2.2. Ceary, the BP poicy is easier to com- pute, but as we have pointed out, its success may strongy depend on the frequency of poicy updates. For this reason, we compare the revenue performance of the SBL approach (EMSR starting point, standard nesting) with 20 updates to the BP poicy updated 20 to 100 times. The resuts of 1,000 simuation runs for the same test cases as in 4.4, but with different sequences of booking requests, are reported in Tabe 10. Even given 100 updates of the BP poicy, the SBL approach with ony 20 updates performed best. As we have pointed out, a better impementation of the BP poicy may be based on dynamic bid prices, but a more thorough comparison of booking-imit and bid-price contro is outside the scope of this research Insights Gained The most important insights gained from the numerica experiments in this section are: The SBL approach can ead to practicay significant revenue improvements over both EMSR-BL and LP-BL. The SBL approach takes into account factors that shoud affect the booking-imit poicy, but that other methods ignore, such as demand uncertainty and the fare structure (compared to LP-BL), the dynamics of the demand process, the ikeihood of a connecting passenger dispacing two oca traveers, future poicy updates, and the operationa nesting poicy. Because both EMSR-BL and LP-BL impicity assume standard nesting, the improvement of the SBL approach over these methods is particuary significant under theft nesting. The SBL approach is computationay tractabe for reaisticay sized probems. The use of vaue functions to account for future poicy updates is essentia for the success of the SBL approach. EMSR-BL provides a better starting point for the SBL random search than LP-BL. Tabe 10. Gain SBL approach with 20 updates over the BP poicy (SBL EMSR BP). 5 spokes 10 spokes 15 spokes # updates BP \ Demand process Póya Poisson Póya Poisson Póya Poisson % 1 89% 2 14% 2 67% 2 00% 2 60% % 1 27% 1 74% 2 22% 1 34% 2 01% % 1 21% 1 29% 2 22% 1 15% 1 76% % 1 17% 1 29% 1 97% 1 09% 1 71% % 1 06% 1 33% 2 06% 1 03% 1 59%

16 Operations Research 53(1), pp , 2005 INFORMS Concusions In this paper, we proposed a framework to address the stochastic and dynamic character of the demand and the nested character of booking-imit contro in a network environment. Starting with any nested booking-imit poicy, we combine a stochastic gradient agorithm and approximate dynamic programming ideas to improve the initia booking imits. Preiminary simuation experiments suggest that this approach (a) is computationay feasibe for reaisticay sized networks, because the computationay demanding part of the agorithm can be done offine, and (b) has the potentia of eading to practicay significant revenue enhancements over poicies based on inaccurate representations of booking-imit contro in a network. The simpified demand mode and the reativey sma number of test probems imit the extent of our concusions, but the potentia revenue gains warrant more extensive testing. Appendix Custering Agorithm On each eg, recursivey custer a booking casses with nonnegative adjusted eg revenue into at most N max (to be chosen beforehand) eg buckets as foows. Let k = 0be the number of buckets Bi 1 i k aready defined, and consider the range of adjusted eg revenues of a booking casses that remain to be custered into at most N max k additiona buckets. This range starts at 0 and ends at { } k UB k+1 = max f odf 0 odf S \ B i Divide this range into N max k equa parts and custer a booking casses whose adjusted eg revenue fas into the ast subinterva into bucket Bk+1 {, thus B k+1 = odf S N } max k 1 N max k UB k+1 f odf UB k+1 which by definition contains the booking cass with the highest adjusted eg revenue that had not yet been custered. Let k k + 1 and terminate the recursion when k = N max, or when no more booking casses with nonnegative adjusted eg revenue remain to be custered. Finay, et N = k + 1 be the tota number of buckets on eg after custering a booking casses with negative adjusted eg revenue into the owest bucket B N = odf S f odf < 0 if this set is not empty. Otherwise, N = k. This basic agorithm, which may be different from what airines use in practice, guarantees that there are no empty buckets and reduces the number of buckets on a eg if the adjusted eg revenues are cosey grouped together. LP-Based Booking-Limit Cacuation Method (LP-BL, Wiiamson 1992) Let b1 = C. Let pi be the number of seats that needs to be protected (protection eve) oneg for buckets 1 to i from bookings for bucket i + 1 or ower, which based on the LP i=1 soution is i p i = x odf j=1 odf Bj i = 1 N 1 Then, it makes intuitive sense to et b i+1 = C p i i = 1 N 1 (6) Note that bi 0 for a i because of the capacity constraints of Mode (1). EMSRb Booking-Limit Cacuation Method (EMSR-BL, Beobaba 1992) Let Di be the aggregate demand for a booking casses in bucket i on eg, and et ri be a weighted average of their adjusted eg revenues; thus, D i = D odf odf B i r i = 1 E D i odf B i E D odf f odf Let D1 i be the aggregate demand for buckets 1 to i; thus, i i D 1 i = D j = D odf j=1 j=1 odf Bj We approximate the distribution of D1 i with the Gaussian distribution with i E D 1 i = E D odf j=1 odf Bj Var D 1 i = i j=1 odf Bj Var D odf The average weighted eg revenue r1 i for these buckets is r 1 i = 1 i E D E D1 i j r j j=1 We can now define the booking imits. As before, et b 1 = C.Fori = 2 N 2, the number of seats that needs to be protected for buckets 1 to i from bookings from bucket i + 1 (and thus from a ower buckets) is p i = max p r 1 i P D 1 i p >r i+1 p integer and the booking imit corresponding to bucket i+1 isagain given by (6). Finay, set bn = 0, because we do not want to accept bookings that seem to have a negative contribution to eg revenue. Acknowedgments The authors thank Garrett van Ryzin and the three reviewers of this paper for numerous comments and suggestions that improved it substantiay. This research was partiay supported by the MIT-Singapore Aiance.

17 106 Operations Research 53(1), pp , 2005 INFORMS References Beobaba, P. P Air trave demand and airine seat inventory management. Ph.D. thesis, Fight Transportation Laboratory, Massachusetts Institute of Technoogy, Cambridge, MA. Beobaba, P. P Appication of a probabiistic decision mode to airine seat inventory contro. Oper. Res Beobaba, P. P Optima vs. heuristic methods for nested seat aocation. Proc. AGIFORS Reservations and Yied Management Study Group. Brusses, Begium. Beobaba, P. P., L. R. Weatherford Comparing decision rues that incorporate customer diversion in perishabe asset revenue management situations. Decision Sci Bertsekas, D. P Dynamic Programming and Optima Contro. Athena Scientific, Bemont, MA. Bertsimas, D., I. Popescu Revenue management in a dynamic network environment. Working paper, Soan Schoo of Management, Massachusetts Institute of Technoogy, Cambridge, MA. Bratu, S Network vaue concept in airine revenue management. Master s thesis, Operations Research Center, Massachusetts Institute of Technoogy, Cambridge, MA. Brumee, S. L., J. I. McGi Airine seat aocation with mutipe nested fare casses. Oper. Res Brumee, S. L., J. I. McGi, T. H. Oum, K. Sawaki, M. W. Tretheway Aocation of airine seats between stochasticay dependent demands. Transportation Sci Chatwin, R. E Muti-period airine overbooking with mutipe fare casses. Nava Res. Logist Chatwin, R. E Muti-period airine overbooking with a singe fare cass. Oper. Res Chen, V. C. P., D. Günther, E. L. Johnson A Markov-decision probem based approach to the airine YM probem. Working paper, The Logistics Institute, Georgia Institute of Technoogy, Atanta, GA. Chen, V. C. P., D. Günther, E. L. Johnson Airine yied management: Optima bid prices for singe-hub probems without canceations. Working paper, The Logistics Institute, Georgia Institute of Technoogy, Atanta, GA. Cross, R. G Revenue Management. Hardcore Tactics for Market Domination. Broadway Books, New York. Curry, R. E Optima airine seat aocation with fare casses nested by origins and destinations. Transportation Sci de Boer, S. V Mathematica programming in airine seat inventory contro. Master s thesis, Department of Mathematics, Vrije Universiteit, Amsterdam, The Netherands. de Boer, S. V., R. Freing, N. Piersma Mathematica programming for network revenue management revisited. Eur. J. Oper. Res Ermoiev, Yu Stochastic quasigradient methods. Yu. Ermoiev, R. J-B. Wets, eds. Numerica Techniques for Stochastic Optimization. Springer-Verag, New York, Gaivoronski, A Stochastic quasigradient methods and their impementation. Yu. Ermoiev, R. J-B. Wets, eds. Numerica Techniques for Stochastic Optimization. Springer-Verag, New York, Gaego, G., G. van Ryzin A muti-product dynamic pricing probem and its appications to network yied management. Oper. Res Gover, F., R. Gover, J. Lorenzo, C. McMian The passenger mix probem in the schedued airines. Interfaces Grande, J Mixed Poisson Processes. Chapman & Ha, London, U.K. Karaesmen, I., G. J. van Ryzin Overbooking with substitutabe inventory casses. Working paper, Graduate Schoo of Business, Coumbia University, New York. Lautenbacher, C. J., S. J. Stidham The underying Markov decision process in the singe-eg airine yied management probem. Transportation Sci Lee, T. C., M. Hersh A mode for dynamic airine seat inventory contro with mutipe seat bookings. Transportation Sci Littewood, K Forecasting and contro of passenger bookings. AGIFORS Sympos. Proc., Vo. 12. Nathanya, Israe. Lye, C A statistica anaysis of the variabiity in aircraft occupancy. AGIFORS Sympos. Proc., Vo. 10. Terriga, Austraia. McGi, J. I., G. J. van Ryzin Revenue management: Research overview and prospects. Transportation Sci Pagé, J. F Experience with O/D management at Air Canada. Presented to AGIFORS Reservations and Yied Management Study Group, London, U.K. Robinson, L. W Optima and approximate contro poicies for airine booking with sequentia nonmonotonic fare casses. Oper. Res Saranathan, K., K. Peters, M. Towns Revenue management at United Airines. Presented to AGIFORS Reservations and Yied Management Study Group, London, U.K. Simpson, R. W Using network fow techniques to find shadow prices for market and seat inventory contro. Memorandum M89-1, Fight Transportation Laboratory, Massachusetts Institute of Technoogy, Cambridge, MA. Smith, B. C., C. W. Penn Anaysis of aternative origin-destination contro strategies. AGIFORS Sympos. Proc., Vo. 28. New Seabury, MA. Subramanian, J., C. J. Lautenbacher, S. J. Stidham Yied management with overbooking, canceations and no shows. Transportation Sci Tauri, K. T., G. J. van Ryzin An anaysis of bid-price contros for network revenue management. Management Sci Tauri, K. T., G. J. van Ryzin A randomized inear programming method for computing network bid prices. Transportation Sci van Ryzin, G. J What is nesting on a network. Presented to AGI- FORS Reservations and Yied Management Study Group, Mebourne, Austraia. van Ryzin, G. J., J. I. McGi Revenue management without forecasting or optimization: An adaptive agorithm for determining airine seat protection eves. Management Sci Weatherford, L. R A tutoria on optimization in the context of perishabe-asset revenue management for the airine industry. G. Yu, ed. Operations Research in the Airine Industry. Kuwer Academic Pubishers, Amsterdam, The Netherands, Weatherford, L. R., S. E. Bodiy, P. E. Pfeifer Modeing the customer arriva process and comparing decision rues in perishabe asset revenue management situations. Transportation Sci Wiiamson, E. L Airine network seat inventory contro: Methodoogies and revenue impacts. Ph.D. thesis, Fight Transportation Laboratory, Massachusetts Institute of Technoogy, Cambridge, MA. Womer, R. D A hub-spoke seat management mode. Unpubished company report, Dougas Aircraft Company, McDonne Dougas Corporation, Long Beach, CA. Womer, R. D An airine seat management mode for a singe eg route when ower fare casses book first. Oper. Res