Optimal operations of transportation fleet for unloading activities at container ports

Transcription

1 Available online at Transportation Research Part B 42 (2008) Optimal operations of transportation fleet for unloading activities at container ports Seungmo Kang, Juan C. Medina, Yanfeng Ouyang * Department of Civil and Environmental Engineering, University of Illinois at Urbana-Champaign, Urbana, IL 61801, USA Received 3 March 2007; received in revised form 9 February 2008; accepted 11 February 2008 Abstract This paper presents mathematical models that optimize the size of transportation fleet (cranes and trucks) for unloading operations at container terminals. A cyclic queue model is used to study the steady-state port throughput, which then yields the optimum fleet size for long-term operations. This model allows for stochastic operations such as exponentially distributed crane service times. In order to allow for generally distributed crane service times and truck travel times, an approach based on Markovian decision process is also proposed. This model provides dynamic operational policies for fleet management. Both models are implemented and examined with empirical data from the Port of Balboa, Panama. These models are also extended to unloading operations that involve multiple berths. Published by Elsevier Ltd. Keywords: Container terminal; Unloading operation; Cyclic queue model; Markovian decision process; Fleet size optimization 1. Introduction The transportation of container cargo has become highly standardized in the intermodal shipping industry. The number of container units handled at maritime ports was estimated to have increased 55% between 1998 and 2005, to a total of 270 million 20-foot equivalent units in 2005 (The World Bank Group, 2006). Although container terminals have increased their capacity to process a greater number of containers per year, the rapid growth in container cargo volume poses a constant need for optimal use of port resources that reduces operating costs and increases cargo throughput. The handling process at container ports is quite sophisticated. In general, each container may go through one or more of the following four phases: loading, unloading, storage, and transfer to other ships or land based transportation services. In the unloading phase, containers are unloaded and transported from the vessel to the storage yard. The equipment involved generally includes quayside cranes (QC) at berths, rubber tire gantry cranes (RTGC) at storage yards, and terminal trucks (or tractors). The QCs are in charge of lifting and * Corresponding author. Tel.: ; fax: addresses: [email protected] (S. Kang), [email protected] (J.C. Medina), [email protected] (Y. Ouyang) /$ - see front matter Published by Elsevier Ltd. doi: /j.trb

2 S. Kang et al. / Transportation Research Part B 42 (2008) moving containers from the vessels to the trucks. A number of trucks travel in a dedicated closed loop to pick up containers at the berth and drop them at the storage yard. Each truck usually handles one container at a time. Once the container arrives at the yard, a RTGC lifts it and stores it inside the yard premises. Delays can occur if trucks are queued at the berth and/or the yard, depending on the number of available cranes and the arrival rate of the trucks. The unloading operation itself can be further decomposed into multiple stages, each of which can be studied separately. For example, Shields (1984) studied the most efficient order for a QC to pick up containers from a vessel, and Narasimhan and Palekar (2002) studied the transtainer routing problem, which seeks the optimal sorting and stacking of containers at storage yards that minimizes operating time. Goodchild and Daganzo (2006) studied a double cycling strategy for more efficient crane operations at the berth. In double cycling, containers are loaded and unloaded in the same crane cycle, potentially increasing the productivity and reducing the operating time per vessel. Furthermore, Goodchild and Daganzo (2007) explored the long-term benefits of a particular proximal stack strategy for unloading and loading containers from and to the vessels. Operating times were reported to decrease by up to 10% while using double cycling. The literature also presents a series of optimization tools for analyzing container loading and unloading operations, mostly focusing on the throughput of a terminal in the steady state. Specific techniques include queuing theory (Koenigsberg and Lam, 1976), network formulations (Steenken et al., 1993; Chen et al., 1998; Vis et al., 2001), and heuristics (Kim and Bae, 1999; Bish et al., 2001). Cyclic queue models have been applied to similar applications since the 1960s. Gordon and Newell (1967) derived general closed-form expressions for the expected distribution of customers in a multiple-stages cyclic queue system. Building on this, Koenigsberg and Lam (1976) described the problem of vessels loading and unloading liquid natural gas between two ports. More recently, Garrido and Allendes (2002) applied a simplified cyclic queue model to a case study on the Port of San Antonio, Chile. All crane service times and truck travel times are assumed to be exponentially distributed. This paper first focuses on the planning problem of finding the optimal size of fleet (i.e., the number of QC, RTGC, and trucks) for unloading operations that suffice to unload containers with the minimum operating cost. 1 Results from a cyclic queue model (Gordon and Newell, 1967; Koenigsberg and Lam, 1976) are used to analyze steady-state performance of the unloading system, which are then used as a subroutine to find the optimum fleet size. The cyclic queue model assumes exponentially distributed service times and steadystate operations. It provides a simple and closed-form estimate of long-run system performance in a highly stochastic operating environment, without requiring detailed information on specific container vessels. At the operational level, the stochastic nature of unloading operations mandates that the fleet configuration reacts to changing conditions in a timely manner. In this light, a Markovian decision process (MDP) model is presented as a complement of the cyclic queue model to find optimal policies for fleet management in real time. The MDP model allows for generally distributed service times and non-steady-state operations. Both models are first formulated for marine terminals with a single berth. Then the models can be used as building blocks for fleet management at terminals with multiple berths, where resources may be shared among the unloading operations of several ships. The exposition of the paper is as follows: Section 2 describes the mathematical models developed for the unloading fleet operations at container terminals. Section 3 illustrates the results of a case study with empirical data from the Port of Balboa in the Panama Canal. Section 4 discusses extended models for fleet management among multiple berths. Finally, Section 5 provides conclusions. 2. Model formulation In this section, we develop mathematical models that can be utilized for marine terminal fleet planning and operations. Section 2.1 presents a cyclic queue model to determine the steady-state performance of any terminal fleet; that performance is used to determine the optimal fleet size that minimizes cost yet satisfies 1 The reverse operation of loading containers from the storage yard to the vessel can be considered as equivalent to the unloading operation as both require the movement of the same type of cargo.

3 972 S. Kang et al. / Transportation Research Part B 42 (2008) throughput requirement. Then we develop in Section 2.2 a Markovian decision process framework for realtime operations Steady-state throughput analysis The process of unloading vessels at container terminals can be represented by a closed loop system similar to the models in Gordon and Newell (1967) and Koenigsberg and Lam (1976). The unloading process is first decomposed into four stages, as in Fig. 1: (i) the movement of an empty truck to the berth, after unloading operations at the storage yard; (ii) the process of loading a truck at the berth, which includes the truck s waiting in queue (if any) at the berth; (iii) the movement of a loaded truck from the berth to the storage yard; and (iv) the process of unloading a truck at the storage yard, including RTGC service time and possible waiting time. Stage 1 Travel Time (Yard to Berth) N Terminal Trucks µ 1 µ 1 N * * * µ 1 µ 4 µ 4 RTGC Stage 4 RTGC at Yard m 4 Servers RTGC m 4 µ 2 µ 2 QC QC m 2 Stage 2 QC at Berth m 2 Servers Stage 3 Travel Time (Berth to Yard) N Terminal Trucks µ 3 µ 3 N * * * µ 3 Fig. 1. A cyclic queue system (adapted from Koenigsberg and Lam, 1976).

4 S. Kang et al. / Transportation Research Part B 42 (2008) Suppose we have m 2 QC servers at the berth and m 4 RTGC servers at the yard, serving N trucks that cycle along the system. The trucks are considered as the customers of the cyclic queue system. Stages 1 and 3 are treated as free flow stages, having one server per truck with service time equal to the travel time between stages 2 and 4. We assume that the service time at each server is equal to 1/l i, where l i is the service rate per unit time at a server in stage i = 1, 2, 3, 4. We also assume that the service times at all servers follow negative exponential distributions. n Let n 1, n 2, n 3, n 4 be the number of trucks at stages 1, 2, 3, and 4 in the steady state, respectively, and X ¼ ðn 1 ; n 2 ; n 3 ; n 4 Þ : P o 4 i¼1 n i ¼ N; n i P 0; integer 8i be the state space of truck distribution at any time. According to Gordon and Newell (1967) and Koenigsberg and Lam (1976), the probability for the system to be in state (n 1, n 2, n 3, n 4 ) is given by the following expression: Pðn 1 ; n 2 ; n 3 ; n 4 Þ¼ ½X 1Š n 1 ½X 3 Š n 3 ½X 2 Š n 2 ½X 4 Š n 4 Pð0; N; 0; 0Þ 8ðn 1 ; n 2 ; n 3 ; n 4 Þ2X; ð1þ n 1!n 3! l 2 l 4 where X i ¼ l 2 l i 8i, and l i ¼ m i!m n i m i i ; if n i > m i. The expected number of trucks at stage i is n i!; if n i 6 m i n i :¼ X n i Pðn 1 ; n 2 ; n 3 ; n 4 Þ; i ¼ 1; 2; 3; 4; ð2þ X and the marginal probability that exactly r of the N trucks are in stage i is P N i ðrþ :¼ X n i¼r XPðn 1 ; n 2 ; n 3 ; n 4 Þ: ð3þ In order to measure the throughput of the system, the expected number of trucks waiting at stage i (applicable for stages 2 and 4 only) is calculated as follows: u i :¼ X1 Xmi 1 mi 1 ðr m i ÞP N i ðrþ ¼n X i rp N i ðrþ m i þ m i P N i ðrþ; i ¼ 2; 4: ð4þ r¼m i r¼0 r¼0 The truck cycle time equals the total average service time at all four stages and the waiting times at stages 2 and 4. The truck cycle time divided by the number of trucks in the system corresponds to the average rate for containers to be processed. Therefore, the expected headway between two consecutive truck arrivals (i.e., related to service " frequency ) is S ¼ 1 X # 4 1 þ W 2 þ W 4 ; ð5þ N l i i¼1 where W i is the expected waiting time for one truck at stage i, i.e., W i :¼ u i S: Combining (5) and (6) we have S ¼ 1 N u 2 u 4 X4 i¼1 1 u i ; and W i ¼ X4 l i N u 2 u 4 i¼1 ð6þ 1 l i ; i ¼ 2; 4: ð7þ The above formulae yield a relationship between the fleet size and the average cycle time in the steady state. This is a direct measure of the long-term average throughput of the unloading system. Very often terminals have to finish unloading containers within a given time frame (i.e., imposing a constraint on the average cycle time or headway). The best combination of fleet that minimizes operating costs while satisfying cycle-time requirements can be obtained based on (7). Suppose that the system cost is proportional to the number of active cranes and trucks (running or idling), as well as the cost for having the vessel waiting during the unloading operation at the berth; i.e., one QC, RTGC, truck, and the vessel incur in a cost of c q, c r, c t, and c w, respectively (per unit time). Assume too that the terminal needs an average container flow of at least D/H, where H is the maximum available time to unload D containers from a vessel. The problem for planning the optimal fleet size can be formulated as follows:

5 974 S. Kang et al. / Transportation Research Part B 42 (2008) Min m 2 ;m 4 ;N s:t: ðc q m 2 þ c r m 4 þ c t N þ c w ÞD S S 6 H=D m 2 ; m 4 ; N P 0; integer Note that S is determined from (1) (4) and (7), and D S is the actual time needed to unload the vessel Operational fleet management The above cyclic queue approach is based on two rather strong assumptions. First, it assumes exponentially distributed crane service times and truck travel times. In reality, this assumption is rarely satisfied; see Garrido and Allendes (2002) for some discussions. Simulation of the cyclic queue system with general service time distributions will address this issue. Second, the steady-state results do not apply to transient states (e.g., the beginning of an unloading process, or the unloading of a small ship). In this section, we present a Markovian decision model to optimally manage the fleet in real time; i.e., to provide guidance on how many QCs, RTGCs and trucks to operate at each specific time. For simplicity, we aggregate stages 2, 3 into one, and 4, 1 into another. Fig. 2 illustrates the operations of this aggregated two-stage system. Consider a time horizon (e.g., the time available to unload a vessel), H, in which all operations in the system (e.g., the truck travel time and crane service time) are stochastic. For any time t 2 [0, H], let x(t), x 0 (t), y(t) and y 0 (t) be the cumulative numbers of trucks that have entered service at the berth, arrived at the yard queue, entered service at the yard and arrived at the berth queue (after unloading a container at the yard), respectively; z(t) be the cumulative arrival of trucks available to enter service at the berth. As the number of containers is reasonably large, we assume that these cumulative values change continuously with time, as illustrated by the cumulative queuing diagrams (Newell, 1982) in Figs. 3 and 4. The shaded areas in Fig. 3 represent the total waiting time of trucks. We assume that the number of active fleet can be altered only at discrete times, t =(k 1)h, for k =1,2,..., K, such that H = Kh. This requirement may be due to machine set-up or labor shift requirements. At t = 0, the vessel, cranes and trucks are ready for unloading containers from the vessel. The decisions at the beginning of the kth time period, i.e., (k 1)h, include the active fleet configuration to use in the following time period; i.e., the number of active QCs, q(k), active RTGCs, r(k), and active trucks, n(k). Suppose that at any time t, the service times of QC and RTGC are 1/p q (t) and 1/p r (t) per container, respectively, such that their expectations over time satisfy E t [p q (t)] = l 2 and E t [p r (t)] = l 4 "t. Stage 2 Arriving at Berth Queue Leaving for Berth RTGCs Entering for Loading QCs Entering for Unloading Queue Arriving at Yard Stage 1 Leaving for Yard Fig. 2. A two-stage cyclic queue system.

6 S. Kang et al. / Transportation Research Part B 42 (2008) Fig. 3. Cumulative container counts processed in port operations. Fig. 4. Boundary between decision periods. We also assume that if any change is made to the fleet configuration (size) at kh, all cranes will stop processing new containers at kh s, where s is a positive value large enough for all trucks in processing or transition to arrive at the queues ahead by time kh. The time s may also be interpreted as the set-up time (or other penalty) associated with any fleet size change. This rather conservative operating strategy significantly reduces modeling complexity by decoupling the operations before and after a fleet configuration change. This ensures that by time kh, if any change is made, no truck will be en route, and thus only x(t) andy(t) are needed to specify the state of the system, 2 as illustrated in Fig. 4. In general, we expect s to be considerably smaller than h. 2 As will be mentioned in (8) (14), the value of z(kh) is solely determined by n(k) and y 0 (t). It is not an independent state variable.

7 976 S. Kang et al. / Transportation Research Part B 42 (2008) Terminal operators may or may not choose to apply the conservative strategy in the day-to-day operations. The model with positive s tends to overestimate the system cost and provides an upper bound to reality. On the other hand, we could alternatively choose to use s = 0 in the same model, assuming that all en route trucks may be advanced to the next queue instantly. This idealized scenario tends to underestimate system cost and provides a lower bound. In Section 3.2, we show with empirical data that these two bounds are not significantly different; i.e., our conservative choice of s values does not seem to significantly overestimate the system operating cost. Regardless of the number of active cranes and trucks, the conservation of truck flow in the cyclic system poses a set of constraints to x(t), y(t), x 0 (t), y 0 (t), and z(t) at any time within a period; i.e., the departure curve can never exceed the arrival curve, and if a queue exists, the throughput at each stage should be equal to handling capacity. For the kth period, we have the following: 8 >< qðkþp q ðtþ if xðtþ < zðtþ; xðtþ < D _xðtþ ¼ minfqðkþp q ðtþ; _zðtþg if zðtþ ¼xðtÞ < D 8t 2½ðk 1Þh; kh ^sðkþþ; >: 0 if xðtþ ¼D 8 >< rðkþp r ðtþ if x0ðtþ > yðtþ _yðtþ ¼ minfrðkþp r ðtþ; _x 0 ðtþg if x 0 ðtþ ¼yðtÞ 8t 2½ðk 1Þh; kh ^sðkþþ; >: 0 if yðtþ ¼D _xðtþ ¼_yðtÞ ¼0 8t 2½kh ^sðkþ; khþ: ð10þ ð8þ ð9þ Note that penalty time ^s is equal to s only if there is a fleet configuration change; i.e., 0; if qðkþ ¼qðk þ 1Þ; rðkþ ¼rðk þ 1Þ; nðkþ ¼nðk þ 1Þ; ^sðkþ ¼ s; otherwise: ð11þ Suppose the number of trucks in the system can be altered by adding or removing trucks only at the gate of the berth. This is reflected in (12) and (13) below. In case the number of trucks increases significantly, the cumulative truck arrival curve z(t) may experience a sudden jump at time kh. This scenario is described by (13): 0 if zðtþ > y0ðtþþnðkþ _zðtþ ¼ _y 0 ðtþ if zðtþ ¼y 0 ðtþþnðkþ 8t 2½ðk 1Þh; khþ; ð12þ zðkh þ Þ¼y 0 ðkhþþnðkþ; if zðkh Þ < y 0 ðkhþþnðkþ 8k; ð13þ x 0 1 ðdþ ¼ min 06j6xðtÞ;j integer x 1 ðjþþa j>x 0 1 ðd 1Þ fx 1 ðjþþa j g; y 0 1 ðdþ ¼ min 06j6yðtÞ;j integer y 1 ðjþþb j >y 0 1 ðd 1Þ fy 1 ðjþþb j g 8d 2½0; DŠ: ð14þ Constraint (14) describes the standard input output relationship of multiple servers with general service times, where random variable a j, for all j, represents an independent realization of QC processing time plus truck travel time from berth to yard. Here x 1 (j) represents the inverse function of x(t). Similarly, random variable b j, for all j, represents an independent realization of RTGC processing time plus truck travel time from yard to berth. Again, the objective is to minimize the total cost for operating the cranes and trucks for a given throughput requirement; e.g., unloading a total number of at least D containers in horizon [0, H], subject to uncertainty. Mathematically, the problem can be formulated as follows: Min s:t: X K k¼1 ð8þ ð14þ; ½c q qðkþþc r rðkþþc t nðkþþc w dðkþš h

8 S. Kang et al. / Transportation Research Part B 42 (2008) ; if xðkh hþ < D dðkþ ¼ 8k ¼ 1; 2;...; K; ð15þ 0; otherwise xð0þ ¼yð0Þ ¼0; zð0þ ¼nð1Þ; xðh Þ P yðh Þ P D; ð16þ qðkþ 2f0; 1;...; q max g; rðkþ 2f0; 1;...; r max g; nðkþ 2f0; 1;...; n max g 8k: ð17þ The objective function includes fleet operating costs and vessel waiting costs. Indicator variable d(k) determines if the vessel has to stay at the berth (i.e., the unloading task is not completed) in the kth period. It is assumed here that these costs incur for the whole time period based on the fleet configuration and vessel status at the beginning of the period. Other cost components can be easily incorporated into the objective function as well; the modeling approach will still apply. Constraint (16) describes boundary conditions at the beginning and the end of the horizon. Note that x(t) and y(t) are always continuous, while z(t) may have jumps at t = kh. Constraint (17) mandates that all active fleet sizes are non-negative integers within a range Solution technique Observing the cost structure and dynamics, the optimization problem can be solved as a Markovian decision process (MDP). Variables x(kh), y(kh) 2 {0,1,...,D} describe the number of containers transported by the end of the kth period. When we face decision making (at the beginning of period k) the state of the system can be specified as s(k) :¼ [x((k 1)h), x((k 1)h) y((k 1)h), k 1]. Note that x((k 1)h) y((k 1)h) is always bounded from above by n max, therefore, the size of the state space S is bounded from above by (D +1) n max K. At the beginning of period k, the terminal operator determines q(k), r(k), and n(k). Any sample-path sequence of fleet sizes in all consecutive periods, k = 1, 2,..., K, can be equivalently described by (i) the sequence of all new fleet sizes and (ii) the number of future periods during which each new fleet size will be used. As such, we may define the control variables as a(k) :¼ [q(k), r(k), n(k), m(k)] to indicate that the same fleet size (q(k), r(k), n(k)) is implemented from period k to period k + m(k) 1. This definition implies that the fleet sizes in period k 1 and period k + m(k) are different from (q(k), r(k), n(k)). The size of the control space A, based on (17), is bounded from above by (q max +1)(r max +1)(n max +1)K. The MDP formulation of the fleet sizing problem is illustrated in Fig. 5. If the state is currently s(k), then implementing a new fleet size a(k) will bring the state to s(k + m(k)); due to stochasticity, the elements in s(k + m(k)) may take a set of values according to a set of transition probabilities Pr(s(k), s(k + m(k))ja(k)). There are stage-wise costs associated with all possible transitions. The costs for transition from s(k) to s(k + m(k)) under control a(k) is denoted by CðsðkÞ; aðkþþ ¼ mðkþh½c q qðkþþc r rðkþþc t nðkþþc w dðkþš: We have just mentioned that our model set-up implies that a new fleet configuration will be implemented at time (k + m(k) 1)h; i.e., the fleet configuration in a(k) must be different from the next fleet configuration in a(k + m(k)). We modify the MDP stage-wise optimization algorithm to accommodate for this constraint. For each state s in S, define J 1 (s) as the minimum expected loss function (achieved with control variable a 1 (s)) and J 2 (s) the second minimum expected loss function (achieved with control variables a 2 (s)). These loss functions and the corresponding optimal policies can be found from the stage-wise optimization below: JðsðkÞÞ ¼ min aðkþ fcðsðkþ; aðkþþ þ E~Jðsðk þ mðkþþjsðkþ; aðkþþg 8sðkÞ; 8k; ð18þ S A S s(k) =[x((k 1)h), x((k 1)h) y((k 1)h), k 1] a(k) = [q(k), r(k), n(k), m(k)] s(k+m(k)) Fig. 5. One step in the Markovian decision process.

9 978 S. Kang et al. / Transportation Research Part B 42 (2008) where E~J ðsðk þ mðkþþjsðkþ; aðkþþ :¼ þ P sðkþmðkþþ aðkþ6¼a 1 ðsðkþmðkþþþ P sðkþmðkþþ aðkþ¼a 1 ðsðkþmðkþþþ ½J 1 ðsðk þ mðkþþþ PrðsðkÞ; sðk þ mðkþþjaðkþþš ½J 2 ðsðk þ mðkþþþ PrðsðkÞ; sðk þ mðkþþjaðkþþš: By labeling two minimum expected loss functions, we are able to use J 2 (s) instead of J 1 (s) in the backward iteration when the best fleet configuration a 1 (k + m(k)) is same as a(k). Due to uncertainty in the service times, it is essential to determine the transition probabilities P(s(k), s(k + m(k))ja(k)). The closed-form result is rather difficult to obtain and a simulation based approach is proposed. Note that constraints (8) (14) are translationally symmetric with regard to k and the value of x(kh). The transition under a decision at time kh is dependent of the other elements in s(k), s(k + m(k)) and a(k), which includes xðkh þ mðkþhþ xðkhþ; xðkhþ yðkhþ; xðkh þ mðkþhþ yðkh þ mðkþhþ and aðk þ 1Þ: The total number of transition probabilities is therefore D (n max ) 2 jaj. Standard stage-wise optimization techniques can then be applied to the MDP model. Before the iterations, the initial values of the loss functions are set to be the following: 0; if xðhþ P yðhþ P D; k ¼ K ej ðsjs; aþ ¼ : 1 otherwise 3. Case study This section presents a case study where the queuing model and the MDP model are applied to practical port terminal operations. The empirical data are provided by the Port of Balboa in the Panama Canal. The problem is to find the optimum fleet combination and operations to unload D = 230 containers in H = 5 h, with hourly time shift (h =1h, K = 5). The terminal is able to provide for each vessel up to 3 QCs, 6 RTGCs, and a fleet of 15 trucks; i.e., q max =3,r max =6,n max = 15. The operating cost for each type of equipment is calculated based on the cost of operating a truck, and includes the labor costs, maintenance, power, and lubricants. It was estimated that the waiting cost of a vessel at the berth and the operating costs of a QC and a RTGC are about 10.5 times, 3.1 times and 1.8 times the cost of operating a truck, respectively; i.e., c w :c q :c r :c t = 10.5:3.1:1.8:1 (US Army Corps of Engineers, 2004; Thomas and Roach, 1987). More detailed cost component analysis is possible, but these values are reasonable to illustrate the proposed methodology. Without losing generality, we will normalize all costs to truck-hour cost equivalent in this analysis. Table 1 presents the statistics of the service times at the cranes and the truck travel times at the Port of Balboa. These times are found to fit into normal or log-normal distributions better than exponential distributions. For example, the goodness-of-fit test of QC unloading times is illustrated in Fig. 6, where normal and 3-parameter log-normal distributions fit the data quite well. The value of s is set to be enough for all trucks in processing and transition to arrive at the queues ahead by the next decision time with 99% statistical significance. It should include the processing times at QC or RTGC and the travel times between yard and berth, each of which approximately follow a lognormal distribution. The Fenton Wilkinson s method (Hekmat, 2006) is used to approximate a sum of lognormal distributions, and the value of s is calculated to be 7.65 min. Table 1 Service time statistics at the Port of Balboa Mean (min) Standard deviation (min) QC service time RTGC service time Travel time from berth Travel time to berth

10 S. Kang et al. / Transportation Research Part B 42 (2008) Fig. 6. Goodness-of-fit for QC unloading times Steady-state analysis with the cyclic queue model We first use (1) (7) to calculate the expected time for unloading all D = 230 containers in the steady state, with various equipment combinations. See Fig. 7. Solutions become feasible for all combinations of QC and RTGC with a minimum of 8 trucks. Also, increasing the number of QC and RTGC from 1 to 2 or 3 does not significantly reduce the operating time when the number of trucks is less than 7. Note that increasing the number of RTGC to 2, while keeping only 1 QC, did not improve the processing rate significantly. This is intuitive since the processing rates of QC and RTGC are similar, and the system can only run as fast as the slower stage. In that case, vehicles will be queued at stage 2 because they are being served at almost twice the rate at stage 4. System operating costs are then calculated for all combinations of QC, RTGC and trucks that are feasible, as shown in Fig. 8. Efficiency enhancement from increasing the number of QC and RTGC are observed only when enough trucks are provided. Out of all combinations it is found that operating 2 QC and 2 RTGC with 15 trucks yields the minimum cost of 87.6 truck-hour cost equivalents. Using only 1 QC and 1 RTGC generates significantly higher costs, given that it is relatively expensive to have the vessel waiting at the deck. Using 3 QC and 3 RTGC does not speed up the operation enough to compensate for the increase of operating costs. These results clearly show the tradeoff between having faster container throughput (while using more resources per unit time) and saving the size of the fleet (while having longer cycles) Operational fleet management with MDP The simulation module and MDP are coded and compiled in the Microsoft Visual C++ Environment, and executed on a personal computer with a 3.0 GHz Pentium 4 CPU and 1.24GB RAM. The MDP is solved within 4 min of CPU time. The transition probabilities are calculated from 500 simulation runs. An example

11 980 S. Kang et al. / Transportation Research Part B 42 (2008) QC + 1RTGC 1QC + 2RTGC 2QC + 2RTGC 2QC + 3RTGC 3QC + 3RTGC 10 9 Hours Number of Trucks Fig. 7. Expected unloading time in steady-state operations. 180 Cost (Truck Wages * Hour) QC + 1RTGC 1QC + 2RTGC 2QC + 2RTGC 2QC + 3RTGC 3QC + 3RTGC Number of Trucks Fig. 8. Operating cost as a function of equipment selection. of the simulated probabilities is depicted in Fig. 9, where x(k) = 0 and x(k) y(k) = 0 before 1 QC, 2 RTGC, and 6 trucks are operated for two hours (i.e., m(k) = 2). At the end of the 2-h period, there is a probability of

12 S. Kang et al. / Transportation Research Part B 42 (2008) Fig. 9. Example of simulated transition probabilities that x(k + m(k)) = 73 containers will be unloaded from the vessel, x(k + m(k)) y(k + m(k)) = 3 trucks will be waiting at the yard, and n(k) [x(k + m(k)) y(k + m(k))] = 3 trucks will be waiting at the berth. It is not easy to plot the optimal policy from MDP due to the high dimensions of the state and control variables. Table 2 gives the most likely sample-path solution for s = 7.65 min, where the optimal action is to use the same a(k) =[q(k), r(k), n(k), m(k)] = [2, 2, 13, 3] for three consecutive hours to complete the unloading task. In this case, the positive value of s serves as a penalty against fleet size changes. The total cost for this sample path solution is truck-hour equivalents. For comparison, we consider the aggressive case where s is set to be zero. The most likely sample-path solution is given in Table 3. The optimal action in the first hour is a(1) = [2, 2, 12, 1]. Due to the stochasticity, there are many possible states at the beginning of the second hour. Among them, s(2) = [x(1), x(1) y(1), 1] = [77, 5, 1] has the largest transition probability of J(s(k)) represents the expected cost-to-go. The minimum expected total cost for this most likely sample-path solution is truck-hour equivalents, which is only 3.2% lower than the cost with s = 7.65 min. For additional comparison, the optimal solution from the cyclic queue model, i.e., a fleet configuration of 2 QC, 2 RTGC and 15 trucks, is examined in the MDP framework with log-normal service times (rather than exponential times). Table 4 shows the most likely sample-path solution under this fixed fleet configuration. The time used for unloading all the containers is 3 h and the total cost is truck-hour cost equivalents, Table 2 The most likely sample-path for operational fleet management with s = 7.65 min Period (k) x(kh) 230 y(kh) 230 x(kh) y(kh) 0 Number of truck, n(k) 13 0 Number of RTGC, r(k) 2 0 Number of QC, q(k) 2 0 m(k) 3 1 J(s(k)) Transition probability

13 982 S. Kang et al. / Transportation Research Part B 42 (2008) Table 3 The most likely sample-path for operational fleet management with s = 0 Period (k) x(kh) y(kh) x(kh) y(kh) Number of truck, n(k) Number of RTGC, r(k) Number of QC, q(k) m(k) J(s(k)) Transition probability Table 4 The most likely sample-path with the cyclic queue model solution Period (k) x(kh) 230 y(kh) 230 x(kh) y(kh) 0 Number of truck, n(k) 15 0 Number of RTGC, r(k) 2 0 Number of QC, q(k) 2 0 m(k) 3 1 J(s(k)) Transition probability about 20% larger than the predicted result of from the cyclic queue model. Although, by definition, the most likely sample-path cost from MDP is not exactly equivalent to the expected cost from the cyclic queue model, it is obvious that the cost difference partly comes from the assumptions of steady-state operations and exponentially distributed service times. The cost of is also higher than , the cost under the conservative strategy (s = 7.65). These observations demonstrate that as a complement of the cyclic queue model, which is suitable for long-term fleet planning, the MDP model can further improve system performance by varying fleet configurations in real time. 4. Fleet allocation among multiple berths In many cases, the terminal has full information on vessels arrival and departure schedules at B > 1 berths and needs to determine an allocation plan for the q max QCs, r max RTGCs, and n max trucks. To simplify the problem, we only allow terminals to change configuration at fixed times kh, k = 1, 2,..., K. Assume that the number of containers unloaded at berth b by time kh is x b (k), and the number of QCs, RTGCs, and trucks allocated to berth b is q b (k), r b (k), and n b (k). The allocation is constrained by available resource: X B b¼1 q b ðkþ 6 q max ; X B b¼1 r b ðkþ 6 r max ; X B b¼1 n b ðkþ 6 n max 8k: ð19þ Assuming that within each period the cyclic queue model approximately holds and the throughput S is given by (7) as a function of q b (k), r b (k), and n b (k). Thus the expected increase in unloaded containers is x b ðk þ 1Þ ¼x b ðkþþh=sðq b ðkþ; r b ðkþ; n b ðkþþ: ð20þ At berth b, the consecutive vessels have scheduled departure times, k b1 h,..., k bj h,...(assuming that they fall onto a time lattice of interval h) and predetermined numbers of containers to unload D b1,..., D bj,... This

14 translates into a constraint on the operations, such that the cumulative number of containers unloaded to be at least the cumulative number scheduled before the departure of that vessel: x b ðk bj Þ P Xj i¼1 D bi 8b; j: ð21þ When we assume that all vessels arrival and departure times are known and fixed, the vessel waiting costs become constant. The fleet management problem is equivalent to minimize the following: Min XK k¼1 X B b¼1 ½c q q b ðkþþc r r b ðkþþc t n b ðkþš h; such that (1) (7), (19), (20) and (21) are satisfied. The solution techniques introduced for a single berth will still be applicable to this multi-berth problem. In case the cyclic queue model is not appropriate to describe the dynamics within interval h, the simulation results presented in Section 2.2 can be applied to substitute (20). 5. Conclusions In this paper we presented modeling frameworks for the planning and management of optimal marine terminal fleets to unload cargos from container vessels. A cyclic queue model was first developed to yield long-term steady-state performances. It assumes exponentially distributed service times in order to obtain closed-form analytical results. A complementary MDP model is then presented to provide real-time operating strategies. The MDP models allow for dynamic fleet allocations and general service time distributions. In practice, the cyclic queue model can help port operators to conduct strategic planning of their fleets, while the MDP model can help improve system performance by allowing for a more efficient use of the equipment in real time. Both models are also suitable for extensions to terminals with multiple berths. The frameworks (e.g., the MDP models) introduced in this paper can be extended in several directions. For example, fixed costs for changing fleet configuration can be easily incorporated into the cost formula. The system dynamics Eqs. (8) (14) can be equivalently written in terms of discrete jump functions and the transition probabilities can be estimated with discrete event simulations. The stochastic effect of crane breakdowns can be considered in the transition probability simulations. More detailed operating requirements, such as handling, storage of refrigerated containers and security screening, can be easily incorporated into the modeling framework. It might also be desirable to extend both the cyclic queue model and MDP model to accommodate the double cycling strategy proposed by Goodchild and Daganzo (2006). Finally, the continuous dynamic programming approach may be applied to optimizing fleet configurations in a non-discrete time horizon. Acknowledgement The authors thank Mr. Edwin Lewis for providing data of the Port of Balboa, Panama. References S. Kang et al. / Transportation Research Part B 42 (2008) Bish, E.K., Leong, T., Li, C., Ng, J.W.C., Simchi-Levi, D., Analysis of a new vehicle scheduling and location problem. Naval Research Logistics 48, Chen, Y., Leong, Y.T., Ng, J.W.C., Demir, E.K., Nelson, B.L., Simchi-Levi, D., Dispatching automated guided vehicles in a mega container terminal. In: Paper presented at the Institute for Operations Research and the Management Sciences (INFORMS) Conference, Montreal, Canada. Garrido, R.A., Allendes, F., Modeling the internal transport system in a container port. Transportation Research Record: Journal of the Transportation Research Board 1782, Goodchild, A.V., Daganzo, C.F., Double-cycling strategies for container ships and their effect on ship loading and unloading operations. Transportation Science 40 (4), Goodchild, A.V., Daganzo, C.F., Crane double cycling in container ports: planning methods and evaluation. Transportation Research Part B 41, Gordon, W.J., Newell, G.F., Closed queuing systems with exponential servers. Operations Research 15 (2), Hekmat, R., Ad-hoc Networks: Fundamental Properties and Network Topologies. Springer, Netherlands.

15 984 S. Kang et al. / Transportation Research Part B 42 (2008) Kim, K.H., Bae, J.W., A dispatching method for automated guided vehicles to minimize delays of containership operations. International Journal of Management Science 5 (1), Koenigsberg, E., Lam, R.C., Cyclic queue models of fleet operations. Operations Research 24 (3), Narasimhan, A., Palekar, U.S., Analysis and algorithms for the transtainer routing problem in container port operations. Transportation Science 36 (1), Newell, G.F., Applications of Queuing Theory. Chapman & Hall, New York. Shields, J.J., Container stowage: a computer aided preplanning system. Marine Technology 21 (4), Steenken, D., Henning, A., Freigang, S., Voss, S., Routing of straddle carriers at a container terminal with the special aspect of internal moves. OR Spektrum 15 (3), The World Bank Group, Port and logistics overview. Washington, DC. < Thomas, B.J., Roach, D.K., Operating and maintenance features of container handling systems. The World Bank Policy planning and Research Staff. Infrastructure and Urban Development Department. Washington, DC. US Army Corps of Engineers, Consolidated implementation of the New York and New Jersey harbor deepening project-limited reevaluation report. < Vis, I.F.A., Koster, R., de Roodbergen, K.J., Peeters, L.W.P., Determination of the number of automated guided vehicles required at a semi-automated container terminal. Journal of the Operational Research Society 52 (4),