Simulation-based optimization methods for urban transportation problems. Carolina Osorio

Transcription

1 Simulation-based optimization methods for urban transportation problems Carolina Osorio Civil and Environmental Engineering Department Massachusetts Institute of Technology (MIT) Joint work with: Prof. Michel Bierlaire (EPFL) and graduate students (MIT): Xiao Chen, Linsen Chong and Kanchana Nanduri. ITE Seminar, Purdue University, March 2013 Carolina Osorio p.1

2 Outline Context Metamodel simulation-based optimization Metamodel for urban transportation problems Signal control case studies Ongoing work Carolina Osorio p.2

3 Context Congested urban networks: complex traffic dynamics. To derive and evaluate traffic management schemes for congested networks: microscopic traffic simulators These simulators embed numerous detailed and realistic traffic models But this realisms leads to functions that are: - nonlinear - stochastic - expensive to evaluate - with no closed form available - gradient estimations are also more involved (noise, availability of source code) Integrating these simulators within an optimization context is an intricate task Carolina Osorio p.3

4 Context Simulation models - capture complexity of the network: realistic - difficult to optimize Analytical models - simple models with sound mathematical properties - optimization friendly: analytical gradients are available, less expensive to evaluate Objective: Use detailed simulation models to efficiently solve complex transportation problems Solve problems under tight computational budgets Approach: to combine information from: simulation model + analytical model Preserve realism + achieve tractability Carolina Osorio p.4

5 Context Problems of interest: Objective function: simulation-based Constraints: general form (non-convex), analytical, differentiable min E[f(x,z;p)] x Ω Tight computational budget: No initial observations available 150 simulation evaluations No a priori solution information: e,g, initialization often with a random point drawn uniformly from the feasible space Carolina Osorio p.5

6 Context Problems of interest: Case studies: traffic signal control problems Scalability: large-scale control Efficient use of vehicle-specific information: energy-efficient control Efficient use of higher-order distributional information: travel time reliable control Carolina Osorio p.6

7 Simulation-based optimization (SO) Metamodel methods min E[f(x,z;p)] x Ω Simulation models are expensive to evaluate Use simpler approximations within the optimization framework. The stochastic response of the simulation is replaced by a deterministic metamodel response function, then deterministic optimization techniques are used. Compute the gradient of a surrogate model (or metamodel), as opposed to using the gradient of the simulation response Carolina Osorio p.7

8 Simulation-based Optimization Optimization routine performance estimates (m(x), m(x)) Trial point (new x) Metamodel Optimization based on a metamodel Update m based on ˆf(x) Simulator Evaluate new x Adapted from Alexandrov et al., 1999 Carolina Osorio p.8

9 Metamodel Methods Metamodels are often a linear combination of basis functions from parametric families General purpose metamodels include: - Linear or quadratic polynomials - Spline models: continuous piecewise polynomials - Radial basis functions: sum of radially symmetric functions centered at different points in the search space (Oeuvray and Bierlaire, 2009) Instead of using a general purpose metamodel we use one that captures the network structure and the interactions between the main network components Carolina Osorio p.9

10 Metamodel Metamodel: functional + physical m(x,y;α,β,q) = αt(x,y;q)+φ(x;β) Combination of: T : the analytical queueing model φ: a quadratic polynomial x : decision vector y : endogenous queueing model variables, such as stationary queue length distributions, congestion indicators q : exogenous queueing model paramaters, such as network topology, total demand α, β : metamodel parameters Carolina Osorio p.10

11 Metamodel Polynomial φ: Quadratic with diagonal second derivative matrix This choice is based on existing numerical experiments which show that they are often more efficient than full quadratic models for derivative-free trust region methods (Powell, 2003) d d φ(x;β) = β 0 + β j x j + β d+j x 2 j j=1 j=1 Carolina Osorio p.11

12 Metamodel m(x,y;α,β,q) = αt(x,y;q)+φ(x;β) At each iteration k the parameters β and α of the metamodel are fitted using the current sample by solving the least squares problem: min α,β n k i=1 { w ki (ˆf(x i,z i ;p) m(x i,y i ;α,β,q))} 2 + (w0.(α 1)) 2 + 2d+1 i=1 (w 0.β i ) 2 x i : i th point in the sample ˆf(x i,z i ;p): corresponding simulated observation w ki : weight associated to the i th observation w 0 : fixed weight for augmented data Least squares problem solved using the Matlab routine lsqlin Carolina Osorio p.12

13 Metamodel n k i=1 { w ki (ˆf(x i,z i ;p) m(x i,y i ;α,β,q))} 2, Weights: Capture the importance of each point with regards to the current iterate Atkeson (1997) surveys weight functions and analyzes their theoretical properties We use the inverse distance weight function, along with the Euclidean distance The weight of a given point is therefore inversely proportional to its distance from the current iterate w ki = 1 1+ x k x i 2 Carolina Osorio p.13

14 Network models m(x,y;α,β,q) = αt(x,y;q)+φ(x;β) Combination of: T : the analytical queueing model φ: a quadratic polynomial We present a metamodel that combines: 1. Calibrated microscopic traffic simulation model of the Lausanne city center Dumont and Bert, Analytical queueing network model Osorio and Bierlaire, 2009 Carolina Osorio p.14

15 Network model 1 Calibrated microscopic traffic simulation model : SIMLO Developed in LAVOC, EPFL (Dumont and Bert, 2006) Lausanne city center Carolina Osorio p.15

16 Network model 2 Analytical queueing model Finite capacity queueing theory Captures the key elements of the underlying network structure, e.g. how upstream and downstream queues interact, and how this interaction is linked to network congestion Blocking : describes how and where congestion arises, propagates and how it impacts the networks performance Novel state space formulation: explicitly models blocking events System of nonlinear equations Carolina Osorio p.16

17 Network model 2 λ eff i = γ i (1 P(N i = K i ))+ j p ji λ eff j λ i 1 µ i = = λ eff i /(1 P(N i = K i ) i ˆµ j) λ eff j /(λeff j I + 1 ˆµ i = 1 µ i +P f i / µ i P f i P(N i = k i ) = ρ i = j = λ i ˆµ i. p ij P(N j = k j ) 1 ρ i ρ k 1 ρ k i +1 i i Endogenous parameters describe congestion in terms of its: - sources (conditional transition probabilities) - frequency (blocking probabilities) - how it propagates/dissipates (unblocking rates) - its impact (expected blocked vehicles) For more details and a case study see Osorio and Bierlaire (2009) It has been successfully used in past work to solve a fixed-time traffic signal control problem (Osorio and Bierlaire, 2008) Carolina Osorio p.17

18 Optimization framework How can we integrate this metamodel within an optimization framework? Conn (2009) framework chosen for 3 reasons: 1. Derivative-free 2. Trust region method 3. Makes no assumption on how these metamodels are derived (interpolation or regression) We integrate our metamodel within the Conn (2009) framework Carolina Osorio p.18

19 DF TR algorithm Builds upon the Basic trust-region algorithm (Conn et. al, 2000) For a given iteration k the algorithm considers a metamodel m k, an iterate x k and a TR radius k. Each iteration consists of 5 main steps. - Criticality step. This step may modify m k and k if the measure of stationarity is close to zero. - Step calculation. Approximately solve the TR subproblem to yield a trial point - Acceptance of the trial point. The actual reduction of the objective function is compared to the reduction predicted by the model, this determines whether the trial point is accepted or rejected - Model improvement. Either certify that m k is fully linear in the TR or carry out improvement steps - TR radius update. Carolina Osorio p.19

20 Simulation-based Optimization Optimization routine performance estimates (m(x), m(x)) Trial point (new x) Metamodel Optimization based on a metamodel Update m based on ˆf(x) Simulator Evaluate new x Adapted from Alexandrov et al., 1999 Carolina Osorio p.20

21 Traffic Control Problem Traffic signal optimization Fixed-time signal control problem Offsets, cycle time, all-red durations and stage structure are given Determine green times for each phase Carolina Osorio p.21

22 Optimization Problem Carolina Osorio p.22

23 Optimization Problem min x,z E[f(x,z;p)] subject to: x(j) = b i, i I j P I (i) x x L x(j): green ratio of phase j (green time of phase j divided by the cycle time of its corresponding intersection) f: simulated performance measure z: endogenous simulation variables p: exogenous simulation parameters Carolina Osorio p.23

24 TR subproblem At a given iteration k the TR subproblem is formulated as follows: min x,y m k(x,y;q,α k,β k ) (1) subject to x(j) = b i, i I (2) j P I (i) l(x,y;q) = 0 (3) x x k 2 k (4) y 0 (5) x x L (6) Includes 2 more constraints than the previous problem: 1. The TR constraint, which uses the Euclidean norm 2. The system of nonlinear equations that define the queueing network model See implementation notes in Osorio (2010) for more details Carolina Osorio p.24

25 Empirical analysis Tight computational budget: No initial observations available 150 simulation evaluations Uniformly drawn initial signal plan We run the corresponding algorithm 10 times, deriving 10 signal plans Simulation model is used to evaluate the effect of the signal plans upon the entire Lausanne network. For each signal plan: 50 replications Empirical cumulative distribution function of the average travel times Carolina Osorio p.25

26 Case studies Large-scale control Energy-efficient control Reliable travel time control Carolina Osorio p.26

27 Large-scale SO Carolina Osorio p.27

28 Highly tractable analytical model λ eff i = γ i (1 P(N i = K i ))+ j p ji λ eff j λ i 1 µ i = = λ eff i /(1 P(N i = K i ) j /(λeff i ˆµ j) j I + λ eff 1 ˆµ i = 1 µ i +P f i / µ i P f i P(N i = k i ) = ρ i = j = λ i ˆµ i. p ij P(N j = k j ) 1 ρ i 1 ρ k i +1 i ρ k i λ eff i ρ eff i P(N i = k i ) = = γ i (1 P(N i = k i ))+ ( j p jiλ )( eff j = λeff i + p µ ij P(N j = k j ) i j D i 1 ρ eff i 1 (ρ eff i )k i +1(ρeff i )k i. ρ eff j j D i ) Carolina Osorio p.28

29 Large-scale SO 603 links and 231 intersections Signal control: city-wide performance, 17 intersections, 99 endogenous phases Queueing model: 902 queues TR subproblem: 2805 variables, 1821 nonlinear equality constraints, 902 linear equality constraints Large-scale for signal control problems (Aboudolas et al., 2010) derivative-free algorithms (Conn et al., 2009) microscopic, stochastic, constrained problem Carolina Osorio p.29

30 Empirical analysis Carolina Osorio p.30

31 Empirical analysis Carolina Osorio p.31

32 Empirical analysis Empirical cdf F(x) m φ x: average travel time in the network [min] x 0 Carolina Osorio p.32

33 Empirical analysis Empirical cdf F(x) Empirical cdf F(x) m 0.1 φ x x: average travel time in the network [min] 0.2 m 0.1 φ x x: average travel time in the network [min] Carolina Osorio p.33

34 Empirical analysis Empirical cdf F(x) m 50 m φ 50 φ φ 200 φ φ 600 φ φ 1500 x x: average travel time in the network [min] Empirical cdf F(x) m 50 m φ 50 φ 150 φ 200 φ φ x x: average travel time in the network [min] Carolina Osorio p.34

35 Energy-efficient signal control General research question: Can we use instantaneous vehicle-specific information (e.g., energy consumption, emissions) to identify improved transportation strategies? Challenge of using very noisy outputs If so, can we do this in a computationally efficient (i.e., practical) manner? Energy consumption Transportation is an energy-intensive sector Light-duty vehicles account for 47% of petroleum consumption (Heywood et al., 2009) Can we derive traditional traffic management strategies that improve both traditional metrics as well as city-wide energy consumption? Macro. vs. micro: Traffic models Fuel consumption models Detail - tractability trade-off Carolina Osorio p.35

36 Fuel consumption model - Micro Traditional micro. fuel consumption model (Akcelik, 1983) Considers 4 operating modes idling (I), deceleration (D), acceleration (A), cruising (C) During a time step of lenght t, the fuel consumption by a given vehicle j is given by: F j = Fj I ti j +FD j td j +(C1 j +C2 j a jv j )t A j +[C3 j (1+ v3 j 2Vj m )+Cj 4 v j]t C j Fj I,FD j,c1 j,c2 j,c3 j,v j m,c4 j : vehicle-specific parameters (can be derived from manufacturer provided details) v j,a j : instantaneous speed and acceleration t M j : Time vehicle j spent in a given mode M {I,D,A,C} Carolina Osorio p.36

37 Energy-efficient signal control Derived analytical (macro.) approximation of fuel consumption metric. Comparison of objective functions, minimize: Expected travel time Expected fuel consumption A weighted combination of the above two Carolina Osorio p.37

38 Energy-efficient signal control Carolina Osorio p.38

39 Travel time reliable signal control General research question: Can we use higher-order distributional information of the main performance measures (e.g. variance, full distributional information) from the simulator to identify more reliable or more robust transportation strategies? Can this be done in a computationally efficient (i.e., practical) manner? Travel time variability Derived analytical approximation of link travel time variance Carolina Osorio p.39

40 Travel time reliable signal control Carolina Osorio p.40

41 Conclusions Framework for simulation-based optimization of congested networks Metamodel that combines information from a simulator and an analytical network model Derivative-free trust region algorithm Use of analytical macroscopic traffic information to design computationally efficiency SO methods Traffic signal control problems Tight computational budget Carolina Osorio p.41

42 Ongoing work Dynamic problems Sustainable traffic management problems: emissions Microscopic model calibration problems Multi-model problems Development of SO techniques for small sample size problems: point selection, computational budget allocation Carolina Osorio p.42

43 References Chick and Inoue (2001). New Two-Stage and Sequential Procedures for Selecting the Best Simulated System Operations Research 49(5): Conn, Scheinberg and Vicente (2009). Global convergence of general derivative-free trust-region algorithms to first- and second-order critical points SIAM Journal on Optimization 20(1):387âĂŞ415. Dumont and Bert (2006). Simulation de l agglomération Lausannoise SIMLO. Technical Report. LAVOC, ENAC, EPFL. Osorio and Bierlaire (2009). An analytic finite capacity queueing network model capturing the propagation of congestion and blocking. European Journal Of Operational Research 196(3): Osorio (2010). Mitigating network congestion: analytical models, optimization methods and their applications Ph.D. thesis, Ecole Polytechnique Fédérale de Lausanne. Osorio and Bierlaire (2010). A simulation-based optimization approach to perform urban traffic control. Proceedings of the Triennial Symposium on Transportation Analysis. Osorio and Chong (2012). Large-scale simulation-based traffic signal control. Proceedings of the International Symposium on Dynamic Traffic Assignment (DTA) Osorio and Nanduri (2012). Energy-efficient traffic management: a microscopic simulation-based approach. Proceedings of the International Symposium on Dynamic Traffic Assignment (DTA) Chen, Osorio and Santos (2012). A simulation-based approach to reliable signal control. Proceedings of the International Symposium on Transportation Network Reliability Osorio and Bidkhori (2012). Combining metamodel techniques and Bayesian selection procedures to derive computationally efficient simulation-based optimization algorithms. Proceedings of the Winter Simulation Conference Carolina Osorio p.43