Distributed Control in Transportation and Supply Networks

Transcription

1 Distributed Control in Transportation and Supply Networks Marco Laumanns, Institute for Operations Research, ETH Zurich Joint work with Harold Tiemessen, Stefan Wörner (IBM Research Zurich) Apostolos Fertis, Martin Fuchsberger, Rico Zenklusen (ETH Zurich) Gabrio Caimi (BLS AG, Bern), Marco Lüthi (systransis Ltd, Zug) Workshop on Distributed Model Predictive Control and Supply Chains, Lund, May 19-21, 2010

2 1 Stochastic Control in Inventory Networks Problem and Model Basetock Policies as Local Heuristics Model with Replenishment Lead Times 2 Multi-level approach Online control of a station area Marco Laumanns Distributed Control in Transportation Networks 19 May / 26

4 A spare parts supply network Marco Laumanns Distributed Control in Transportation Networks 19 May / 26

5 Stochastic control model central warehouse stocking points i u 1 v1 x 1 x 2 x 3... u 11 u ij u 22 u 33 customer locations j d 1 d 2 d 3 d 4 Marco Laumanns Distributed Control in Transportation Networks 19 May / 26

6 Stochastic control model central warehouse stocking points i u 1 v1 x 1 x 2 x 3... u 11 u ij u 22 u 33 customer locations j d 1 d 2 d 3 d 4 Model with linear dynamics and constraints x i (t + 1) = x i (t) + u i (t) j C i u ij (t) and d j (t + 1) = D j (t) where v j (t) + i S j u ij (t) = d j (t) and j C i u ij (t) x i (t) Marco Laumanns Distributed Control in Transportation Networks 19 May / 26

7 Stochastic control model central warehouse stocking points i u 1 v1 x 1 x 2 x 3... u 11 u ij u 22 u 33 customer locations j d 1 d 2 d 3 d 4 Immediate costs g(x, u, v) = i h i x i } {{ } inventory costs + i j C i c ij u ij } {{ } transportation costs + m j v j j } {{ } emergency shipments Marco Laumanns Distributed Control in Transportation Networks 19 May / 26

8 Simplified model central warehouse stocking points u 1 v1 x 1 x 2 u 11 u ij u 22 customer locations d 1 d 2 Marco Laumanns Distributed Control in Transportation Networks 19 May / 26

9 Simplified model central warehouse u 1 v1 u 1 v 1 v 2 u 2 stocking points x 1 x 2 u x 21 1 x 2 u 12 u 11 u ij u 22 D 1 D 2 customer locations d 1 d 2 Marco Laumanns Distributed Control in Transportation Networks 19 May / 26

10 Simplified model central warehouse u 1 v1 u 1 v 1 v 2 u 2 stocking points x 1 x 2 u x 21 1 x 2 u 12 u 11 u ij u 22 D 1 D 2 customer locations d 1 d 2 Simplified dynamics, constraints, and costs x 1 (t + 1) = x 1 (t) + u 1 (t) D 1 (t) + u 21 (t) + v 1 (t) where u 21 (t) + v 1 (t) x 1 (t) and cost g 1 (x, u, v) = h 1 (x 1 + u 21 + v 1 ) + c 21 u 21 + m 1 v 1 Marco Laumanns Distributed Control in Transportation Networks 19 May / 26

12 Basestock policies Basestock policy { S i x i (t) if x i S i u i (t) = 0 else S i : basestock level u 1 v 1 v 2 u 2 u x 21 1 x 2 u 12 D 1 D 2 Marco Laumanns Distributed Control in Transportation Networks 19 May / 26

13 Basestock policies Basestock policy { S i x i (t) if x i S i u i (t) = 0 else S i : basestock level u 1 v 1 v 2 u 2 u x 21 1 x 2 u 12 D 1 D 2 Case I: independent stocks without transshipment Equivalent to single warehouse with lost sales. Average cost per time step is which is convex in S i. λ(s i ) = E[m i (D i S i ) + ] + E[h i (S i D i ) + ] Marco Laumanns Distributed Control in Transportation Networks 19 May / 26

14 Basestock policies Basestock policy { S i x i (t) if x i S i u i (t) = 0 else S i : basestock level u 1 v 1 v 2 u 2 u x 21 1 x 2 u 12 D 1 D 2 Case II: with transshipment λ(s 1, S 2 ) = min E[h 1 x (s) 1 + h 2 x (s) 2 + m 1 v (s) 1 + m 2 v (s) 2 + c 12 u (s) 12 + c 21u (s) 21 ] s.t. x (s) 1 = S 1 d (s) 1 + v (s) 1 + u (s) 21 u(s) 12 s ( scenarios ) x (s) 2 = S 2 d (s) 2 + v (s) 2 + u (s) 12 u(s) 21 s x (s) 1, x (s) 2, v (s) 1, v (s) 2, u(s) 12, u(s) 21 0 s Marco Laumanns Distributed Control in Transportation Networks 19 May / 26

15 Some numerical results Data Example 1 Example 2 demand costs Solution d P[D i = d] h = 1 c = 2 m = h = 1 c = 2 m = 8 no transshipment S 1 = S 2 = 4 S 1 = S 2 = 3 λ = 4 λ = 4.8 with transshipm. S 1 = 4, S 2 = 3 S 1 = S 2 = 3 λ = 3.76 λ = 3.75 Marco Laumanns Distributed Control in Transportation Networks 19 May / 26

16 Basestock policies - general case central warehouse stocking points i u 1 v1 x 1 x 2 x 3... u 11 u ij u 22 u 33 customer locations j d 1 d 2 d 3 d 4 Case II: with transshipment, general case Problem is a two-stage stochastic LP Recourse function is min-cost flow problem - with S 1, S 2 as parameter Average cost λ(s 1,..., S n ) is convex. Marco Laumanns Distributed Control in Transportation Networks 19 May / 26

18 Lead times via state augmentation u 1 v 1 v 2 u 2 u x 21 1 x 2 u 12 D 1 D 2 Marco Laumanns Distributed Control in Transportation Networks 19 May / 26

19 Lead times via state augmentation u 1 u 2 x 1 x v 2 1 v 2 u x 21 1 x 2 u 12 D 1 D 2 Marco Laumanns Distributed Control in Transportation Networks 19 May / 26

20 Lead times via state augmentation u 1 u 2 x 1 x v 2 1 v 2 u x 21 1 x 2 u 12 D 1 D 2 Augmented dynamics, constraints, and costs x 1 (t + 1) = x 1 (t) + x 1 (t) D 1 (t) + u 21 (t) + v 1 (t) x 1 (t + 1) = u 1 (t) where u 21 (t) + v 1 (t) x 1 (t) and cost g 1 (x, u, v) = h 1 (x 1 + u 21 + v 1 ) + c 21 u 21 + m 1 v 1 Marco Laumanns Distributed Control in Transportation Networks 19 May / 26

21 Lead times via state augmentation u 1 u 2 x 1 x v 2 1 v 2 u x 21 1 x 2 u 12 What is the marginal value of an additional stock unit? D 1 D 2 Augmented dynamics, constraints, and costs x 1 (t + 1) = x 1 (t) + x 1 (t) D 1 (t) + u 21 (t) + v 1 (t) x 1 (t + 1) = u 1 (t) where u 21 (t) + v 1 (t) x 1 (t) and cost g 1 (x, u, v) = h 1 (x 1 + u 21 + v 1 ) + c 21 u 21 + m 1 v 1 Marco Laumanns Distributed Control in Transportation Networks 19 May / 26

22 Parametric dynamic programming Goal We would like to find the differential cost function d (x), which fulfills λ + d (x) = (Td )(x) := min u U(x) E [g(x, u, D) + d (f (x, u, D)] for all x. Marco Laumanns Distributed Control in Transportation Networks 19 May / 26

23 Parametric dynamic programming Goal We would like to find the differential cost function d (x), which fulfills λ + d (x) = (Td )(x) := min u U(x) E [g(x, u, D) + d (f (x, u, D)] for all x. If d is piecewise linear and convex, this property is preserved under the Bellman operator T in our case. Marco Laumanns Distributed Control in Transportation Networks 19 May / 26

24 Parametric dynamic programming Goal We would like to find the differential cost function d (x), which fulfills λ + d (x) = (Td )(x) := min u U(x) E [g(x, u, D) + d (f (x, u, D)] for all x. If d is piecewise linear and convex, this property is preserved under the Bellman operator T in our case. Jones, Baric, Morari: Multiparametric Linear Programming with Applications to Control, Diehl, Björnberg: Robust Dynamic Programming for Min-Max Model Predictive Control of Contrained Uncertain Systems, de la Pena, Bemporad, Filippi: Robust Explicit MPC Based on Approximate Multiparametric Convex Programming, Lincoln, Rantzer: Relaxing Dynamic Programming, Marco Laumanns Distributed Control in Transportation Networks 19 May / 26

25 Randomized relative value iteration RRVI 1 Initialize k := 0, set d 0 (x) : 0 and choose some ˆx 2 Evaluate Td k (ˆx) and add plane to set of planes V k+1 3 Sample N points x, for each x Evaluate Td k (x) and determine corresponding plane Add plane to V k+1 if not redundant 4 Set d k+1 (x) to maximum over planes 5 Set d k+1 (x) := d k+1 (x) d k+1 (ˆx) 6 Set k := k + 1 and repeat from 2. Marco Laumanns Distributed Control in Transportation Networks 19 May / 26

26 Randomized relative value iteration RRVI 1 Initialize k := 0, set d 0 (x) : 0 and choose some ˆx 2 Evaluate Td k (ˆx) and add plane to set of planes V k+1 3 Sample N points x, for each x Evaluate Td k (x) and determine corresponding plane Add plane to V k+1 if not redundant 4 Set d k+1 (x) to maximum over planes 5 Set d k+1 (x) := d k+1 (x) d k+1 (ˆx) 6 Set k := k + 1 and repeat from 2. Lower bound Every differential cost function d(x) yields a lower bound λ = min x Td(x) d(x) Marco Laumanns Distributed Control in Transportation Networks 19 May / 26

27 Some more numerical results Data Example 2 demand costs d P[D i = d] h = 1 c = 2 m = 8 Solution lead time 1 lead time 2 no transshipment S 1 = S 2 = 3 S 1 = S 2 = 5 λ = 4.8 λ = with transshipm. S 1 = S 2 = 3 S 1 = 4, S 2 = 5 basestock λ = 3.75 λ = 4.98 with transshipm. λ = RRVI λ = 4.95 Marco Laumanns Distributed Control in Transportation Networks 19 May / 26

28 Conclusions Considered inventory-distribution problem (no lead time) is easy for any number of stocking points and customers - Basestock-policies are optimal - Basestock levels easy to determinine Lead time (time delays) can make problem hard - Value function dependent on augmented state Approximation of differential cost function yields - a policy (MPC by value function approximation) - a lower bound (to evaluate the quality of heuristics) Marco Laumanns Distributed Control in Transportation Networks 19 May / 26

31 Problem Find exact route and speed profile for each train, at any time Marco Laumanns Distributed Control in Transportation Networks 19 May / 26

32 Problem and geographical subdivision Find exact route and speed profile for each train, at any time Marco Laumanns Distributed Control in Transportation Networks 19 May / 26

33 Macroscopic model Periodic Event Scheduling Problem Find passing times at critical locations under commercial requirements (connections) simplified dynamics (constant speed) and safety model (headway) Marco Laumanns Distributed Control in Transportation Networks 19 May / 26

34 Microscopic model 0 x! T r u 0 inbound train path shifted (inbound) train path u shifted (outbound) train path v! T: Inbound entrance time window r u 1! T: Outbound departure time window r u 2 r u 3 r u 4! T r v 0 r v 1 r v 2 r v 3 Junction / Switch area t Marco Laumanns Distributed Control in Transportation Networks 19 May / 26

36 Receding horizon control current practise Marco Laumanns Distributed Control in Transportation Networks 19 May / 26

37 MPC concept for station region Marco Laumanns Distributed Control in Transportation Networks 19 May / 26

38 MPC concept for station region 0 x! T r u 0! T r u 1 T r u 2 r u 3 r u 4! T r v 0 r v 1 r v 2 r v 3 t Marco Laumanns Distributed Control in Transportation Networks 19 May / 26

39 MPC concept for station region 0 x! T r u 0! T r u 1 T r u 2 r u 3 r u 4! T r v 0 r v 1 r v 2 r v 3 t Marco Laumanns Distributed Control in Transportation Networks 19 May / 26

40 MPC IP formulation Maximize X arrival time difference z } { x p f (A z A(p)) + X x p departure time difference z } { g(d z D(p)) p P (z) z Z p P (z) z Z + + X lz c i,z j yz c i,z j (z i,z j ) weakly connected X lz s i,z j yz s i,z j (z i,z j ) weakly sequenced X F z F (p) p P(z),z Z h(f z, F (p)) x p (connections kept) (sequences kept) (platform changes) subject to X p P m, m z M x p = 1, z Z Marco Laumanns Distributed Control in Transportation Networks 19 May / 26

41 Conclusions Offline train scheduling problem already intractable decomposition and simplification necessary MPC approach for online train control of a single station area Coordination an open problem, options: - local coordination beween neighboring nodes - global coordination via macroscopic layer Marco Laumanns Distributed Control in Transportation Networks 19 May / 26