Stochastic Gradient Method: Applications

Stochastic Gradient Method: Applications February 03, 2015 P. Carpentier Master MMMEF Cours MNOS 2014-2015 114 / 267

Lecture Outline 1 Two Elementary Exercices on the Stochastic Gradient Two-Stage Recourse Problem Trade-off Between Investment and Operation 2 Financial Problem Modeling Computing Efficiently the Price Two Algorithms 3 P. Carpentier Master MMMEF Cours MNOS 2014-2015 115 / 267

Two-Stage Recourse Problem Trade-off Between Investment and Operation 1 Two Elementary Exercices on the Stochastic Gradient Two-Stage Recourse Problem Trade-off Between Investment and Operation 2 Financial Problem Modeling Computing Efficiently the Price Two Algorithms 3 P. Carpentier Master MMMEF Cours MNOS 2014-2015 116 / 267

Two-Stage Recourse Problem Trade-off Between Investment and Operation 1 Two Elementary Exercices on the Stochastic Gradient Two-Stage Recourse Problem Trade-off Between Investment and Operation 2 Financial Problem Modeling Computing Efficiently the Price Two Algorithms 3 P. Carpentier Master MMMEF Cours MNOS 2014-2015 117 / 267

A Basic Two-Stage Recourse Problem Two-Stage Recourse Problem Trade-off Between Investment and Operation We consider the management of a water reservoir. Water is drawn from the reservoir by way of random consumers. In order to ensure the reservoir supply, 2 decisions are taken at successive time steps. A first supply decision q 1 is taken without any knowledge of the effective consumption, the associated cost being equal to 1 2 c 1( q1 ) 2, with c1 > 0. Once the consumption d has been observed (realization of a r.v. D defined over a probability space (Ω, A, P)), a second supply decision q 2 is taken in order to maintain the reservoir at its initial level, that is, q 2 = d q 1, the cost associated to this second decision being equal to 1 2 c 2( q2 ) 2, with c2 > c 1 > 0. The problem is to minimize the expected cost of operation. P. Carpentier Master MMMEF Cours MNOS 2014-2015 118 / 267

Mathematical Formulation and Solution Problem Formulation q 1 is a deterministic decision variable, Two-Stage Recourse Problem Trade-off Between Investment and Operation whereas q 2 is the realization of a random variable Q 2. 1 min (q 1,Q 2 ) 2 c ( ) 2 1 ( 1 q1 + 2 E ( ) ) 2 c 2 Q2 s.t. q 1 + Q 2 = D. Equivalent Problem 1 ( min q 1 R 2 E ( ) 2 ( ) ) 2 c 1 q1 + c2 D q1 Analytical solution: q 1 = c 2 c 1 + c 2 E ( D ). P. Carpentier Master MMMEF Cours MNOS 2014-2015 119 / 267

Stochastic Gradient Algorithm Two-Stage Recourse Problem Trade-off Between Investment and Operation Q (k+1) 1 = Q (k) 1 1 ( (c 1 + c 2 )Q (k) 1 c 2 D (k+1)). k Algorithm (initialization) // // Random generator // rand( normal ); rand( seed,123); // // Random consumption // moy = 10.; ect = 5.; // // Criterion // c1 = 3.; c2 = 1.; // // Initialization // x = [ ]; y = [ ]; Algorithm (iterations) // // Algorithm // qk = 0.; for k = 1:100 dk = moy + (ect*rand(1)); gk = ((c1+c2)*qk) - (c2*dk); ek = 1/k; qk = qk - (ek*gk); x = [x ; k]; y = [y ; qk]; end // // Trajectory plot // plot2d(x,y); xtitle( Stochastic Gradient, Iter., Q1 ); P. Carpentier Master MMMEF Cours MNOS 2014-2015 120 / 267

A Realization of the Algorithm Two-Stage Recourse Problem Trade-off Between Investment and Operation 5.0 Stochastic Gradient Algorithm 4.5 4.0 3.5 3.0 Q1 2.5 2.0 1.5 1.0 0.5 0.0 0 10 20 30 40 50 60 70 80 90 100 Iter. P. Carpentier Master MMMEF Cours MNOS 2014-2015 121 / 267

More Realizations... Two-Stage Recourse Problem Trade-off Between Investment and Operation 5.0 Stochastic Gradient Algorithm 4.5 4.0 3.5 3.0 Q1 2.5 2.0 1.5 1.0 0.5 0.0 0 50 100 150 200 250 300 350 400 450 500 Iter. P. Carpentier Master MMMEF Cours MNOS 2014-2015 122 / 267

Two-Stage Recourse Problem Trade-off Between Investment and Operation Slight Modification of the problem As in the basic two-stage recourse problem, a first supply decision q 1 is taken without any knowledge of the effective consumption, the associated cost being equal to 1 2 c 1( q1 ) 2, a second supply decision q 2 is taken once the consumption d has been observed (realization of a r.v. D), the cost of this second decision being equal to 1 2 c 2( q2 ) 2. The difference between supply and demand is penalized thanks to an additional cost term 1 2 c 3( q1 + q 2 d ) 2. The new problem is : 1 ( min (q 1,Q 2 ) 2 E ( ) 2 ( ) 2 ( c 1 q1 + c2 Q2 + c3 q1 + Q 2 D ) ) 2. Question: how to solve it using a stochastic gradient algorithm? P. Carpentier Master MMMEF Cours MNOS 2014-2015 123 / 267

Two-Stage Recourse Problem Trade-off Between Investment and Operation 1 Two Elementary Exercices on the Stochastic Gradient Two-Stage Recourse Problem Trade-off Between Investment and Operation 2 Financial Problem Modeling Computing Efficiently the Price Two Algorithms 3 P. Carpentier Master MMMEF Cours MNOS 2014-2015 124 / 267

Two-Stage Recourse Problem Trade-off Between Investment and Operation Trade-off Between Investment and Operation (1) A company owns N production units and has to meet a given (non stochastic) demand d. For each unit i, the decision maker first takes an investment decision u i R, the associated cost being I i (u i ). Then a discrete disturbance w i {w i,a, w i,b, w i,c } occurs. Knowing all noises, the decision maker selects for each unit i an operating point v i R, which leads to a cost c i (v i, w i ) and a production e i (v i, w i ). The goal is to minimize the overall expected cost, subject to the following constraints: investment limitation: Θ(u 1,..., u N ) 0, operating limitation: v i ϕ i (u i ), i = 1..., N, demand satisfaction: N i=1 e i(v i, w i ) d = 0. P. Carpentier Master MMMEF Cours MNOS 2014-2015 125 / 267

Two-Stage Recourse Problem Trade-off Between Investment and Operation Trade-off Between Investment and Operation (2) Questions. 1 Write down the optimization problem. Is it possible to apply the stochastic gradient algorithm in a straightforward manner? 2 Extract the optimization subproblem obtained when both the investment u = (u 1,..., u N ) and the noise w = (w 1,..., w N ) are fixed. The value of this subproblem is denoted f (u, w). Discuss the resolution of this subproblem. Give assumptions in order to have a nice function f. Compute the partial derivatives of f w.r.t. u. 3 Reformulate the initial optimization problem using this new function f and apply the stochastic gradient algorithm in the two following cases: the investment limitation reduces to u i [u i, u i ], i = 1,..., N, the investment limitation has the form Θ(u 1,..., u N ) 0. P. Carpentier Master MMMEF Cours MNOS 2014-2015 126 / 267

Financial Problem Modeling Computing Efficiently the Price Two Algorithms 1 Two Elementary Exercices on the Stochastic Gradient Two-Stage Recourse Problem Trade-off Between Investment and Operation 2 Financial Problem Modeling Computing Efficiently the Price Two Algorithms 3 P. Carpentier Master MMMEF Cours MNOS 2014-2015 127 / 267

Financial Problem Modeling Computing Efficiently the Price Two Algorithms 1 Two Elementary Exercices on the Stochastic Gradient Two-Stage Recourse Problem Trade-off Between Investment and Operation 2 Financial Problem Modeling Computing Efficiently the Price Two Algorithms 3 P. Carpentier Master MMMEF Cours MNOS 2014-2015 128 / 267

Financial Problem Modeling Computing Efficiently the Price Two Algorithms The Financial Problem The price of an option with payoff ψ(s t, 0 t T ) is given by ( P = E e rt ) ψ(s t, 0 t T ), where the dynamics of the underlying n-dimensional asset S is described by the following stochastic differential equation ds t = S t ( rdt + σ(t, St )dw t ), S0 = x, r being the interest rate and σ(t, y) being the volatility function. P. Carpentier Master MMMEF Cours MNOS 2014-2015 129 / 267

Discretization Financial Problem Modeling Computing Efficiently the Price Two Algorithms Most of the time, the exact solution is not available. To overcome the difficulty, one considers a discretized approximation of S (by using an Euler s scheme), so that the price P is approximated by ( P = E e rt ) ψ(ŝ t 1,..., Ŝ t d ). In such cases, the discretized function can be expressed in terms of the Brownian increments, or equivalently using a random normal vector. A compact form for the discretized price is P = E ( φ(g ) ), where G is a n d-dimensional Gaussian vector with identity covariance matrix and zero-mean. P. Carpentier Master MMMEF Cours MNOS 2014-2015 130 / 267

Financial Problem Modeling Computing Efficiently the Price Two Algorithms 1 Two Elementary Exercices on the Stochastic Gradient Two-Stage Recourse Problem Trade-off Between Investment and Operation 2 Financial Problem Modeling Computing Efficiently the Price Two Algorithms 3 P. Carpentier Master MMMEF Cours MNOS 2014-2015 131 / 267

A Parameterized Change of Variable Financial Problem Modeling Computing Efficiently the Price Two Algorithms The problem is now to compute P = E ( φ(g ) ) using Monte Carlo simulations. By a change of variables, we obtain that ) θ 2 θ,g P = E (φ(g + θ)e 2, for any θ R n d. Let us denote by V (θ) the associated variance: ( V (θ) = E φ(g + θ) 2 e 2 θ,g θ 2) ( 2 E φ(g )), ) ( = E (φ(g ) 2 θ 2 2 θ,g + e 2 E φ(g )). The last expression shows that function V is strictly convex and differentiable without any specific assumptions on φ. Moreover, V ) (θ) = E ((θ G )φ(g ) 2 θ 2 θ,g + e 2, so that the gradient of V does not involves any derivative of φ. P. Carpentier Master MMMEF Cours MNOS 2014-2015 132 / 267

Variance Minimization Financial Problem Modeling Computing Efficiently the Price Two Algorithms The goal is to compute the parameter θ such that the variance V (θ) related to P is as small as possible, in order to optimize the convergence speed of the Monte Carlo simulations: ( min E φ(g ) 2 e θ R d θ 2 θ,g + 2 ). This optimization problem is well suited for being solved by the stochastic gradient algorithm: θ (k+1) = θ (k) ɛ (k)( θ (k) G (k+1)) φ ( G (k+1)) 2 e θ (k),g (k+1) + θ(k) 2 2, and its unique solution is denoted θ. P. Carpentier Master MMMEF Cours MNOS 2014-2015 133 / 267

Financial Problem Modeling Computing Efficiently the Price Two Algorithms 1 Two Elementary Exercices on the Stochastic Gradient Two-Stage Recourse Problem Trade-off Between Investment and Operation 2 Financial Problem Modeling Computing Efficiently the Price Two Algorithms 3 P. Carpentier Master MMMEF Cours MNOS 2014-2015 134 / 267

Financial Problem Modeling Computing Efficiently the Price Two Algorithms Non adaptive Algorithm 1 Using a N-sample of G, obtain an approximation θ (N) of θ by N iterations of the stochastic gradient algorithm. 2 Once θ (N) has been obtained, use the standard Monte Carlo method to compute an approximation of the price P by using another N-sample of G : P (N) = 1 N N k=1 φ(g (N+k) + θ (N) )e θ(n),g (N+k) θ(n) 2 2. This algorithm requires 2N evaluations of the function φ, whereas a crude Monte Carlo method evaluates φ only N times. The non adaptive algorithm is efficient as soon as V (θ ) V (0)/2. P. Carpentier Master MMMEF Cours MNOS 2014-2015 135 / 267

Adaptive Algorithm Financial Problem Modeling Computing Efficiently the Price Two Algorithms Combine the 2 previous algorithms, and compute simultaneously approximations of θ and P by using the same N-sample of G : 10 θ (k+1) = θ (k) ɛ (k)( θ (k) G (k+1)) φ ( G (k+1)) 2 e θ (k),g (k+1) + θ(k) 2 2, P (k+1) = P (k) 1 (P (k) φ(g (k+1) + θ (k) ) e θ(k),g (k+1) θ(k) ) 2 2 k + 1 A Central Limit Theorem is available for this algorithm: ( (N) N P P) D N (0, V ) (θ ).. 10 the last relation being the recursive form of: P (N) = 1 N N k=1 φ(g (k+1) + θ (k) ) e θ(k),g (k+1) θ(k) 2 2. P. Carpentier Master MMMEF Cours MNOS 2014-2015 136 / 267

1 Two Elementary Exercices on the Stochastic Gradient Two-Stage Recourse Problem Trade-off Between Investment and Operation 2 Financial Problem Modeling Computing Efficiently the Price Two Algorithms 3 P. Carpentier Master MMMEF Cours MNOS 2014-2015 137 / 267

1 Two Elementary Exercices on the Stochastic Gradient Two-Stage Recourse Problem Trade-off Between Investment and Operation 2 Financial Problem Modeling Computing Efficiently the Price Two Algorithms 3 P. Carpentier Master MMMEF Cours MNOS 2014-2015 138 / 267

Mission to Mars P. Carpentier Master MMMEF Cours MNOS 2014-2015 139 / 267

Satellite Model dr dt = v, dm dt = T g 0 I sp δ. dv dt = µ r r 3 + F m κ, (8a) (8b) (8a): 6-dimensional state vector (position r and velocity v). (8b): 1-dimensional state vector (mass m including fuel). κ involves the direction cosines of the thrust and the on-off switch δ of the engine (3 controls), and µ, F, T, g 0, I sp are constants. The deterministic control problem is to drive the satellite from the initial condition at t i to a known final position r f and velocity v f at t f (given) while minimizing fuel consumption m(t i ) m(t f ). P. Carpentier Master MMMEF Cours MNOS 2014-2015 140 / 267

Deterministic Optimization Problem Using equinoctial coordinates for the position and velocity state vector x R 7, and cartesian coordinates for the thrust of the engine control vector u R 3, the deterministic optimization problem writes as follows: subject to: x(t i ) = x i, min K( x(t f ) ) u( ) x (t) = f ( x(t), u(t) ), u(t) 1 t [t i, t f ], C ( x(t f ) ) = 0. P. Carpentier Master MMMEF Cours MNOS 2014-2015 141 / 267

Engine Failure Sometimes, the engine may fail to work when needed: the satellite drifts away from the deterministic optimal trajectory. After the engine control is recovered, it is not always possible to drive the satellite to the final target at t f. By anticipating such possible failures and by modifying the trajectory followed before any such failure occurs, one may increase the possibility of eventually reaching the target. But such a deviation from the deterministic optimal trajectory results in a deterioration of the economic performance. The problem is thus to balance the increased probability of eventually reaching the target despite possible failures against the expected economic performance, that is, to quantify the price of safety one is ready to pay for. P. Carpentier Master MMMEF Cours MNOS 2014-2015 142 / 267

Stochastic Formulation (1) A failure is modeled using two random variables: t p : random initial time of the failure, t d : random duration of the failure. For any realization (tp, ξ t ξ d ) of a failure: 1 u( ) denotes the control used prior to the failure u is defined over [t i, t f ] but implemented over [t i, tp] ξ and corresponds to an open-loop control, 2 the control during the failure is 0 over [tp, ξ tp ξ + t ξ d ], 3 v ξ ( ) denotes the control used after the failure v ξ is defined over [tp ξ + t ξ d, t f] (if nonempty) and corresponds to a closed-loop strategy V. The satellite dynamics in the stochastic formulation writes: x ξ (t i ) = x i, x ξ (t) = f ξ( x ξ (t), u(t), v ξ (t) ). P. Carpentier Master MMMEF Cours MNOS 2014-2015 143 / 267

Stochastic Formulation (2) The problem is to minimize the expected cost (fuel consumption) w.r.t. the open-loop control u and the closed-loop strategy V, the probability to hit the target at t f being at least equal to p. ( min min E K ( x ξ (t f ) )) u( ) V( ) subject to: x ξ (t i ) = x i, ( min min E K ( x ξ (t f ) ) ( C x ξ (t f ) ) ) = 0 u( ) V( ) x ξ (t) = f ξ( x ξ (t), u(t), v ξ (t) ), u(t) 1 t [t i, t f ], v ξ (t) 1 t [tp ξ + t ξ d, t f], ( P C ( x ξ (t f ) ) ) = 0 p. P. Carpentier Master MMMEF Cours MNOS 2014-2015 144 / 267

1 Two Elementary Exercices on the Stochastic Gradient Two-Stage Recourse Problem Trade-off Between Investment and Operation 2 Financial Problem Modeling Computing Efficiently the Price Two Algorithms 3 P. Carpentier Master MMMEF Cours MNOS 2014-2015 145 / 267

Indicator Function Consider the real-valued indicator function: { 1 if y = 0, I(y) = 0 otherwise. Then ( P C ( x ξ (t f ) ) ) ( = 0 = E I ( C ( x ξ (t f ) ) )), and ( E K ( x ξ (t f ) ) ( C x ξ (t f ) ) ) E = 0 = (K ( x ξ (t f ) ) I ( C ( x ξ (t f ) ) )) E (I ( C ( x ξ (t f ) ) )). P. Carpentier Master MMMEF Cours MNOS 2014-2015 146 / 267

Problem Reformulation The problem is (shortly) reformulated as s.t. E min min u( ) V( ) (K ( x ξ (t f ) ) I ( C ( x ξ (t f ) ) )) E (I ( C ( x ξ (t f ) ) )) ( E I ( ( C x ξ (t f ) ) )) p. Such a formulation is however not well-suited for a numerical implementation (e.g. Arrow-Hurwicz algorithm), because a ratio of expectations is not an expectation! P. Carpentier Master MMMEF Cours MNOS 2014-2015 147 / 267

An Useful Lemma Using compact notation, the previous problem writes: min u J(u) Θ(u) in which J and Θ assume positive values. s.t. Θ(u) p, (9) 1 If u is a solution of (9) and if Θ(u ) = p, then u is also a solution of min u J(u) s.t. Θ(u) p. (10) 2 Conversely, if u is a solution of (10), and if an optimal Kuhn-Tucker multiplier β satisfies the condition then u is also a solution of (9). β J(u ) Θ(u ), P. Carpentier Master MMMEF Cours MNOS 2014-2015 148 / 267

Back to a Cost in Expectation Using the previous lemma, we aim at solving a problem in which the cost and the constraint functions correspond to expectations: ( min min E K ( x ξ (t f ) ) I ( C ( x ξ (t f ) ) )) u( ) V( ) ( s.t. E I ( ( C x ξ (t f ) ) )) p, or equivalently (Interchange Theorem [R&W, 1998]): s.t. ( min E min K ( x ξ (t f ) ) I ( ( C x ξ (t f ) ) )) u( ) v ξ ( ) ( E I ( C ( x ξ (t f ) ) )) p. P. Carpentier Master MMMEF Cours MNOS 2014-2015 149 / 267

1 Two Elementary Exercices on the Stochastic Gradient Two-Stage Recourse Problem Trade-off Between Investment and Operation 2 Financial Problem Modeling Computing Efficiently the Price Two Algorithms 3 P. Carpentier Master MMMEF Cours MNOS 2014-2015 150 / 267

Lagrangian Formulation s.t. ( min E min K ( x ξ (t f ) ) I ( ( C x ξ (t f ) ) )) u( ) v ξ ( ) ( p E I ( C ( x ξ (t f ) ) )) 0 µ Assume there exists a saddle point for the associated Lagrangian. In order to solve { ( ( ( max min µ p + E min K x ξ (t f ) ) µ ) I ( C ( x ξ (t f ) ) ) ) }. µ 0 u( ) v ξ ( ) } {{ } W (u, µ, ξ) that is, { max min µ p + E ( W (u, µ, ξ) )}, µ 0 u( ) we use an adapted Arrow-Hurwicz algorithm ([Culioli, 1994]). P. Carpentier Master MMMEF Cours MNOS 2014-2015 151 / 267

Algorithm Overview Arrow-Hurwicz algorithm At iteration k, 1 draw a failure ξ k = (tp ξk, t ξk ) according to its probability law, 2 compute the gradient of W w.r.t. u and update u( ): ) u k+1 = Π B (u k ε k u W (u k, µ k, ξ k ), d 3 compute the gradient of W w.r.t. µ and update µ: ( µ k+1 = max 0, µ k + ρ k( p + µ W (u k+1, µ k, ξ k ) )). P. Carpentier Master MMMEF Cours MNOS 2014-2015 152 / 267

First Difficulty: I Is Not a Smooth Function At every iteration k, we must evaluate function W as well as its derivatives w.r.t. u(.) and µ. But W is not differentiable! { ( ) 1 if y = 0, 1 y 2 2 if y [ r, r], I(y) = I r (y) = r 2 0 otherwise, 0 otherwise. 1.0 1.0 0.8 0.8 0.6 0.6 0.4 0.4 0.2 0.2 There are specific rules to drive r to 0 as the iteration number k goes to infinity in order to obtain the best asymptotic Mean Quadratic Error of the gradient estimates ([Andrieu et al., 2007]). P. Carpentier Master MMMEF Cours MNOS 2014-2015 153 / 267

Second Difficulty: Solving the Inner Problem The approximated closed-loop problem to solve at each iteration is: { (K W r k (u k, ξ k, µ k ( ) = min x ξ (t f ) ) µ k) ( I r v ξ k C ( x ξ (t f ) ) )}. ( ) In this setting, we have to check if the target is reached up to r k. Different cases have to be considered: 1 the target can be reached accurately, 2 the target can be reached up to r k only, 3 the target cannot be reached up to r k. Note that if reaching the target is possible but too expensive (that is, if K ( x ξ (t f ) ) µ k ), the best thing to do is to stop the engine! In practice, the solution of the approximated problem is derived from the resolution of two standard optimal control problems... P. Carpentier Master MMMEF Cours MNOS 2014-2015 154 / 267

Parameters Tuning Gradient step length: ε k = a b + k, ρ k = c d + k, usual for a stochastic gradient algorithm. Smoothing parameter: r k = α β + k 1 3, MQE reduced by a factor 1000 in about 100.000 iterations. P. Carpentier Master MMMEF Cours MNOS 2014-2015 155 / 267

1 Two Elementary Exercices on the Stochastic Gradient Two-Stage Recourse Problem Trade-off Between Investment and Operation 2 Financial Problem Modeling Computing Efficiently the Price Two Algorithms 3 P. Carpentier Master MMMEF Cours MNOS 2014-2015 156 / 267

Example: Interplanetary Mission t i = 0.70 and t f = 8.70 (normalized units), t p : exponential distribution: P ( t p t f ) 0.58 = πf, t d : exponential distribution: P ( 0.035 t d 0.125 ) 0.80. 1.0 0.8 0.6 The deterministic optimal control has a bang off bang shape. 0.4 0.2 0.0 0.2 Along the optimal trajectory, the probability to recover a failure is: p det 0.94. 0.4 0 1 2 3 4 5 6 7 8 9 Comp. normale w Comp. tangentielle s Comp. radiale q P. Carpentier Master MMMEF Cours MNOS 2014-2015 157 / 267

0.681 0.680 0.679 0.678 0.677 0.676 0.675 0.674 0 10000 Masse 20000 finale / iterations 30000 40000 50000 60000 70000 80000 90000 100000 0.326 0.324 0.322 0.320 1.0 0.8 0.6 0.4 0.2 0.0 0.2 0.318 0.316 0.314 0 10000 Multiplicateur 20000 probabilite 30000 / iterations 40000 50000 60000 70000 80000 90000 100000 0.4 0 1 2 3 4 5 6 7 8 9 Comp. normale w Comp. tangentielle s Comp. radiale q Figure: Probability level p = 0.750 P. Carpentier Master MMMEF Cours MNOS 2014-2015 158 / 267

0.681 0.680 0.679 0.678 0.677 0.676 0.675 0.674 0.673 0 10000 Masse 20000 finale / iterations 30000 40000 50000 60000 70000 80000 90000 100000 0.332 0.330 0.328 0.326 0.324 0.322 0.320 0.318 0 10000 Multiplicateur 20000 probabilite 30000 / iterations 40000 50000 60000 70000 80000 90000 100000 1.0 0.8 0.6 0.4 0.2 0.0 0.2 0.4 0 1 2 3 4 5 6 7 8 9 Comp. normale w Comp. tangentielle s Comp. radiale q Figure: Probability level p = 0.960 P. Carpentier Master MMMEF Cours MNOS 2014-2015 159 / 267

0.681 0.680 0.679 0.678 0.677 0.676 0.675 0.674 0 10000 Masse 20000 finale / iterations 30000 40000 50000 60000 70000 80000 90000 100000 0.365 0.360 0.355 0.350 0.345 0.340 0.335 0.330 0.325 0.320 0 10000 Multiplicateur 20000 probabilite 30000 / iterations 40000 50000 60000 70000 80000 90000 100000 1.0 0.8 0.6 0.4 0.2 0.0 0.2 0.4 0 1 2 3 4 5 6 7 8 9 Comp. normale w Comp. tangentielle s Comp. radiale q Figure: Probability level p = 0.990 P. Carpentier Master MMMEF Cours MNOS 2014-2015 160 / 267

The Price of Safety... 0.3222 0.3220 0.3218 0.3216 0.3214 0.3212 0.3210 0.3208 0.3206 0.3204 0.85 0.90 0.95 1.00 Consommation sans panne / Probabilite P. Carpentier Master MMMEF Cours MNOS 2014-2015 161 / 267

H. Robbins and S. Monro. A stochastic approximation method. Annals of Mathematical Statistics, 22:400 407, 1951. B. T. Polyak and A. B. Juditsky. Acceleration of stochastic approximation by averaging. SIAM Journal on Control and Optimization, 30(4):838 855, 1992. M. Duflo. Algorithmes stochastiques. Springer Verlag, 1996. H. J. Kushner and D. S. Clark. Stochastic Approximation Methods for Constrained and Unconstrained Systems. Springer Verlag, 1978. J.-C. Culioli and G. Cohen. Decomposition-coordination algorithms in stochastic optimization. SIAM Journal on Control and Optimization, 28(6):1372 1403, 1990. J.-C. Culioli and G. Cohen. Optimisation stochastique sous contraintes en espérance. CRAS, t.320, Série I, pp. 75-758, 1995. Laetitia Andrieu, Guy Cohen and Felisa J. Vázquez-Abad. Gradient-based simulation optimization under probability constraints. European Journal of Operational Research, 212, 345-351, 2011. R. T. Rockafellar and R. J.-B. Wets. Variational Analysis, volume 317 of A series of Comprehensive Studies in Mathematics. Springer Verlag, 1998. P. Carpentier Master MMMEF Cours MNOS 2014-2015 161 / 267