Optimal Control. Palle Andersen. Aalborg University. Opt lecture 1 p. 1/2
|
|
|
- Stuart Richard
- 7 years ago
- Views:
Transcription
1 Opt lecture 1 p. 1/2 Optimal Control Palle Andersen pa@control.aau.dk Aalborg University
2 Opt lecture 1 p. 2/2 Optimal Control, course outline 1st lecture: Introduction to optimal control and quadratic cost functions. State space models and state feedback control. Dynamic programming.
3 Opt lecture 1 p. 2/2 Optimal Control, course outline 1st lecture: Introduction to optimal control and quadratic cost functions. State space models and state feedback control. Dynamic programming. 2nd lecture: Stationary Linear Quadratic control. Use models of references and disturbances.
4 Opt lecture 1 p. 2/2 Optimal Control, course outline 1st lecture: Introduction to optimal control and quadratic cost functions. State space models and state feedback control. Dynamic programming. 2nd lecture: Stationary Linear Quadratic control. Use models of references and disturbances. 3rd lecture: Optimal control including reference and disturbance. Deterministic and stochastic disturbance models. Choice of disturbance models and performance function to reject steady state errors. Integral action.
5 Opt lecture 1 p. 2/2 Optimal Control, course outline 1st lecture: Introduction to optimal control and quadratic cost functions. State space models and state feedback control. Dynamic programming. 2nd lecture: Stationary Linear Quadratic control. Use models of references and disturbances. 3rd lecture: Optimal control including reference and disturbance. Deterministic and stochastic disturbance models. Choice of disturbance models and performance function to reject steady state errors. Integral action. 4th lecture: State space models with stochastic disturbances and stochastic measurement noise. Optimal state estimation using Kalman observer. Optimal control with output feedback using observer. Separation results. Linear Quadratic Gaussian (LQG) Control. Integral action with observer
6 Opt lecture 1 p. 2/2 Optimal Control, course outline 1st lecture: Introduction to optimal control and quadratic cost functions. State space models and state feedback control. Dynamic programming. 2nd lecture: Stationary Linear Quadratic control. Use models of references and disturbances. 3rd lecture: Optimal control including reference and disturbance. Deterministic and stochastic disturbance models. Choice of disturbance models and performance function to reject steady state errors. Integral action. 4th lecture: State space models with stochastic disturbances and stochastic measurement noise. Optimal state estimation using Kalman observer. Optimal control with output feedback using observer. Separation results. Linear Quadratic Gaussian (LQG) Control. Integral action with observer 5th lecture: Stability properties of controllers with optimal state feedback and observer based controllers. Robustness with simple uncertainty models.
7 Opt lecture 1 p. 3/2 Example: Tracking antenna Multiple inputs: Voltages u 1, u 2 and u 3. Multiple outputs: Pointing angles δ 1 and δ 2. State variables: Angles and angular velocities
8 Opt lecture 1 p. 4/2 Classic Methods are Insufficient With methods from SISO-systems we would control each output with a single input. An extra input possibility would need its specific goal. Cross couplings may cause disturbance in one output to influence other outputs.
9 Opt lecture 1 p. 4/2 Classic Methods are Insufficient With methods from SISO-systems we would control each output with a single input. An extra input possibility would need its specific goal. Cross couplings may cause disturbance in one output to influence other outputs. In SISO-systems pole placement giving reasonable output and control signal is difficult.
10 Opt lecture 1 p. 4/2 Classic Methods are Insufficient With methods from SISO-systems we would control each output with a single input. An extra input possibility would need its specific goal. Cross couplings may cause disturbance in one output to influence other outputs. In SISO-systems pole placement giving reasonable output and control signal is difficult. In MIMO-systems pole placement can not determine all state feedback coefficients.
11 Opt lecture 1 p. 5/2 Mimo pole placement is difficult Suppose a linear model is adequate x(k) = u(k) = [ x(k + 1) = Φx(k) + Γu(k) y(k) = Hx(k) u 1 (k) u 2 (k) u 3 (k) y(k) = [ δ 1 (k) δ 2 (k) ω 1 (k) θ 1 (k) ω 2 (k) θ 2 (k) ω 3 (k) θ 3 (k) ] ] T
12 Opt lecture 1 p. 6/2 Mimo pole placement is difficult Specification of 6 closed loop poles is not enough to determine 3 x 6 controller coefficients u(k) = Lx(k) u(k) = [u 1 (k) u 2 (k) u 3 (k)] T x(k) = [ω 1 (k) θ 1 (k) ω 2 (k) θ 2 (k) ω 3 (k) θ 3 (k)] T L = l 11 l 12 l 13 l 14 l 15 l 16 l 21 l 22 l 23 l 24 l 25 l 26 l 31 l 32 l 33 l 34 l 35 l 36
13 Opt lecture 1 p. 7/2 What can Optimal Control offer? Optimal control is born multi variable
14 Opt lecture 1 p. 7/2 What can Optimal Control offer? Optimal control is born multi variable Weighting of control cost vs quality directly in the performance function
15 Opt lecture 1 p. 7/2 What can Optimal Control offer? Optimal control is born multi variable Weighting of control cost vs quality directly in the performance function Optimal control can handle disturbances and references. Measurements of disturbances may be used in the controller as feed forward signals.
16 Opt lecture 1 p. 7/2 What can Optimal Control offer? Optimal control is born multi variable Weighting of control cost vs quality directly in the performance function Optimal control can handle disturbances and references. Measurements of disturbances may be used in the controller as feed forward signals. Stochastic properties of disturbances and measurement noise can be used
17 Opt lecture 1 p. 7/2 What can Optimal Control offer? Optimal control is born multi variable Weighting of control cost vs quality directly in the performance function Optimal control can handle disturbances and references. Measurements of disturbances may be used in the controller as feed forward signals. Stochastic properties of disturbances and measurement noise can be used Optimal control has been formulated in continuous as well as discrete time for linear systems
18 Opt lecture 1 p. 8/2 The performance function I = N k=0 H(x(k),u(k))
19 Opt lecture 1 p. 8/2 The performance function I = N H(x(k),u(k)) k=0 We will only consider H functions which are quadratic in u(k) and x(k). H(x(k),u(k)) = x T (k)q 1 x(k) + u T (k)q 2 u(k)
20 Opt lecture 1 p. 8/2 The performance function I = N H(x(k),u(k)) k=0 We will only consider H functions which are quadratic in u(k) and x(k). H(x(k),u(k)) = x T (k)q 1 x(k) + u T (k)q 2 u(k) Weighting between x(k) and u(k) determines how you want to weight good control with small states vs good economy with small control effort.
21 Opt lecture 1 p. 8/2 The performance function I = N H(x(k),u(k)) k=0 We will only consider H functions which are quadratic in u(k) and x(k). H(x(k),u(k)) = x T (k)q 1 x(k) + u T (k)q 2 u(k) Weighting between x(k) and u(k) determines how you want to weight good control with small states vs good economy with small control effort. The time horizon N will determine the weighting of the long term steady state performance (large N) versus short term dynamic performance (small N).
22 Opt lecture 1 p. 9/2 Block diagram of control problem r + + u d 1 Statespace system x d 2 y use of performance func Figure 1: Practical relevant closed loop problem.
23 Opt lecture 1 p. 10/2 Basic problem and extensions Basic problem: Find u(k), k = 0, 1, 2,,N, to minimize I given x(0).
24 Opt lecture 1 p. 10/2 Basic problem and extensions Basic problem: Find u(k), k = 0, 1, 2,,N, to minimize I given x(0). Introduce reference: r(k) different from 0
25 Opt lecture 1 p. 10/2 Basic problem and extensions Basic problem: Find u(k), k = 0, 1, 2,,N, to minimize I given x(0). Introduce reference: r(k) different from 0 Deterministic disturbances d 1 and d 2 are introduced
26 Opt lecture 1 p. 10/2 Basic problem and extensions Basic problem: Find u(k), k = 0, 1, 2,,N, to minimize I given x(0). Introduce reference: r(k) different from 0 Deterministic disturbances d 1 and d 2 are introduced Stochastic disturbances d 1 and noise d 2 are considered
27 Opt lecture 1 p. 11/2 Dynamic Programming Determine controller by breaking the complex optimization into simpler problems A control strategy which is optimal in the interval [0;N] must also be optimal in a part of the interval [k;n]
28 Opt lecture 1 p. 11/2 Dynamic Programming Determine controller by breaking the complex optimization into simpler problems A control strategy which is optimal in the interval [0;N] must also be optimal in a part of the interval [k;n] Proof: if it was possible to improve the performance in the interval [k;n] this would also improve the performance in the entire interval [0;N]
29 Opt lecture 1 p. 11/2 Dynamic Programming Determine controller by breaking the complex optimization into simpler problems A control strategy which is optimal in the interval [0;N] must also be optimal in a part of the interval [k;n] Proof: if it was possible to improve the performance in the interval [k;n] this would also improve the performance in the entire interval [0;N] If we know the optimal performance Jk+1 N (x(k + 1)) in dependence of x(k + 1) we will be able to calculate an optimal u(k) depending of x(k) to minimize Jk N(x(k))
30 Opt lecture 1 p. 11/2 Dynamic Programming Determine controller by breaking the complex optimization into simpler problems A control strategy which is optimal in the interval [0;N] must also be optimal in a part of the interval [k;n] Proof: if it was possible to improve the performance in the interval [k;n] this would also improve the performance in the entire interval [0;N] If we know the optimal performance Jk+1 N (x(k + 1)) in dependence of x(k + 1) we will be able to calculate an optimal u(k) depending of x(k) to minimize Jk N(x(k)) Start from behind: Find the solution on the interval [N 1;N] and find the optimal input sequence stepwise backwards
31 Opt lecture 1 p. 12/2 Dynamic Programming Notation I N 0 = N k=0 H(x(k),u(k)) = I0 N x(0),u(0),u(1)...u(n) x(1) is determined from x(0) and u(0) etc.
32 Opt lecture 1 p. 12/2 Dynamic Programming Notation I N 0 = N k=0 H(x(k),u(k)) = I0 N x(0),u(0),u(1)...u(n) x(1) is determined from x(0) and u(0) etc. Optimal sequence J N 0 (x(0)) = min u(0)...u(n) I N 0 x(0),u(0),u(1)...u(n)
33 Opt lecture 1 p. 13/2 Dynamic Programming Optimal sequence J N 0 (x(0)) = min u(0)...u(n) I N 0 x(0),u(0)...u(n)
34 Opt lecture 1 p. 13/2 Dynamic Programming Optimal sequence J N 0 (x(0)) = min u(0)...u(n) I N 0 x(0),u(0)...u(n) We may divide the interval in two parts and determine optimum for last part of the interval first J N k (x(k)) = min u(k)...u(n) IN k = min u(k) [H(x(k),u(k)) + x(k),u(k),u(k + 1)...u(N) min u(k+1)...u(n) N i=k+1 H(x(i),u(i))] = min [H(x(k),u(k)) + J k+1 N (x(k + 1))] u(k) where x(k + 1) = G(x(k),u(k))
35 Opt lecture 1 p. 14/2 Final algorithm STEP 0: J N N = H(x(N),0), (Note u(n) = 0 is optimal)
36 Opt lecture 1 p. 14/2 Final algorithm STEP 0: JN N STEP 1: Find minimizing control u(n 1) = u (N 1) = H(x(N),0), (Note u(n) = 0 is optimal) J N N 1 = min u(n 1) [H(x(N 1),u(N 1)) + J N N (x(n))] Note that JN N(x(N)) = J N N known from step 1 (G(x(N 1),u(N 1))) is a
37 Opt lecture 1 p. 14/2 Final algorithm STEP 0: JN N STEP 1: Find minimizing control u(n 1) = u (N 1) = H(x(N),0), (Note u(n) = 0 is optimal) J N N 1 = min u(n 1) [H(x(N 1),u(N 1)) + J N N (x(n))] Note that JN N(x(N)) = J N N known from step 1 (G(x(N 1),u(N 1))) is a STEP i: Find minimizing control signal u (N i) J N N i = min u(n i)[h(x(n i),u(n i))+j N N i+1 (x(n i+1))]
38 Opt lecture 1 p. 14/2 Final algorithm STEP 0: JN N STEP 1: Find minimizing control u(n 1) = u (N 1) = H(x(N),0), (Note u(n) = 0 is optimal) J N N 1 = min u(n 1) [H(x(N 1),u(N 1)) + J N N (x(n))] Note that JN N(x(N)) = J N N known from step 1 (G(x(N 1),u(N 1))) is a STEP i: Find minimizing control signal u (N i) J N N i = min u(n i)[h(x(n i),u(n i))+j N N i+1 (x(n i+1))] STEP N: Find minimizing control signal u (0) J N 0 = min u(0) [H(x(0),u(0)) + J N 1 (x(1))]
39 Opt lecture 1 p. 14/2 Final algorithm STEP 0: JN N STEP 1: Find minimizing control u(n 1) = u (N 1) = H(x(N),0), (Note u(n) = 0 is optimal) J N N 1 = min u(n 1) [H(x(N 1),u(N 1)) + J N N (x(n))] Note that JN N(x(N)) = J N N known from step 1 (G(x(N 1),u(N 1))) is a STEP i: Find minimizing control signal u (N i) J N N i = min u(n i)[h(x(n i),u(n i))+j N N i+1 (x(n i+1))] STEP N: Find minimizing control signal u (0) J N 0 = min u(0) [H(x(0),u(0)) + J N 1 (x(1))]
40 Opt lecture 1 p. 15/2 1st order example System equation: x(k + 1) = ax(k) + bu(k) Performance: I = Σ N k=0 (x2 (k) + qu 2 (k)) Horizon: Step 0: N = 2 J 2 2 (x(2)) = x 2 (2)
41 Opt lecture 1 p. 16/2 1st order example Step 1: use x(2) = ax(1) + bu(1) and minimize J1 2 (x(1)) = min u(1) [x 2 (1) + qu 2 (1) + (ax(1) + bu(1)) 2 ) (1) dj 2 1 (x(1)) du(1) = 2qu(1) + 2(ax(1) + bu(1))b = 0 (2) u (1) = J 2 1 (x(1)) = (1 + q ab q + b2x(1) (3) a 2 q + b 2)x2 (1) (4)
42 Opt lecture 1 p. 17/2 1st order example Step 2: use x(1) = ax(0) + bu(0) and minimize J 2 0 (x(0)) = min u(0) [x 2 (0)+qu 2 (0)+(1+q a 2 q + b 2)(ax(0)+bu(0))2 ] (5) u (0) = ab(1 + q a 2 q+b ) 2 x(0) q + b 2 (6) (1 + q q+b a2 ) 2 J 2 0 (x(0)) = (1 + q a 2 (1 + q q+b a2 ) 2 q + b 2 (1 + q q+b a2 ) )x2 (0) (7) 2
43 Note that you may calculate (u (0),u (1)) in advance from x(0) as open loop control or calculate u (0) from x(0) and u (1) from x(1) in a feedback manner. Opt lecture 1 p. 18/2 1st order example Now this calculation has given us the optimal control sequence u (0) = ab(1 + q a 2 q+b ) 2 q + b 2 (1 + q a2 u (1) = u (2) = 0 q+b 2 ) x(0) = l(0)x(0) (8) ab q + b2x(1) = l(1)x(1) (9) = ab q + b 2(a b ab(1 + q a 2 q + b 2 (1 + q a2 q+b 2 ) q+b 2 ) )x(0) (10) (11)
44 Opt lecture 1 p. 19/2 General state space system I = N 1 k=0 x(k + 1) = Φx(k) + Γu(k) (x T (k)q 1 x(k) + u T (k)q 2 u(k)) + x T (N)Q N x(n) The matrices Q N, Q 1 og Q 2 are quadratic and have the dimensions (n n), (n n) and (p p). We will suppose Q 1 0 Q 2 > 0
45 Opt lecture 1 p. 20/2 General state space system We will suppose And find J N k+1 (x(k + 1)) = xt (k + 1)S(k + 1)x(k + 1) J N k (x(k)) = min u(k) [xt (k)q 1 x(k) + u T (k)q 2 u(k) + x T (k + 1)S(k + 1)x(k + 1)] = min u(k) [xt (k)q 1 x(k) + u T (k)q 2 u(k) + (Φx(k) + Γu(k)) T S(k + 1)(Φx(k) + Γu(k))]
46 Opt lecture 1 p. 21/2 General state space system In minimum the derivative with respect to u(k) will be zero J N k (x) u = 2Q 2 u + 2Γ T S(k + 1)(Φx + Γu) = 0 Giving the optimal controlsignal u (k) = [Q 2 +Γ T S(k +1)Γ] 1 Γ T S(k +1)Φx(k) = L(k)x(k) with L(k) = [Q 2 + Γ T S(k + 1)Γ] 1 Γ T S(k + 1)Φ
47 Opt lecture 1 p. 22/2 General state space system We will now insert u = Lx in the expression for Jk N(x) and obtain: Jk N (x) = xt Q 1 x + ( Lx) T Q 2 ( Lx) + (Φx ΓLx) T S(k + 1)(Φ = x T [Q 1 + L T Q 2 L + (Φ ΓL) T S(k + 1)(Φ ΓL)]x = x T S(k)x with S(k) = Q 1 +L T (k)q 2 L(k)+(Φ ΓL(k)) T S(k +1)(Φ ΓL(k))
48 Opt lecture 1 p. 23/2 Summary of LQ for DT systems Linear discrete time, dynamical system x(k + 1) = Φx(k) + Γu(k) quadratic performance function I = N 1 (x T (k)q 1 x(k) + u T (k)q 2 u(k)) + x T (N)Q N x(n) k=0 Q 1 and Q N positive semidefinite and Q 2 is positive definite. Optimal input sequence: u (k) = L(k)x(k)
49 Opt lecture 1 p. 24/2 Summary of LQ for DT systems and L(k) = [Q 2 + Γ T S(k + 1)Γ] 1 Γ T S(k + 1)Φ S(k) = Q 1 + L T (k)q 2 L(k) + (Φ ΓL(k)) T S(k + 1)(Φ ΓL(k)) = Q 1 + Φ T S(k + 1)Φ Φ T S(k + 1)Γ[Q 2 + Γ T S(k + 1)Γ] = Q 1 + Φ T S(k + 1)[Φ ΓL(k)] with S(N) = Q N
50 Opt lecture 1 p. 25/2 Time varying vs Receding horizon In rare problems the interval [0;N] is the life time of the control sequence where a true time varying feedback u(k) = L(k)x(k) is used. Steering of start or landing of space vehicle. Batch process. More frequent control with constant feedback u(k) = L(0)x(k) is used. This is called receding horizon, where you push the time horizon of the performance in front of you. This is also the scheme of predictive control.
51 Opt lecture 1 p. 26/2 Calculation of optimal controller 1. k := N 2. S(k) = Q N 3. REPEAT k := k 1 L(k) = [Q 2 + Γ T S(k + 1)Γ] 1 Γ T S(k + 1)Φ S(k) = Q 1 + Φ T S(k + 1)[Φ ΓL(k)] UNTIL k = 0 4. Use the linear feedback u(k) = L(0)x(k) 5. The obtained performance function will be J0 N(x(0)) = xt (0)S(0)x(0)
C21 Model Predictive Control
C21 Model Predictive Control Mark Cannon 4 lectures Hilary Term 216-1 Lecture 1 Introduction 1-2 Organisation 4 lectures: week 3 week 4 { Monday 1-11 am LR5 Thursday 1-11 am LR5 { Monday 1-11 am LR5 Thursday
19 LINEAR QUADRATIC REGULATOR
19 LINEAR QUADRATIC REGULATOR 19.1 Introduction The simple form of loopshaping in scalar systems does not extend directly to multivariable (MIMO) plants, which are characterized by transfer matrices instead
A Multi-Model Filter for Mobile Terminal Location Tracking
A Multi-Model Filter for Mobile Terminal Location Tracking M. McGuire, K.N. Plataniotis The Edward S. Rogers Sr. Department of Electrical and Computer Engineering, University of Toronto, 1 King s College
Nonlinear Systems and Control Lecture # 15 Positive Real Transfer Functions & Connection with Lyapunov Stability. p. 1/?
Nonlinear Systems and Control Lecture # 15 Positive Real Transfer Functions & Connection with Lyapunov Stability p. 1/? p. 2/? Definition: A p p proper rational transfer function matrix G(s) is positive
Formulations of Model Predictive Control. Dipartimento di Elettronica e Informazione
Formulations of Model Predictive Control Riccardo Scattolini Riccardo Scattolini Dipartimento di Elettronica e Informazione Impulse and step response models 2 At the beginning of the 80, the early formulations
Sensorless Control of a Brushless DC motor using an Extended Kalman estimator.
Sensorless Control of a Brushless DC motor using an Extended Kalman estimator. Paul Kettle, Aengus Murray & Finbarr Moynihan. Analog Devices, Motion Control Group Wilmington, MA 1887,USA. Paul.Kettle@analog.com
POTENTIAL OF STATE-FEEDBACK CONTROL FOR MACHINE TOOLS DRIVES
POTENTIAL OF STATE-FEEDBACK CONTROL FOR MACHINE TOOLS DRIVES L. Novotny 1, P. Strakos 1, J. Vesely 1, A. Dietmair 2 1 Research Center of Manufacturing Technology, CTU in Prague, Czech Republic 2 SW, Universität
11. Time series and dynamic linear models
11. Time series and dynamic linear models Objective To introduce the Bayesian approach to the modeling and forecasting of time series. Recommended reading West, M. and Harrison, J. (1997). models, (2 nd
Chapter 12 Modal Decomposition of State-Space Models 12.1 Introduction The solutions obtained in previous chapters, whether in time domain or transfor
Lectures on Dynamic Systems and Control Mohammed Dahleh Munther A. Dahleh George Verghese Department of Electrical Engineering and Computer Science Massachuasetts Institute of Technology 1 1 c Chapter
Understanding and Applying Kalman Filtering
Understanding and Applying Kalman Filtering Lindsay Kleeman Department of Electrical and Computer Systems Engineering Monash University, Clayton 1 Introduction Objectives: 1. Provide a basic understanding
Discrete mechanics, optimal control and formation flying spacecraft
Discrete mechanics, optimal control and formation flying spacecraft Oliver Junge Center for Mathematics Munich University of Technology joint work with Jerrold E. Marsden and Sina Ober-Blöbaum partially
Applications to Data Smoothing and Image Processing I
Applications to Data Smoothing and Image Processing I MA 348 Kurt Bryan Signals and Images Let t denote time and consider a signal a(t) on some time interval, say t. We ll assume that the signal a(t) is
Controller Design in Frequency Domain
ECSE 4440 Control System Engineering Fall 2001 Project 3 Controller Design in Frequency Domain TA 1. Abstract 2. Introduction 3. Controller design in Frequency domain 4. Experiment 5. Colclusion 1. Abstract
Enhancing the SNR of the Fiber Optic Rotation Sensor using the LMS Algorithm
1 Enhancing the SNR of the Fiber Optic Rotation Sensor using the LMS Algorithm Hani Mehrpouyan, Student Member, IEEE, Department of Electrical and Computer Engineering Queen s University, Kingston, Ontario,
1.5. Factorisation. Introduction. Prerequisites. Learning Outcomes. Learning Style
Factorisation 1.5 Introduction In Block 4 we showed the way in which brackets were removed from algebraic expressions. Factorisation, which can be considered as the reverse of this process, is dealt with
Direct Methods for Solving Linear Systems. Matrix Factorization
Direct Methods for Solving Linear Systems Matrix Factorization Numerical Analysis (9th Edition) R L Burden & J D Faires Beamer Presentation Slides prepared by John Carroll Dublin City University c 2011
How To Solve A Sequential Mca Problem
Monte Carlo-based statistical methods (MASM11/FMS091) Jimmy Olsson Centre for Mathematical Sciences Lund University, Sweden Lecture 6 Sequential Monte Carlo methods II February 3, 2012 Changes in HA1 Problem
Reinforcement Learning
Reinforcement Learning LU 2 - Markov Decision Problems and Dynamic Programming Dr. Martin Lauer AG Maschinelles Lernen und Natürlichsprachliche Systeme Albert-Ludwigs-Universität Freiburg martin.lauer@kit.edu
Interactive applications to explore the parametric space of multivariable controllers
Milano (Italy) August 28 - September 2, 211 Interactive applications to explore the parametric space of multivariable controllers Yves Piguet Roland Longchamp Calerga Sàrl, Av. de la Chablière 35, 14 Lausanne,
TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAAA
2015 School of Information Technology and Electrical Engineering at the University of Queensland TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAAA Schedule Week Date
Stochastic control of HVAC systems: a learning-based approach. Damiano Varagnolo
Stochastic control of HVAC systems: a learning-based approach Damiano Varagnolo Something about me 2 Something about me Post-Doc at KTH Post-Doc at U. Padova Visiting Scholar at UC Berkeley Ph.D. Student
Reinforcement Learning
Reinforcement Learning LU 2 - Markov Decision Problems and Dynamic Programming Dr. Joschka Bödecker AG Maschinelles Lernen und Natürlichsprachliche Systeme Albert-Ludwigs-Universität Freiburg jboedeck@informatik.uni-freiburg.de
SOLVING LINEAR SYSTEMS
SOLVING LINEAR SYSTEMS Linear systems Ax = b occur widely in applied mathematics They occur as direct formulations of real world problems; but more often, they occur as a part of the numerical analysis
Chapter 3 Nonlinear Model Predictive Control
Chapter 3 Nonlinear Model Predictive Control In this chapter, we introduce the nonlinear model predictive control algorithm in a rigorous way. We start by defining a basic NMPC algorithm for constant reference
COMPLETE MARKETS DO NOT ALLOW FREE CASH FLOW STREAMS
COMPLETE MARKETS DO NOT ALLOW FREE CASH FLOW STREAMS NICOLE BÄUERLE AND STEFANIE GRETHER Abstract. In this short note we prove a conjecture posed in Cui et al. 2012): Dynamic mean-variance problems in
HETEROGENEOUS AGENTS AND AGGREGATE UNCERTAINTY. Daniel Harenberg daniel.harenberg@gmx.de. University of Mannheim. Econ 714, 28.11.
COMPUTING EQUILIBRIUM WITH HETEROGENEOUS AGENTS AND AGGREGATE UNCERTAINTY (BASED ON KRUEGER AND KUBLER, 2004) Daniel Harenberg daniel.harenberg@gmx.de University of Mannheim Econ 714, 28.11.06 Daniel Harenberg
8. Linear least-squares
8. Linear least-squares EE13 (Fall 211-12) definition examples and applications solution of a least-squares problem, normal equations 8-1 Definition overdetermined linear equations if b range(a), cannot
Lecture 5: Variants of the LMS algorithm
1 Standard LMS Algorithm FIR filters: Lecture 5: Variants of the LMS algorithm y(n) = w 0 (n)u(n)+w 1 (n)u(n 1) +...+ w M 1 (n)u(n M +1) = M 1 k=0 w k (n)u(n k) =w(n) T u(n), Error between filter output
TTT4120 Digital Signal Processing Suggested Solution to Exam Fall 2008
Norwegian University of Science and Technology Department of Electronics and Telecommunications TTT40 Digital Signal Processing Suggested Solution to Exam Fall 008 Problem (a) The input and the input-output
Vectors VECTOR PRODUCT. Graham S McDonald. A Tutorial Module for learning about the vector product of two vectors. Table of contents Begin Tutorial
Vectors VECTOR PRODUCT Graham S McDonald A Tutorial Module for learning about the vector product of two vectors Table of contents Begin Tutorial c 2004 g.s.mcdonald@salford.ac.uk 1. Theory 2. Exercises
Lecture 2. Marginal Functions, Average Functions, Elasticity, the Marginal Principle, and Constrained Optimization
Lecture 2. Marginal Functions, Average Functions, Elasticity, the Marginal Principle, and Constrained Optimization 2.1. Introduction Suppose that an economic relationship can be described by a real-valued
Mapping an Application to a Control Architecture: Specification of the Problem
Mapping an Application to a Control Architecture: Specification of the Problem Mieczyslaw M. Kokar 1, Kevin M. Passino 2, Kenneth Baclawski 1, and Jeffrey E. Smith 3 1 Northeastern University, Boston,
Lecture 5 Principal Minors and the Hessian
Lecture 5 Principal Minors and the Hessian Eivind Eriksen BI Norwegian School of Management Department of Economics October 01, 2010 Eivind Eriksen (BI Dept of Economics) Lecture 5 Principal Minors and
ECON20310 LECTURE SYNOPSIS REAL BUSINESS CYCLE
ECON20310 LECTURE SYNOPSIS REAL BUSINESS CYCLE YUAN TIAN This synopsis is designed merely for keep a record of the materials covered in lectures. Please refer to your own lecture notes for all proofs.
Estimating an ARMA Process
Statistics 910, #12 1 Overview Estimating an ARMA Process 1. Main ideas 2. Fitting autoregressions 3. Fitting with moving average components 4. Standard errors 5. Examples 6. Appendix: Simple estimators
Background 2. Lecture 2 1. The Least Mean Square (LMS) algorithm 4. The Least Mean Square (LMS) algorithm 3. br(n) = u(n)u H (n) bp(n) = u(n)d (n)
Lecture 2 1 During this lecture you will learn about The Least Mean Squares algorithm (LMS) Convergence analysis of the LMS Equalizer (Kanalutjämnare) Background 2 The method of the Steepest descent that
The Degrees of Freedom of Compute-and-Forward
The Degrees of Freedom of Compute-and-Forward Urs Niesen Jointly with Phil Whiting Bell Labs, Alcatel-Lucent Problem Setting m 1 Encoder m 2 Encoder K transmitters, messages m 1,...,m K, power constraint
Statistical Machine Learning
Statistical Machine Learning UoC Stats 37700, Winter quarter Lecture 4: classical linear and quadratic discriminants. 1 / 25 Linear separation For two classes in R d : simple idea: separate the classes
Optimal order placement in a limit order book. Adrien de Larrard and Xin Guo. Laboratoire de Probabilités, Univ Paris VI & UC Berkeley
Optimal order placement in a limit order book Laboratoire de Probabilités, Univ Paris VI & UC Berkeley Outline 1 Background: Algorithm trading in different time scales 2 Some note on optimal execution
Sales forecasting # 2
Sales forecasting # 2 Arthur Charpentier arthur.charpentier@univ-rennes1.fr 1 Agenda Qualitative and quantitative methods, a very general introduction Series decomposition Short versus long term forecasting
Examples of Functions
Examples of Functions In this document is provided examples of a variety of functions. The purpose is to convince the beginning student that functions are something quite different than polynomial equations.
Two Topics in Parametric Integration Applied to Stochastic Simulation in Industrial Engineering
Two Topics in Parametric Integration Applied to Stochastic Simulation in Industrial Engineering Department of Industrial Engineering and Management Sciences Northwestern University September 15th, 2014
Torgerson s Classical MDS derivation: 1: Determining Coordinates from Euclidean Distances
Torgerson s Classical MDS derivation: 1: Determining Coordinates from Euclidean Distances It is possible to construct a matrix X of Cartesian coordinates of points in Euclidean space when we know the Euclidean
ECE 3510 Final given: Spring 11
ECE 50 Final given: Spring This part of the exam is Closed book, Closed notes, No Calculator.. ( pts) For each of the time-domain signals shown, draw the poles of the signal's Laplace transform on the
Single item inventory control under periodic review and a minimum order quantity
Single item inventory control under periodic review and a minimum order quantity G. P. Kiesmüller, A.G. de Kok, S. Dabia Faculty of Technology Management, Technische Universiteit Eindhoven, P.O. Box 513,
4F7 Adaptive Filters (and Spectrum Estimation) Least Mean Square (LMS) Algorithm Sumeetpal Singh Engineering Department Email : sss40@eng.cam.ac.
4F7 Adaptive Filters (and Spectrum Estimation) Least Mean Square (LMS) Algorithm Sumeetpal Singh Engineering Department Email : sss40@eng.cam.ac.uk 1 1 Outline The LMS algorithm Overview of LMS issues
Pattern Analysis. Logistic Regression. 12. Mai 2009. Joachim Hornegger. Chair of Pattern Recognition Erlangen University
Pattern Analysis Logistic Regression 12. Mai 2009 Joachim Hornegger Chair of Pattern Recognition Erlangen University Pattern Analysis 2 / 43 1 Logistic Regression Posteriors and the Logistic Function Decision
Level Set Framework, Signed Distance Function, and Various Tools
Level Set Framework Geometry and Calculus Tools Level Set Framework,, and Various Tools Spencer Department of Mathematics Brigham Young University Image Processing Seminar (Week 3), 2010 Level Set Framework
The Method of Partial Fractions Math 121 Calculus II Spring 2015
Rational functions. as The Method of Partial Fractions Math 11 Calculus II Spring 015 Recall that a rational function is a quotient of two polynomials such f(x) g(x) = 3x5 + x 3 + 16x x 60. The method
Web-based Supplementary Materials for Bayesian Effect Estimation. Accounting for Adjustment Uncertainty by Chi Wang, Giovanni
1 Web-based Supplementary Materials for Bayesian Effect Estimation Accounting for Adjustment Uncertainty by Chi Wang, Giovanni Parmigiani, and Francesca Dominici In Web Appendix A, we provide detailed
4F7 Adaptive Filters (and Spectrum Estimation) Kalman Filter. Sumeetpal Singh Email : sss40@eng.cam.ac.uk
4F7 Adaptive Filters (and Spectrum Estimation) Kalman Filter Sumeetpal Singh Email : sss40@eng.cam.ac.uk 1 1 Outline State space model Kalman filter Examples 2 2 Parameter Estimation We have repeated observations
Lecture 24: Oscillators. Clapp Oscillator. VFO Startup
Whites, EE 322 Lecture 24 Page 1 of 10 Lecture 24: Oscillators. Clapp Oscillator. VFO Startup Oscillators are circuits that produce periodic output voltages, such as sinusoids. They accomplish this feat
a 11 x 1 + a 12 x 2 + + a 1n x n = b 1 a 21 x 1 + a 22 x 2 + + a 2n x n = b 2.
Chapter 1 LINEAR EQUATIONS 1.1 Introduction to linear equations A linear equation in n unknowns x 1, x,, x n is an equation of the form a 1 x 1 + a x + + a n x n = b, where a 1, a,..., a n, b are given
Lecture L3 - Vectors, Matrices and Coordinate Transformations
S. Widnall 16.07 Dynamics Fall 2009 Lecture notes based on J. Peraire Version 2.0 Lecture L3 - Vectors, Matrices and Coordinate Transformations By using vectors and defining appropriate operations between
CONTROLLABILITY. Chapter 2. 2.1 Reachable Set and Controllability. Suppose we have a linear system described by the state equation
Chapter 2 CONTROLLABILITY 2 Reachable Set and Controllability Suppose we have a linear system described by the state equation ẋ Ax + Bu (2) x() x Consider the following problem For a given vector x in
Systems of Linear Equations
Systems of Linear Equations Beifang Chen Systems of linear equations Linear systems A linear equation in variables x, x,, x n is an equation of the form a x + a x + + a n x n = b, where a, a,, a n and
M/M/1 and M/M/m Queueing Systems
M/M/ and M/M/m Queueing Systems M. Veeraraghavan; March 20, 2004. Preliminaries. Kendall s notation: G/G/n/k queue G: General - can be any distribution. First letter: Arrival process; M: memoryless - exponential
PID Controller Design for Nonlinear Systems Using Discrete-Time Local Model Networks
PID Controller Design for Nonlinear Systems Using Discrete-Time Local Model Networks 4. Workshop für Modellbasierte Kalibriermethoden Nikolaus Euler-Rolle, Christoph Hametner, Stefan Jakubek Christian
(Refer Slide Time: 01:11-01:27)
Digital Signal Processing Prof. S. C. Dutta Roy Department of Electrical Engineering Indian Institute of Technology, Delhi Lecture - 6 Digital systems (contd.); inverse systems, stability, FIR and IIR,
Methods for Vibration Analysis
. 17 Methods for Vibration Analysis 17 1 Chapter 17: METHODS FOR VIBRATION ANALYSIS 17 2 17.1 PROBLEM CLASSIFICATION According to S. H. Krandall (1956), engineering problems can be classified into three
COMPUTER SIMULATION OF REAL TIME IDENTIFICATION FOR INDUCTION MOTOR DRIVES
Proceedings of the International Conference on Theory and Applications of Mathematics and Informatics - ICTAMI 2004, Thessaloniki, Greece COMPUTER SIMULATION OF REAL TIME IDENTIFICATION FOR INDUCTION MOTOR
ANALYZING INVESTMENT RETURN OF ASSET PORTFOLIOS WITH MULTIVARIATE ORNSTEIN-UHLENBECK PROCESSES
ANALYZING INVESTMENT RETURN OF ASSET PORTFOLIOS WITH MULTIVARIATE ORNSTEIN-UHLENBECK PROCESSES by Xiaofeng Qian Doctor of Philosophy, Boston University, 27 Bachelor of Science, Peking University, 2 a Project
Chapter 2: Binomial Methods and the Black-Scholes Formula
Chapter 2: Binomial Methods and the Black-Scholes Formula 2.1 Binomial Trees We consider a financial market consisting of a bond B t = B(t), a stock S t = S(t), and a call-option C t = C(t), where the
Lecture 3. Linear Programming. 3B1B Optimization Michaelmas 2015 A. Zisserman. Extreme solutions. Simplex method. Interior point method
Lecture 3 3B1B Optimization Michaelmas 2015 A. Zisserman Linear Programming Extreme solutions Simplex method Interior point method Integer programming and relaxation The Optimization Tree Linear Programming
Factorization Theorems
Chapter 7 Factorization Theorems This chapter highlights a few of the many factorization theorems for matrices While some factorization results are relatively direct, others are iterative While some factorization
1 Inner Products and Norms on Real Vector Spaces
Math 373: Principles Techniques of Applied Mathematics Spring 29 The 2 Inner Product 1 Inner Products Norms on Real Vector Spaces Recall that an inner product on a real vector space V is a function from
ARX-Model based Model Predictive Control with Offset-Free Tracking.
0 th European Symposium on Computer Aided Process Engineering ESCAPE0 S. Pierucci and G. Buzzi Ferraris (Editors) 00 Elsevier B.V. All rights reserved. ARX-Model based Model Predictive Control with Offset-Free
PID, LQR and LQR-PID on a Quadcopter Platform
PID, LQR and LQR-PID on a Quadcopter Platform Lucas M. Argentim unielargentim@fei.edu.br Willian C. Rezende uniewrezende@fei.edu.br Paulo E. Santos psantos@fei.edu.br Renato A. Aguiar preaguiar@fei.edu.br
Lecture 7: Finding Lyapunov Functions 1
Massachusetts Institute of Technology Department of Electrical Engineering and Computer Science 6.243j (Fall 2003): DYNAMICS OF NONLINEAR SYSTEMS by A. Megretski Lecture 7: Finding Lyapunov Functions 1
Variance Reduction. Pricing American Options. Monte Carlo Option Pricing. Delta and Common Random Numbers
Variance Reduction The statistical efficiency of Monte Carlo simulation can be measured by the variance of its output If this variance can be lowered without changing the expected value, fewer replications
QAM Demodulation. Performance Conclusion. o o o o o. (Nyquist shaping, Clock & Carrier Recovery, AGC, Adaptive Equaliser) o o. Wireless Communications
0 QAM Demodulation o o o o o Application area What is QAM? What are QAM Demodulation Functions? General block diagram of QAM demodulator Explanation of the main function (Nyquist shaping, Clock & Carrier
1 Introduction to Matrices
1 Introduction to Matrices In this section, important definitions and results from matrix algebra that are useful in regression analysis are introduced. While all statements below regarding the columns
LECTURE: INTRO TO LINEAR PROGRAMMING AND THE SIMPLEX METHOD, KEVIN ROSS MARCH 31, 2005
LECTURE: INTRO TO LINEAR PROGRAMMING AND THE SIMPLEX METHOD, KEVIN ROSS MARCH 31, 2005 DAVID L. BERNICK dbernick@soe.ucsc.edu 1. Overview Typical Linear Programming problems Standard form and converting
Manufacturing Equipment Modeling
QUESTION 1 For a linear axis actuated by an electric motor complete the following: a. Derive a differential equation for the linear axis velocity assuming viscous friction acts on the DC motor shaft, leadscrew,
Safety margins for unsystematic biometric risk in life and health insurance
Safety margins for unsystematic biometric risk in life and health insurance Marcus C. Christiansen Preprint Series: 20-04 Fakultät für Mathematik und Wirtschaftswissenschaften UNIVERSITÄT ULM Safety margins
1 Lecture: Integration of rational functions by decomposition
Lecture: Integration of rational functions by decomposition into partial fractions Recognize and integrate basic rational functions, except when the denominator is a power of an irreducible quadratic.
Introduction to General and Generalized Linear Models
Introduction to General and Generalized Linear Models General Linear Models - part I Henrik Madsen Poul Thyregod Informatics and Mathematical Modelling Technical University of Denmark DK-2800 Kgs. Lyngby
A Simple Pseudo Random Number algorithm
Lecture 7 Generating random numbers is a useful technique in many numerical applications in Physics. This is because many phenomena in physics are random, and algorithms that use random numbers have applications
SECOND DERIVATIVE TEST FOR CONSTRAINED EXTREMA
SECOND DERIVATIVE TEST FOR CONSTRAINED EXTREMA This handout presents the second derivative test for a local extrema of a Lagrange multiplier problem. The Section 1 presents a geometric motivation for the
Modern Optimization Methods for Big Data Problems MATH11146 The University of Edinburgh
Modern Optimization Methods for Big Data Problems MATH11146 The University of Edinburgh Peter Richtárik Week 3 Randomized Coordinate Descent With Arbitrary Sampling January 27, 2016 1 / 30 The Problem
Power Electronics. Prof. K. Gopakumar. Centre for Electronics Design and Technology. Indian Institute of Science, Bangalore.
Power Electronics Prof. K. Gopakumar Centre for Electronics Design and Technology Indian Institute of Science, Bangalore Lecture - 1 Electric Drive Today, we will start with the topic on industrial drive
Radar Systems Engineering Lecture 6 Detection of Signals in Noise
Radar Systems Engineering Lecture 6 Detection of Signals in Noise Dr. Robert M. O Donnell Guest Lecturer Radar Systems Course 1 Detection 1/1/010 Block Diagram of Radar System Target Radar Cross Section
Content. Professur für Steuerung, Regelung und Systemdynamik. Lecture: Vehicle Dynamics Tutor: T. Wey Date: 01.01.08, 20:11:52
1 Content Overview 1. Basics on Signal Analysis 2. System Theory 3. Vehicle Dynamics Modeling 4. Active Chassis Control Systems 5. Signals & Systems 6. Statistical System Analysis 7. Filtering 8. Modeling,
Lectures 5-6: Taylor Series
Math 1d Instructor: Padraic Bartlett Lectures 5-: Taylor Series Weeks 5- Caltech 213 1 Taylor Polynomials and Series As we saw in week 4, power series are remarkably nice objects to work with. In particular,
Cooperative Object Tracking for Many-on-Many Engagement
Cooperative Object Tracking for Many-on-Many Engagement Pradeep Bhatta a and Michael A. Paluszek a a Princeton Satellite Systems, 33 Witherspoon Street, Princeton, NJ, USA ABSTRACT This paper presents
Evaluating System Suitability CE, GC, LC and A/D ChemStation Revisions: A.03.0x- A.08.0x
CE, GC, LC and A/D ChemStation Revisions: A.03.0x- A.08.0x This document is believed to be accurate and up-to-date. However, Agilent Technologies, Inc. cannot assume responsibility for the use of this
1.2 Solving a System of Linear Equations
1.. SOLVING A SYSTEM OF LINEAR EQUATIONS 1. Solving a System of Linear Equations 1..1 Simple Systems - Basic De nitions As noticed above, the general form of a linear system of m equations in n variables
Abstract: We describe the beautiful LU factorization of a square matrix (or how to write Gaussian elimination in terms of matrix multiplication).
MAT 2 (Badger, Spring 202) LU Factorization Selected Notes September 2, 202 Abstract: We describe the beautiful LU factorization of a square matrix (or how to write Gaussian elimination in terms of matrix
EDUMECH Mechatronic Instructional Systems. Ball on Beam System
EDUMECH Mechatronic Instructional Systems Ball on Beam System Product of Shandor Motion Systems Written by Robert Hirsch Ph.D. 998-9 All Rights Reserved. 999 Shandor Motion Systems, Ball on Beam Instructional
Mathematics of Life Contingencies MATH 3281
Mathematics of Life Contingencies MATH 3281 Life annuities contracts Edward Furman Department of Mathematics and Statistics York University February 13, 2012 Edward Furman Mathematics of Life Contingencies
MOBILE ROBOT TRACKING OF PRE-PLANNED PATHS. Department of Computer Science, York University, Heslington, York, Y010 5DD, UK (email:nep@cs.york.ac.
MOBILE ROBOT TRACKING OF PRE-PLANNED PATHS N. E. Pears Department of Computer Science, York University, Heslington, York, Y010 5DD, UK (email:nep@cs.york.ac.uk) 1 Abstract A method of mobile robot steering
CONTROL SYSTEMS, ROBOTICS, AND AUTOMATION - Vol. V - Relations Between Time Domain and Frequency Domain Prediction Error Methods - Tomas McKelvey
COTROL SYSTEMS, ROBOTICS, AD AUTOMATIO - Vol. V - Relations Between Time Domain and Frequency Domain RELATIOS BETWEE TIME DOMAI AD FREQUECY DOMAI PREDICTIO ERROR METHODS Tomas McKelvey Signal Processing,
Coding and decoding with convolutional codes. The Viterbi Algor
Coding and decoding with convolutional codes. The Viterbi Algorithm. 8 Block codes: main ideas Principles st point of view: infinite length block code nd point of view: convolutions Some examples Repetition
FE570 Financial Markets and Trading. Stevens Institute of Technology
FE570 Financial Markets and Trading Lecture 13. Execution Strategies (Ref. Anatoly Schmidt CHAPTER 13 Execution Strategies) Steve Yang Stevens Institute of Technology 11/27/2012 Outline 1 Execution Strategies
G(s) = Y (s)/u(s) In this representation, the output is always the Transfer function times the input. Y (s) = G(s)U(s).
Transfer Functions The transfer function of a linear system is the ratio of the Laplace Transform of the output to the Laplace Transform of the input, i.e., Y (s)/u(s). Denoting this ratio by G(s), i.e.,
Real-Time Systems Versus Cyber-Physical Systems: Where is the Difference?
Real-Time Systems Versus Cyber-Physical Systems: Where is the Difference? Samarjit Chakraborty www.rcs.ei.tum.de TU Munich, Germany Joint work with Dip Goswami*, Reinhard Schneider #, Alejandro Masrur
The continuous and discrete Fourier transforms
FYSA21 Mathematical Tools in Science The continuous and discrete Fourier transforms Lennart Lindegren Lund Observatory (Department of Astronomy, Lund University) 1 The continuous Fourier transform 1.1
PSTricks. pst-ode. A PSTricks package for solving initial value problems for sets of Ordinary Differential Equations (ODE), v0.7.
PSTricks pst-ode A PSTricks package for solving initial value problems for sets of Ordinary Differential Equations (ODE), v0.7 27th March 2014 Package author(s): Alexander Grahn Contents 2 Contents 1 Introduction
MAT188H1S Lec0101 Burbulla
Winter 206 Linear Transformations A linear transformation T : R m R n is a function that takes vectors in R m to vectors in R n such that and T (u + v) T (u) + T (v) T (k v) k T (v), for all vectors u
Mathematical finance and linear programming (optimization)
Mathematical finance and linear programming (optimization) Geir Dahl September 15, 2009 1 Introduction The purpose of this short note is to explain how linear programming (LP) (=linear optimization) may