Model Predictive Control of Hybrid Systems

st HYCON PhD School on Hybrid Systems www.unisi.it www.ist-hycon.org Model Predictive Control of Hybrid Systems Alberto Bemporad University of Siena, Italy bemporad@dii.unisi.it scimanyd suounitnoc enibmoc smetsys dirbyh lacipyt (snoitauqe ecnereffid ro laitnereffid) scimanyd etercsid dna stnalp lacisyhp fo fo lacipyt (snoitidnoc lacigol dna atamotua) fo senilpicsid gninibmoc yb.cigol lortnoc,yroeht lortnoc dna smetsys dna ecneics retupmoc dilos a edivorp smetsys dirbyh no hcraeser,sisylana eht rof sloot lanoitatupmoc dna yroeht fo ngised lortnoc dna,noitacifirev,noitalumis egral a ni desu era dna,''smetsys deddebme`` ria,smetsys evitomotua) snoitacilppa fo yteirav ssecorp,smetsys lacigoloib,tnemeganam ciffart.(srehto ynam dna,seirtsudni HYSCOM IEEE CSS Technical Committee on Hybrid Systems Siena, July 9-, 5 - Rectorate of the University of Siena 9

Model Predictive Control of Hybrid Systems Alberto Bemporad Dept. Information Engineering - University of Siena, Italy http://www.dii.unisi.it/~bemporad Outline Model Predictive Control (MPC) concepts Hybrid models for MPC MPC of hybrid systems Explicit MPC (multiparametric programming) Optimization-based reachability analysis University y of Siena (founded in ) Examples EIHS European Institute of Hybrid Systems st HYCON PhD School on Hybrid Systems (Siena, /7/5) Model Predictive Control Optimizer Plant Model Predictive Control Reference r(t) Input u(t) Output y(t) MPC concepts Linear MPC Matlab tools for linear MPC Measurements MODEL: a model of the plant is needed to predict the future behavior of the plant PREDICTIVE: optimization is based on the predicted future evolution of the plant CONTROL: control complex constrained multivariable plants 3 At time t : Receding Horizon Philosophy Solve an optimal control problem over a finite future horizon N : minimize t t+ r(t) y(t+k) Predicted outputs Manipulated Inputs u(t+k) t+n Receding Horizon - Example MPC is like playing chess! subject to constraints t+ t+ t+n+ Only apply the first optimal move Get new measurements, and repeat the optimization at time t+ Advantage of on-line optimization: FEEDBACK! 5 6

Linear Model: Constrained Optimal Control Constrained Optimal Control Optimization problem: By substituting, we get Constraints: (quadratic) Constrained optimal control problem (quadratic performance index): Convex QUADRATIC PROGRAM (QP) (linear) 7 8 MPC of Linear Systems Model Predictive Control Toolbox At time t: past future t t+ Predicted outputs y(t+k t) Manipulated Inputs u(t+k) t+n Get/estimate the current state x(t) MPC Toolbox. (Bemporad, Ricker, Morari, 998-): Object-oriented implementation (MPC object) MPC Simulink Library MPC Graphical User Interface RTW extension (code generation) Solve the QP problem and let U={u * (),...,u * (N-)} be the solution (=finite-horizon constrained open-loop optimal control) Apply only optimal inputs Go to time t+ and discard the remaining 9 Only linear models are handled http://www.mathworks.com Example: AFTI-6 Example: AFTI-6 Linearized model: see demo afti6.m (MPC-Tbx)

Convergence Convergence Proof Lyapunov function (Keerthi and Gilbert, 988)(Bemporad et al., 99) Proof: Use value function as Lyapunov function 3 Global optimum is not needed to prove convergence! Convergence Proof Convergence Proof 5 6 Consider the MPC control law: MPC and LQR Jacopo Francesco Riccati (676-75) Outline Model Predictive Control (MPC) concepts Hybrid models for MPC MPC of hybrid systems Explicit MPC (multiparametric programming) Optimization-based reachability analysis In a polyhedral region around the origin the MPC control law is equivalent to the constrained LQR controller with weights Q,R. (Chmielewski, Manousiouthakis, 996) (Scokaert and Rawlings, 998) MPC constrained LQR The larger the horizon, the larger the region where MPC=LQR 7 Examples 8

Hybrid Systems Computer Science Control Theory Hybrid Models for MPC Finite state machines Continuous dynamical systems Discrete Hybrid Automata (DHA) B A Hybrid systems Mixed Logical Dynamical (MLD) Systems Piecewise Affine (PWA) Systems B C B C u(t) system y(t) C A 9 Intrinsically Hybrid Systems continuous inputs Cruise Control Problem GOAL: command gear ratio, gas pedal, and brakes to track a desired speed and minimize consumptions discrete input Discrete input (,N,,3,) + Continuous inputs (brakes, gas, clutch) + Continuous dynamical states (velocities, torques, air-flows, fuel level) CHALLENGES: continuous and discrete inputs dynamics depends on gear nonlinear torque/speed maps Key Requirements for Hybrid Models Descriptive enough to capture the behavior of the system Piecewise Affine Systems state+input space continuous dynamics (physical laws) logic components (switches, automata, software code) interconnection between logic and dynamics Simple enough for solving analysis and synthesis problems? (Sontag 98) linear systems linear hybrid systems nonlinear systems Make everything as simple as possible, but not simpler. Albert Einstein Can approximate nonlinear/discontinuous dynamics arbitrarily well

Discrete Hybrid Automaton Event Generator (Torrisi, Bemporad, ) Switched Affine System Finite State Machine time or event counter Switched Affine System s Mode Selector mode The affine dynamics depend on the current mode i(k): 5 6 Event Generator Mode Selector Event variables are generated by linear threshold conditions over continuous states, continuous inputs, and time: The active mode i(k) is selected by a Boolean function of the current binary states, binary inputs, and event variables: Example: [δ=] [x c (k) ] 7 Example: the system has 3 modes 8 Finite State Machine Logic and Inequalities Glover 975, Williams 977 The binary state of the finite state machine evolves according to a Boolean state update function: Finite State Machine Mode Selector Switched Affine System Event Generator Example: s 9 3

Switched Affine System s Switched Affine Systems The state-update equation can be rewritten as a combination of affine terms and if-then-else conditions: Logic Inequalities: Symbolic Approach. Given a Boolean statement. Convert to Conjunctive Normal Form (CNF):. Transform into inequalities: where. polyhedron If-then-else conditions are converted to inequalities. Output equations y c (t)=c i x c (t)+d i u c (t)+g i admit a similar transformation. 3 Any logic proposition can be translated into linear integer ineq. 3 Logic Inequalities: Symbolic Approach Example: F (X,X,X 3 )=[X 3 X X ] Logic Inequalities: Geometric Approach Example: logic AND F (X,X,X 3 )=[X 3 X X ] δ 3 (,,). Convert to Conjunctive Normal Form (CNF): (see e.g.: http://www.oursland.net/aima/propositionapplet.html or just search [CNF + applet] on Google...) T: X X X 3 (,,) δ (X 3 X X ) (X X 3 ) (X X 3 ) Key idea: White points cannot be in the hull of black points (,,) (,,) δ. Transform into inequalities: δ 3 +( δ )+( δ ) δ +( δ 3 ) δ +( δ 3 ) Convex hull algorithms: cdd, lrs, qhull, chd, Hull, Porto 33 3 Mixed Logical Dynamical Systems Discrete Hybrid Automaton System: A Simple Example HYSDEL (Torrisi, Bemporad, ) Associate and transform Mixed Logical Dynamical (MLD) Systems (Bemporad, Morari 999) Then Continuous and binary variables Computationally oriented (mixed-integer programming) Rewrite as a linear equation Suitable for controller synthesis, verification,... 35 36

Features: Hybrid Toolbox for Matlab (Bemporad, 3-) Hybrid model (MLD and PWA) design and simulation Control design for linear systems w/ constraints and hybrid systems (on-line optimization via QP/MILP/MIQP) Explicit control (via multiparametric programming) C-code generation Simulink HYSDEL (HYbrid Systems DEscription Language) Describe hybrid systems: Automata Logic Lin. Dynamics Interfaces Constraints Automatically generate MLD models in Matlab (Torrisi, Bemporad, ) Download: http://www.dii.unisi.it/hybrid/toolbox http://www.dii.unisi.it/hybrid/toolbox 37 Reference: http://control.ethz.ch/~hybrid/hysdel 38 Example: Bouncing Ball Bouncing Ball Time Discretization x x F= m g N -mg How to model this system in MLD form? x 39 HYSDEL - Bouncing Ball SYSTEM bouncing_ball { INTERFACE { /* Description of variables and constants */ STATE { REAL height [-,]; REAL velocity [-,]; } PARAMETER { REAL g; REAL dissipation; /* =elastic, =completely anelastic */ REAL Ts; } } IMPLEMENTATION { AUX { REAL z; REAL z; BOOL negative; } AD { negative = height <= ; } Systems Theory for Discrete-Time Linear Hybrid Systems Analysis Well-posedness Realization & Transformation Reachability (=Verification) Observability Stability Identification }} DA { z = { IF negative THEN height-ts*velocity ELSE height+ts*velocity-ts*ts*g}; z = { IF negative THEN -(-dissipation)*velocity ELSE velocity-ts*g}; } CONTINUOUS { height = z; velocity=z;} Synthesis Control State estimation go to demo /demos/hybrid/bball.m

Well-posedness Are state and output trajectories defined? Uniquely defined? Persistently defined? Equivalences of Hybrid Models MLD well-posedness : î(t) =F(x(t),u(t)) z(t) =G(x(t),u(t)) {x(t),u(t)} {x(t +)} {x(t),u(t)} {y(t)} Σ x (k), y (k) are single valued u(k) x ()=x () Σ x (k), y (k) Numerical test based on mixed-integer programming available (Bemporad, Morari, Automatica, 999) 3 MLD and PWA Systems Example: Room Temperature Theorem MLD systems and PWA systems are equivalent AC system Hybrid Dynamics (Bemporad, Ferrari-Trecate, Morari, IEEE TAC,) Proof is constructive: given an MLD system it returns its equivalent PWA form u hot u cold # turns the heater (air conditioning) on whenever he is cold (hot) Drawback: it needs the enumeration of all possible combinations of binary states, binary inputs, and δ variables Most of such combinations lead to empty regions Heater T amb T T T T If # is cold he turns the heater on, unless # is hot If # is hot he turns the air conditioning on, unless # is cold Efficient algorithms are available for converting MLD models into PWA models avoiding such an enumeration: Otherwise, heater and air conditioning are off A. Bemporad, Efficient Algorithms for Converting Mixed Logical Dynamical Systems into an Equivalent Piecewise Affine Form, IEEE Trans. Autom. Contr.,. T. Geyer, F.D. Torrisi and M. Morari, Efficient Mode Enumeration of Compositional Hybrid Models, HSCC 3 5 T = α (T T amb )+k (u hot u cold ) (body temperature dynamics of #) T = α (T T amb )+k (u hot u cold ) (body temperature dynamics of #) go to demo /demos/hybrid/heatcool.m 6 HYSDEL Model MLD model Hybrid MLD Model Hybrid Toolbox for Matlab (Bemporad, 3-5) http://www.dii.unisi.it/hybrid/toolbox continuous states: continuous input: auxiliary continuous vars: (temperatures T,T ) (room temperature T amb ) (power flows u hot, u cold ) >>S=mld('heatcoolmodel',Ts) >>[XX,TT]=sim(S,x,U); get the MLD model in Matlab simulate the MLD model 6 auxiliary binary vars: ( thresholds + for OR condition) mixed-integer inequalities Possible combination of integer variables: 6 = 6 7 8

PWA model Hybrid PWA Model Simulation in Simulink continuous states: (temperatures T,T ) continuous input: (room temperature T amb ) 5 polyhedral regions (partition does not depend on input) Temperature T (deg C) 5 3 >>P=pwa(S); - - 3 5 Temperature T(deg C) uhot = u cold = u hot = u cold = ŪC uhot = ŪH u cold = Other Existing Hybrid Models Linear complementarity (LC) systems x(t +)=Ax(t)+B u(t)+b w(t) y(t) =Cx(t)+D u(t)+d w(t) v(t) =E x(t)+e u(t)+e 3 w(t)+e ô v(t) w(t) õ Ex: mechanical systems circuits with diodes etc. Extended linear complementarity (ELC) systems Generalization of LC systems Min-max-plus-scaling (MMPS) systems x(t +)=M x (x(t),u(t),d(t)) y(t) =M y (x(t),u(t),d(t)) õ M c (x(t),u(t),d(t)) Example: x(t + ) = max(x(t), ) + min(à u(t), ) (Heemels, 999) M := x i ë max(m,m ) min(m,m ) M + M ìm 9 (De Schutter, De Moor, ) (De Schutter, Van den Boom, ) MMPS function: defined by the grammar Used for modeling discrete-event systems (t=event counter) 5 Equivalence Results DHA? PWA?? MLD?? LC? ELC MMPS Theorem All the above six classes of discrete-time hybrid models are equivalent (possibly under some additional assumptions, such as boundedness of input and state variables) (Heemels, De Schutter, Bemporad, Automatica, ) Theoretical properties and analysis/synthesis tools can be transferred from one class to another??? (Torrisi, Bemporad, IEEE CST, 3) (Bemporad and Morari, Automatica, 999) (Bemporad, Ferrari-T.,Morari, IEEETAC, ) 5 5 MLD and LC Systems (Heemels, De Schutter, Bemporad, Automatica, ) Hybrid System Identification Sometimes a hybrid model of the process (or of a part of it) cannot be derived manually from available knowledge. Therefore, a model must be either Estimated from data (model unknown) or hybridized before it can be used for control/analysis (model known but nonlinear) Proof: x(t +)=Ax(t)+B u(t) + B î(t)+b 3z(t) y(t) =Cx(t)+D u(t) + D î(t)+d 3z(t) E î(t)+e 3z(t) ô E x(t)+e u(t)+e 5 x(t +)=Ax(t)+B u(t)+b w(t) y(t) =Cx(t)+D u(t)+d w(t) v(t) =E x(t)+e u(t)+e 3w(t)+e ô v(t) w(t) õ For each complementarity pair v i (t),w i (t) introduce a binary variable î i (t) {, } [î i (t) =] [v i (t) =, w i (t) õ ] w i (t) ô Mî i (t) v [î i (t) =] [v i (t) õ, w i (t) =] i (t) ô M( à î i (t)) w i (t) õ v i (t) õ Set z i (t) =w i (t) and substitute v(t) =E x(t)+e u(t)+e 3 w(t)+e 53 If a linear model is enough, no problem: several algorithms are available (e.g.: use Ljung s ID TBX) If switching modes are known and data can be generated for each mode, no problem: we identify one linear model per mode (e.g.: use Ljung s ID TBX) If modes & dynamics must be identified together, we need hybrid system identification 5

PWA Identification Problem Outline Estimate from data both the parameters of the affine submodels and the partition of the PWA map Example Let the data be generated by the PWARX system Model Predictive Control (MPC) concepts Hybrid models for MPC MPC of hybrid systems Explicit MPC (multiparametric programming) Optimization-based reachability analysis Examples 55 56 Hybrid Control Problem hybrid process MPC of Hybrid Systems continuous inputs continuous states binary inputs binary states Problem setup Convergence properties Computational aspects Matlab tools for hybrid MPC on-line decision maker desired behavior constraints 57 58 MLD model Model Predictive Control of Hybrid Systems MPC for Hybrid Systems past future t t+ Predicted outputs y(t+k t) Manipulated Inputs u(t+k) t+t Model Predictive (MPC) Control Reference r(t) Controller Input u(t) Hybrid System Output y(t) At time t solve with respect to the finite-horizon open-loop, optimal control problem: Measurements MODEL: use an MLD or PWA model of the plant to predict the future behavior of the hybrid system PREDICTIVE: optimization is still based on the predicted future evolution of the hybrid system CONTROL: the goal is to control the hybrid system 59 Apply only u(t)=u ã (t) (discard the remaining optimal inputs) Repeat the whole optimization at time t+ 6

Closed-Loop Convergence Convergence Proof Lyapunov function (Bemporad, Morari 999) Proof: Easily follows from standard Lyapunov arguments More stability results: see (Lazar, Heemels, Weiland, Bemporad, 5) 6 Note: Global optimum not needed for convergence! 6 PWA system: Hybrid MPC - Example Closed loop: Hybrid MPC - Example Constraint: Open loop behavior go to demo /demos/hybrid/bm99sim.m 63 6 Hybrid MPC Temperature Control Hybrid MPC Temperature Control >>refs.x=; % just weight state # >>Q.x=; >>Q.rho=Inf; % hard constraints >>Q.norm=; % quadratic costs >>N=; % optimization horizon >>limits.xmin=[5;-inf]; >>C=hybcon(S,Q,N,limits,refs); >> C Hybrid controller based on MLD model S <heatcoolmodel.hys> state measurement(s) output reference(s) input reference(s) state reference(s) reference(s) on auxiliary continuous z-variables optimization variable(s) (8 continuous, binary) 6 mixed-integer linear inequalities sampling time =.5, MILP solver = 'glpk' Type "struct(c)" for more details. >> >>[XX,UU,DD,ZZ,TT]=sim(C,S,r,x,Tstop); 65 66

MIQP Formulation of MPC (Bemporad, Morari, 999) Optimal Control of Hybrid Systems: Computational Aspects Mixed Integer Quadratic Program (MIQP) 67 68 MILP Formulation of MPC (Bemporad, Borrelli, Morari, ) Mixed-Integer Program Solvers Mixed-Integer Programming is NP-hard Phase transitions have been found in computationally hard problems. 3 5 var var var Introduce slack variables: min x min ï s.t. ï õ x ï õàx BUT Cost of Computation 3 5 6 7 8 Ratio of Constraints to Variables (Monasson et al., Nature, 999) ï x k ï u k õ kqy(t + k t)k õ kru(t + k)k Set ø,[ï x,...,ïx Ny,ï u,...,ï Tà,U,î,z] Mixed Integer Linear Program (MILP) ï x k õ [Qy(t + k t)] i i =,...,p, k =,..., T à ï x k õ à[qy(t + k t)] i i =,...,p, k =,...,Tà ï u k õ [Ru(t + k))] i i =,...,m, k =,...,Tà ï u k õ à[ru(t + k))] i i =,...,m, k =,...,Tà Tà min J(ø, x(t)) = X ï x + i ïu i ø k= s.t.gø ô W + Sx(t) Mixed-Integer Programming Solvers Main drawbacks: Loss of the original discrete structure (Boolean formulas) 69 General purpose Branch & Bound/Branch & Cut solvers available for MILP and MIQP (CPLEX, Xpress-MP, BARON, GLPK,...) More solvers and benchmarks: http://plato.la.asu.edu/bench.html No need to reach global optimum (see proof of the theorem), although performance deteriorates Mixed-Integer Programming Solvers Main drawbacks: Loss of the original discrete structure (Boolean formulas) 7 On-line combinatorial optimization On-line combinatorial optimization Good for large sampling times (e.g., h) / expensive hardware but not for fast sampling (e.g. ms) / cheap hardware! Good for large sampling times (e.g., h) / expensive hardware but not for fast sampling (e.g. ms) / cheap hardware! 7 7

Exploiting Logic Structures Main drawbacks when using Mixed-Integer Programming for implementing hybrid MPC control laws:. Loss of the original Boolean structure TRUE FALSE x`(k +)=x`(k) u`(k) Efficiency of MIP solver usually not good when continuous LP/QP relaxations are not tight Hybrid Solvers Combine MIP and Constraint Satisfaction (CSP) techniques to exploit the discrete structure of the problem Why CSP? More flexible modeling than MIP (e.g.: constraint logic programming (CLP) and Satisfiability of Boolean formulas (SAT)) Structure is kept and exploited to direct the search. Why MIP? Specialized techniques for highly structured problems (e.g. LP problems); Better for handling continuous vars A wide range of tight relaxations are available 73 Why a combined approach? Performance increase already shown in other application domains (Harjunkoski, Jain, Grossmann, ) 7 SAT-Based Branch&Bound The basic modeling framework has the following form: (Bockmayr, Kasper,998) Computations: MILP vs. LB-B&B B&B Convex cost function Continuous constraints Mixed constraints Purely logic constraints optimization variables Computation time and # LP solved for finding an optimal control sequence SAT-based B&B ingredients (Bemporad, Giorgetti, 3) SAT-based B&B Pure B&B A relaxed convex problem (pure logic constraints dropped) A SAT feasibility problem (for logic constraints only) #nodes= #nodes=67 Convex solver (e.g.: LP/QP) SAT solver 75 Pentium IV.8GHz SAT solver: zchaff 3.7. 76 SAT vs. MILP Uniform Random 3CNF benchmarks (from http://www.satlib.org) All 3SAT instances are in the phase transition region. Mixed-Integer Programming Solvers Main drawbacks: Loss of the original discrete structure (Boolean formulas) On-line combinatorial optimization All clauses are passed to CPLEX new MILP solver as logic constraints (we exploit the new feature of release 9.). Solvers: CPLEX 9. PC: P.8GHz + GB RAM zchaff 3.. 77 Good for large sampling times (e.g., h) / expensive hardware but not for fast sampling (e.g. ms) / cheap hardware! 78

Outline Model Predictive Control (MPC) concepts Hybrid models for MPC MPC of hybrid systems Explicit MPC (multiparametric programming) Optimization-based reachability analysis Examples Explicit Model Predictive Control Multiparametric quadratic programming Explicit linear MPC Explicit hybrid MPC Matlab tools for explicit MPC 79 8 MPC: Pros and Cons PRO: systematic design approach Multivariable systems State and Input Constraints Stability guarantees Reference preview CON: computation complexity! Large sampling time/fast hardware Software reliability On-Line vs. Off-Line Optimization On-line optimization: given x(t) solve the problem at each time step t (the control law u=u(x) is implicitly defined by the QP solver) Quadratic Program (QP) Off-line optimization: solve the QP for all x(t) to find the control law u=u(x) explicitly multi-parametric Quadratic Program (mp-qp) 8 8 Multiparametric Quadratic Programming (Bemporad et al., ) x X Linearity of the Solution solve QP to find identify active constraints at form matrices active constraints: by collecting KKT optimality conditions: Objective: solve the QP for all Assumptions: (always satisfied if QP problem originates from optimal control problem) From () : From () : In some neighborhood of x, λ and U are explicit (can be easily satisfied, e.g. by choosing positive input weights) 83 affine functions of x! 8

Determining a Critical Region Impose primal and dual feasibility: Neighboring Regions (Tøndel, Johansen, Bemporad, 3) linear inequalities in x! x x x CR CR CR Remove redundant constraints: (this requires solving LP s) x x x critical region CR CR x-space CR is the set of all and only parameters x for which is the optimal combination of active constraints at the optimizer x X 85 The active set of a neighboring region is found by using the active set of the current region + knowledge of the type of hyperplane we are crossing: The corresponding constraint is added to the active set The corresponding constraint is withdrawn from the active set 86 Mp-QP Properties Complexity Reduction.5.5 CR CR CR3 CR CR5 CR6 CR7 CR8 CR9.5.5 CR CR CR3 CR CR5 CR6 CR7.5.5 continuous, piecewise affine convex, continuous, piecewise quadratic, C (if no degeneracy) -.5 - -.5 - -.5 -.5 - -.5 - -.5.5.5.5 -.5 - -.5 - -.5 -.5 - -.5 - -.5.5.5.5 Corollary: The linear MPC controller is a continuous piecewise affine function of the state Regions where the first component of the solution is the same can be joined (when their union is convex). (Bemporad, Fukuda, Torrisi, Computational Geometry, ) 87 88 Double Integrator Example mp-qp solution System: Constraints: sampling + ZOH T s = s N u = Control objective: minimize Optimization problem: for N u = (cost function is normalized by max svd(h)) go to demo /demos/linear/doubleintexp.m (Hyb-Tbx) 89 9

Complexity N u =3 N u = N u =5 Linearized model: Example: AFTI-6 Section with x=[ ], u=[ ] N u =6 CPU time 5 5 8 6 r # Regions 3 - - -6 5 5 # free moves (is the number of regions finite for N u?) 9 Explicit controller: 8 parameters, 5 regions go to demo linear/afti6.m -8 (Hyb-Tbx) - - -8-6 - - 6 8 r 9 Explicit Hybrid MPC (MLD) Multiparametric MILP min ø={øc,ø d } f ø c + d ø d s.t. Gø c + Eø d ô W + Fx ø c R n ø d {, } m On-line optimization: solve the problem for each given x(t) Mixed-Integer Linear Program (MILP) mp-milp can be solved (by alternating MILPs and mp-lps) (Dua, Pistikopoulos, 999) Theorem: The multiparametric solution ø ã (x) is piecewise affine Off-line optimization: solve the MILP for all x(t) in advance The MPC controller is piecewise affine in x,r (x,r)-space M 3 5 6 multi-parametric Mixed Integer Linear Program (mp-milp) 93 9 Explicit Hybrid MPC (PWA) Solution u(x,r) found via a combination of - Dynamic programming or enumeration of feasible mode sequences, multiparametric linear or quadratic programming, and polyhedral computation. (Borrelli, Baotic, Bemporad, Morari, 3) (Mayne, ECC ) Explicit Hybrid MPC (PWA) Method A: (Borrelli, Baotic, Bemporad, Morari, Automatica, 5) Use a combination of DP (dynamic programming) and mplp (-norm, -norm), or mpqp (quadratic forms) Method B: (Bemporad, Hybrid Toolbox, 3) (Alessio, Bemporad, 5) (Mayne, ECC ) - Use backwards (=DP) reachability analysis for enumerating all feasible mode sequences I={i(),i(),,i(T-)}; - For each fixed sequence I, solve the explicit finite-time optimal control problem for the corresponding linear time-varying system (mpqp or mplp); The MPC controller is piecewise affine in x,r (x,r)-space M 3 - Case / -norm: Compare value functions and split regions. Quadratic case: keep overlapping regions (possibly eliminate overlaps that are never optimal) and compare on-line (if needed). Note: in the -norm case the partition may not be fully polyhedral 3 5 6 95 Note: in the -norm case, the fully explicit partition may not be polyhedral 96

Hybrid Control - Example PWA system: Closed loop: Hybrid Control Example (Revisited) Constraints: Objective: Open loop behavior: 97 HybTbx: /demos/hybrid/bm99sim.m 98 Explicit PWA Controller 8 6 x - - -6-8 - - -8-6 - - 6 8 x PWA law Section with r= ñ MPC law 3 5 Closed loop: Hybrid MPC - Example HybTbx: /demos/hybrid/bm99sim.m (CPU time:.5 s, Pentium M.GHz) Objective: Explicit PWA Regulator 99 Explicit MPC Temperature Control >>E=expcon(C,range,options); >> E 8 6 Prediction Horizon N= CR CR CR3 CR CR5 CR6 CR7 CR8 8 6 Prediction Horizon N= Prediction Horizon N=3 8 6 Explicit controller (based on hybrid controller C) 3 parameter(s) input(s) partition(s) sampling time =.5 The controller is for hybrid systems (tracking) This is a state-feedback controller. x x x Type "struct(e)" for more details. >> - - - - - - -6-6 -6-8 -8-8 - - -8-6 - - 6 8 x - - -8-6 - - 6 8 x - - -8-6 - - 6 8 x Section in the (T,T )-space for T ref = 3 HybTbx: /demos/hybrid/bm99benchmark.m

Explicit MPC Temperature Control Comments on Explicit Solutions Alternatives: either () solve an MIP on-line or () evaluate a PWA function For problems with many variables and/or long horizons: MIP may be preferable Generated C-code utils/expcon.h For simple problems (short horizon/few constraints): - time to evaluate the control law is shorter than MIP - control code is simpler (no complex solver must be included in the control software!) - more insight in controller s behavior physical system hybrid model Hybrid Control Design Flow simulation and validation control objectives & constraints optimal control setup (on-line MIP) simulation and validation 3 multiparametric solver PWA control law Features: Hybrid Toolbox for Matlab Hybrid model (MLD and PWA) design and simulation Control design for linear systems w/ constraints and hybrid systems (on-line optimization via QP/MILP/MIQP) Explicit control (via multiparametric programming) C-code generation Simulink (Bemporad, 3-) implementation (C code) and experiments yes no too complex? 5 http://www.dii.unisi.it/hybrid/toolbox 6 State Estimation / Fault Detection Problem: given past output measurements and inputs, estimate the current state/faults Solution: Use Moving Horizon Estimation for MLD systems (dual of MPC) Augment the MLD model with: State Estimation / Fault Detection Input disturbances Output disturbances ø R n ð R p min P T At each time t solve kyê(t à k t) à y(t à k) k +... and get estimates xê(t) the optimization: k= MHE optimization = MIQP Convergence can be guaranteed (Bemporad, Mignone, Morari, ACC 999) (Ferrari-T., Mignone, Morari, ) 7 Fault detection: augment MLD with unknown binary distubances þ {,} p 8

Example: Three Tank System COSY Benchmark problem, ESF Example: Three Tank System COSY Benchmark problem, ESF þ : leak in tank for s ô t ô 6s þ : valve V blocked for t õ s þ : leak in tank for s ô t ô 6s þ : valve V blocked for t õ s Add logic constraint [h ô h v ] þ = 9 Outline Model Predictive Control (MPC) concepts Hybrid models for MPC MPC of hybrid systems Explicit MPC (multiparametric programming) Optimization-based reachability analysis Examples Optimization-based Reachability Analysis (Verification of Safety Properties via Mixed-integer integer Programming) Hybrid Verification Problem continuous disturbances hybrid process continuous states Verification Algorithm QUERY: Is the target set X f reachable after N steps from some initial state x X for some input profile u U? Computation: Solve the mixed-integer linear program (MILP) binary disturbances binary states verification algorithm counterexample always safe! answer safety query set of possible initial states set of possible disturbances 3 with respect to u(),δ(),z(),, u(n-),δ(n-),z(n-),x() Alternative solutions: Exploit the special structure of the problem and use polyhedral computation. (Torrisi, 3) Use abstractions (LPs) + SAT solvers (Giorgetti, Pappas, 5)

Verification Example Verification Example MLD model: room temperature system (set of unsafe states) (set of initial states) (set of possible inputs) N= (time horizon) >>[flag,x,u]=reach(s,n,xf,x,umin,umax); 5 6 Verification Example Verification Example MLD model: semiactive suspension system (set of unsafe states) (set of initial states) N= (set of possible inputs) (time horizon) >>[flag,x,u]=reach(s,n,xf,x,umin,umax); 7 8 Integral Switched Affine System Event Generator Integral Switched Affine System (Bemporad, Di Cairano, Julvez, 5) Integral dynamics: Finite State Machine time s Event-Driven / Continuous-Time Hybrid Automata Mode Selector mode k = event index t k = event instant Change of variables: 9 Same as discrete-time dynamics!

Event Generator Asynchronous Finite State Machine Event Generator Events occur at time t if: Event Generator Integral Switched Affine System () or Integral Switched Affine System Finite State Machine time s () or Finite State Machine time s Mode Selector mode (3) Mode Selector mode Asynchronous FSM dynamics: xc(t) h jx c + h jx c >w j h j x c + h j x c <w j x c(t k ) h jx c + h jx c = w j Example: for some threshold index j Discrete state transitions, events, and mode switches might occur at any (continuous) time t! Hybrid Automaton with Integral Continuous(-time) time) Dynamics (icha( icha) Wrapping up: event-driven (asynchronous) FSM + integral dynamics = same as discrete hybrid automaton Event Generator Integral Switched Affine System Finite State tim Machine e s mod Mode Selector e Outline Model Predictive Control (MPC) concepts Hybrid models for MPC MPC of hybrid systems Explicit MPC (multiparametric programming) Optimization-based reachability analysis Examples states inputs All numerical control/verification techniques developed for DHA can be immediately applied to icha! 3 Hybrid Control Problem Hybrid Control Example: Cruise Control System GOAL: command gear ratio, gas pedal, and brakes to track a desired speed and minimize consumption 5 6

Hybrid Model Hybrid Model Vehicle dynamics Engine torque à C à e (ω) ô M ô C+ e (ω) mẍ =F e à F b à ìxç xç = vehicle speed Max engine torque C + e (ω) F e = traction force F b = brake force 8 discretized with sampling time T s =.5 s Transmission kinematics Piecewise-linearization: (PWL Toolbox, Julián, 999) 6 R ω = g(i) ks xç F e = R g(i) ks M ω M i = engine speed = engine torque = gear requires: binary aux variables continuous aux variables Min engine torque 8 6 C à e (ω) =ë + ì ω 3 7 8 Hybrid Model Hysdel Model Gear selection: define a binary input for each gear #i, g i {, } Gear selection (traction force): F e = R g(i) ks M depends on gear #i define auxiliary continuous variables: R IF g i = THEN F ei = g(i) M ELSE ks F e = F er + F e + F e + F e3 + F e + F e5 Gear selection (engine/vehicle speed): R ω = g(i) ks xç similarly, also requires 6 auxiliary continuous variables 9 go to demo /demos/cruise/init.m 3 MLD model Hybrid Model x(t +)=Ax(t)+B u(t) + B î(t)+b 3 z(t) y(t) =Cx(t)+D u(t) + D î(t)+d 3 z(t) E î(t)+e 3 z(t) ô E x(t)+e u(t)+e 5 continuous states: continuous inputs: 6 binary inputs: continuous output: x, v M, F b g R,g,g,g 3,g,g 5 v (vehicle position and speed) (engine torque, brake force) (gears) (vehicle speed) 6 auxiliary continuous vars: (6 traction force, 6 engine speed, PWL max engine torque) auxiliary binary vars: (PWL max engine torque breakpoints) 96 mixed-integer inequalities Max-speed controller max J(u t,x(t)),v(t + t) ut ú MLD model subj. to x(t t) =x(t) MILP optimization problem Linear constraints Continuous variables Binary variables Parameters Time to solve mp- MILP (Sun Ultra ) Number of regions Hybrid Controller 96 8 5 s v(t) Objective: maximize speed (to reproduce max acceleration plots) 5 5 5 x(t) ( x(t) is irrelevant) 3 (Parameters: Renault Clio.9 DTI RXE) 3

Max-speed controller Hybrid Controller Tracking controller Hybrid Controller 5 Velocity (km/h) 5 3 Gear Fraction of Max Torque (Nm).5.5 Brakes (Nm) min J(u t,x(t)), v(t + t) à v d (t) + ú ω ut ú MLD model subj. to x(t t) =x(t) 5 5 -.5 -.5-5 5 5 Road Slope (deg) Engine speed (rpm) Engine Torque (Nm) 6 5.5 5 3 -.5 5-5 Power (kw) 6 5 3 MILP optimization problem Linear constraints Continuous variables Binary variables Parameters Time to solve mp-milp (PC 85Mhz) Number of regions 98 9 3 s 9 v d (t) 5 5 8 6 v(t) - 5 5 5 5 33 go to demo /demos/cruise/init_exp.m 3 Tracking controller Hybrid Controller min ut J(u t,x(t)), v(t + t) à v d(t) + ú ω Hybrid Controller Smoother tracking controller Velocity (km/h), Desired velocity (km/h) Gear Fraction of Max Torque (Nm) Brakes (Nm) 5 8 8.5 6 6 3 -.5 ú =. - Road Slope (deg) Engine speed (rpm) Engine Torque (Nm) Power (kw) 6 6 5 5 3 - - -5 - -6 - - min J(u t,x(t)), v(t + t) à v d (t) + ú ω ut v(t + t) à v(t) <T s a max subj. to MLD model x(t t) =x(t) MILP optimization problem Linear constraints Continuous variables Binary variables Parameters Time to solve mp-milp (PC 85Mhz) Number of regions 9 7 s 5 v d (t) 5 5 5 8 6 v(t) 35 36 Hybrid Controller Smoother tracking controller Velocity (km/h), Desired velocity (km/h) 5 Gear Fraction of Max Torque (Nm) Brakes (Nm) 8.5 8 6 6 3 -.5 - Traction Control System 6 Road Slope (deg) 6 Engine speed (rpm) Engine Torque (Nm) Power (kw) 5 5 (joint work with F. Borrelli,, M. Fodor, D. Hrovat) 3 - - -5 - -6 - - 37 38

Vehicle Traction Control Tire Force Characteristics Improve driver s ability to control a vehicle under adverse external conditions (wet or icy roads) Longitudinal Force Lateral Force Tire Forces Slip Target Zone Model nonlinear, uncertain, constraints Controller suitable for real-time implementation Maximum Braking Maximum Cornering Maximum Acceleration Steer Angle Tire Slip Longitudinal Force Lateral Force MLD hybrid framework + optimization-based control strategy 39 Simple Traction Model Hybrid Model Mechanical system ò ó ü t ωç e = ü c à b e ω e à Je gr ü t vç v = mv r t Manifold/fueling dynamics ü c = b i ü d (t à ü f ) Torque μ Slip Nonlinear tire torque τ t =f(δω, μ) PWA Approximation (PWL Toolbox, Julian, 999) Tire torque τ t is a function of slip Δω and road surface adhesion coefficient μ ω e v v Δω = à gr rt Wheel slip Torque μ Slip HYSDEL (Hybrid Systems Description Language) Mixed-Logical Dynamical (MLD) Hybrid Model (discrete time) Δω MLD Model Performance and Constraints Control objective: min P N k= subj. to. Δω(k t) à Δω des MLD Dynamics State x(t) variables Input u(t) variable Aux. Binary vars δ(t) variable Aux. Continuous vars z(t) 3 variables Mixed-integer inequalities Constraints: Limits on the engine torque: à Nm ô ü d ô 76Nm Logic Constraint: Hysteresis The MLD matrices are automatically generated in Matlab format by HYSDEL go to demo /demos/traction/init.m 3

Experimental Results Experiment Wheel Speeds, [rad/s] 5 5 Engine Torque Command, [Nm] 5 5 3 6 9 3 Controller Region 5 5 5 3 6 9 Time, [s] 5 regions ms sampling time Pentium 66Mhz + Labview 5 6 DISC Engine Control Problem Goal: Develop a controller for a Direct-Injection Stratified Charge (DISC) engine. Main features of DISC engines: Hybrid Control of a DISC Engine (joint work with N. Giorgetti, I. Kolmanovsky and D. Hrovat) Direct injection of fuel (as Diesel engines); Two operating modes (homogeneous/stratified); Reduction of consumptions up to 5%; Highly complex post-treatment system of exhaust gas; Different control objectives and constraints that depend on the operating mode (Photo: Courtesy Mitsubishi) Strategy: Develop a hybrid model that captures the two operating modes; Design a constrained MPC controller based on the hybrid model 7 8 Two distinct regimes: Regime Homogeneous combustion Stratified combustion DISC Engine Mode is switched by changing fuel injection timing (late / early) Better fuel economy during stratified mode Periodical cleaning of the aftertreatment system needed (λ=., homogeneous regime) normal fuel injection intake stroke compression stroke purge air-tofuel ratio λ=.6 λ.6 the stratified operation can only be sustained in a restricted part of the engine operating range 9 DISC Engine States/Controlled outputs: Intake manifold pressure (p m ); Air-to-fuel ratio (λ); Engine brake torque (τ); Inputs (continuous): W th Air mass flow rate through throttle (W th ); Mass flow rate of fuel (W f ); Spark timing (δ); Inputs (binary): ρ = regime of combustion (homogeneous/stratified); Constraints on: Air-to-fuel ratio (due to engine roughness, misfiring, smoke emiss.) Spark timing (to avoid excessive engine roughness) Mass flow rate on intake manifold (constraints on throttle) Dynamic equations are nonlinear Dynamics and constraints depend on regime ρ ρ p m W f λ δ τ 5

DISC Dynamics Nonlinear model of the engine developed and validated at Ford Research (Kolmanovsky, Sun, ) Assumptions: - no EGR (exhaust gas recirculation) rate, - engine speed= rpm. Hybridization of DISC Model DYNAMICS (intake pressure, air-to-fuel ratio, torque): Definition of two operating points; Linearization of nonlinear dynamics; Time discretization of the linear models. ρ-dependent dynamic equations Intake manifold pressure: CONSTRAINTS on: In-cylinder Air-to-Fuel ratio: Engine torque: with functions of p m Air-to-Fuel Ratio: λ min (ρ) λ(t) λ max (ρ); Mass of air through the throttle: W th K; Spark timing: δ(t) δ mbt (λ, ρ) ρ dependent constraints where θ a, θ b, δ mbt are functions of λ, δ and ρ. Good for simulation Not suitable for controller synthesis 5 Hybrid system with modes (switching affine system) 5 MPC for Hybrid Systems past future Predicted outputs y(t+k t) Manipulated Inputs u(t+k) t t+ t+t At time t solve with respect to the finite-horizon open-loop, optimal control problem: Model Predictive (MPC) Control Integral Action Integrators on torque error and air-to-fuel ratio error are added to obtain zero offsets in steady-state: brake torque and air-to-fuel references = sampling time Simulation based on nonlinear model confirms zero offsets in steady-state Apply only (discard the remaining optimal inputs); Repeat the whole optimization at time t+ 53 (despite the model mismatch) 5 MPC of DISC Engine DISC Engine - HYSDEL List where: and: N = control horizon x(t) = current state SYSTEM hysdisc{ INTERFACE{ STATE{ REAL pm [,.35]; REAL xtau [-e3, e3]; REAL xlam [-e3, e3]; [, ]; REAL taud REAL lamd [, 6]; } OUTPUT { REAL lambda; /* [, 5]; */ REAL tau; /* [, ]; */ } INPUT{ REAL Wth [,38.58]; REAL Wf [, ]; REAL delta [, ]; BOOL rho; } PARAMETER{ REAL Ts, pm, pm; REAL l, l, lc; REAL l,l,lc; REAL t,t,t3,t,t5; REAL t,t,t3,t,t5; } } IMPLEMENTATION{ AUX{ REAL lam, taul; REAL lmin,lmax; REAL dmbt; } DA{ lam={ IF rho THEN l*pm+l*wf+lc ELSE l*pm+l*wf+lc}; taul={if rho THEN t*pm+t*wf+t3*delta+t*lam+t5 ELSE t*pm+t*wf+t3*delta+t*lam+t5 }; lmin={if rho THEN 3 ELSE 9}; lmax ={IF rho THEN ELSE 38}; dmbt ={IF rho THEN -8.7+3.85*lam ELSE.877+.8*lam}; } CONTINUOUS{ pm=pm*pm+pm*wth; xtau=xtau+ts*(taud-taul); xlam=xlam+ts*(lamd-lam); taud=taud; lamd=lamd; } OUTPUT { lambda=lam; tau=taul; } MUST{ lmin-lam <=; lam-lmax <=; delta-dmbt <=; } }} 55 56

MPC Torque Control Mode Simulation Results Engine Brake Torque Air-to-Fuel Ratio Weights: Solve MIQP problem (mixed-integer quadratic program) to compute u(t) p m λ [Nm] homogeneous Combustion mode Simulation in torque control mode: Control horizon N=; (Purge Lean NOx Trap) r ρ q τ q λ s ετ s ελ W th ρ W f δ τ Simulation time T=.5 s; Sampling time Ts= ms; CPU time for simulation ' 8 s; (on an Intel Centrino. GHz, 6 Mb RAM with Cplex 9.) (prevents unneeded chattering) main emphasis on torque MPC 57 stratified ' 3 ms per time step Not directly implementable! 58 Multiparametric (Explicit) Solution Mixed Integer Quadratic Program Explicit control law: where: Explicit MPC Controller N= (control horizon) 75 partitions On-line optimization: given x(t), solve an MIQP at each t. Off-line optimization: solve the MIQP for all x(t) off line - find feasible switching sequences using reachability analysis - solve sequence of multi-parametric quadratic programs (using the Hybrid Toolbox) The MPC controller is piecewise affine in x,r (x,r)-space M Cross-section by the τ ref -λ ref plane ρ= ρ= Time to compute the explicit MPC: 3.75 s; Simulation time T=.5 s; Sampling time Ts= ms; CPU time for simulation:.5 s ' μs per time step 3 5 6 59 6 Microcontroller Implementation C-code automatically generated by the Hybrid Toolbox Microcontroller Motorola MPC 555 (custom made for Ford) Explicit MPC Controller (N=) Explicit control law: where: N= (control horizon) 5637 partitions 3 Kb memory available Engine Brake Torque Air-to-Fuel Ratio Floating point arithmetic 3ms execution time sampling period = ms Implementable! Further reduction of number of partitions possible C-code can be further optimized (Alessio, Bemporad, 5) Closed-loop N= Closed-loop N= Adequate! 6 6

Conclusions Hybrid systems are a modeling framework for automotive control problems where continuous switched dynamics and logic are relevant (and linear models are not enough!) MPC control design handle all performance specs and constraints in a natural and direct way. Quite complex systems can be controlled using on-line optimization Piecewise affine MPC controllers can be synthesized, off-line, and implemented as look-up tables of linear gains (x,r)-space Matlab tools available to assist the whole design process (models, simulation, MPC design, code generation): MPC Toolbox (linear), Hybrid Toolbox (hybrid, explicit), Multi-Parametric Toolbox (PWA, explicit) M 5 6 3 63 Key References [] A. Bemporad, Modeling, control, and reachability analysis of discrete-time hybrid systems, Lecture Notes - DISC School on Hybrid Systems, Mar. 3. [] F.D. Torrisi and A. Bemporad, HYSDEL - A tool for generating computational hybrid models, IEEE Trans. on Control Systems Technology, vol., no., pp. 35-9, Mar. [3] A. Bemporad and M. Morari, Control of systems integrating logic, dynamics, and constraints, Automatica, vol. 35, no. 3, pp. 7-7, Mar. 999. [] A. Bemporad, Efficient conversion of mixed logical dynamical systems into an equivalent piecewise affine form, IEEE Trans. Automatic Control, vol. 9, no. 5, pp. 83-838,. [5] W.P.H.M Heemels, B. de Schutter, and A. Bemporad, Equivalence of hybrid dynamical models, Automatica, vol. 37, no. 7, pp. 85-9, July [6] A. Bemporad, M. Morari, V. Dua, and E.N. Pistikopoulos, The explicit linear quadratic regulator for constrained systems, Automatica, vol. 38, no., pp. 3-,. more on http://www.dii.unisi.it/~bemporad/publications 6 Key References [7] A. Bemporad, F. Borrelli, and M. Morari, Piecewise linear optimal controllers for hybrid systems, In Proc. American Control Conference,, pp. 9-9. The End [8] F. Borrelli, M. Baotic, A. Bemporad, and M. Morari, Dynamic programming for constrained optimal control of discrete-time linear hybrid systems, Automatica, vol., no., Oct. 5. [9] A. Bemporad and N. Giorgetti, Logic-based methods for optimal control of hybrid systems, IEEE Trans. Autom. Control, 5, to appear [] A. Bemporad, D. Mignone, and M. Morari, Moving horizon estimation for hybrid systems and fault detection, in Proc. American Control Conf., 999, Chicago, IL, pp. 7-75. [] A. Bemporad, A. Garulli, S. Paoletti, and A. Vicino, A bounded-error approach to piecewise affine system identification, IEEE Trans. Automatic Control, 5, to appear. [] A. Bemporad, Hybrid Toolbox User s Guide, Dec. 3, http://www.dii.unisi.it/hybrid/toolbox. [3] M. Lazar, M. Heemels, S. Weiland, A. Bemporad, Stability of Hybrid Model Predictive Control, IEEE Trans. Automatic Control, 5, submitted. more on http://www.dii.unisi.it/~bemporad/publications 65 MPC controller - SIMO DC-Servomotor Hybrid Toolbox http://www.dii.unisi.it/hybrid/toolbox