Control with Random Communication Delays via a Discrete-Time Jump System Approach

Control with Random Communication Delays via a Discrete-Time Jum System Aroach Lin Xiao Arash Hassibi Jonathan P How Information Systems Laboratory Stanford University Stanford, CA 9435, USA Abstract Digital control systems with random but bounded delays in the feedback loo can be modeled as finite-dimensional, discrete-time jum linear systems, with the transition jums being modeled as finite-state Markov chains This tye of system can be called a stochastic hybrid system Due to the structure of the augmented state-sace model, control of such a system is an outut feedback roblem, even if a state feedback law is intended for the original system We roose a V -K iteration algorithm to design switching and non-switching controllers for such systems This algorithm involves an iterative rocess that requires the solution of a convex otimization roblem constrained by linear matrix inequalities at each ste Keywords: V -K iteration, jum linear system, random delays, stochastic stability, linear matrix inequality (LMI, bilinear matrix inequality (BMI Introduction In many comlex systems, articularly those with remote sensors, actuators and rocessors, a communication network can be used to gather sensor data and send control signals However, the utilization of a multi-user network with random demands affecting the network traffic could result in random delays in the feedback loo, from the sensors to the rocessors, and/or from the rocessors to the actuators These delays will deteriorate the system erformance as well as stability This tye of system can be modeled as finite-dimensional, discrete-time jum linear systems [, 5], with the transition jums being modeled as finite-state Markov rocesses The stability results of Markovian jum linear systems are well estabilished [6, 9, ], and state feedback roblem for such systems can be formulated as a convex otimization over a set of LMI s [, ], thus can be very efficiently solved by interior-oint methods [4, ] However, for system with random delays, the augmented discrete-time jum system has a secial structure, which leads to an outut feedback control roblem, even if a state feedback law is intended for the original system This change in the roblem formulation greatly comlicates the control design within the convex otimization framework This aer rooses a V -K iteration algorithm to design switching and non-switching outut feedback stabilizing controllers for such systems This control synthesis algorithm involves an iterative rocess that Contact author E-mail: lxiao@stanfordedu LinXiao is suorted by a Stanford Graduate Fellowshi requires the solution of a convex otimization roblem constrained by LMI s at each ste In section, the general system setting with random communication delays is described and modeled as a jum linear system Stability results for discrete-time jum systems are reviewed in section 3 In section 4, we describe the V -K iteration algorithm used to design stabilizing controllers In section 5, both switching and non-switching controllers are designed for a cart and inverted endulum system, which is robust to random delays in the feedback loo System Modeling Delayed state feedback model First consider the simle system setu in Figure The discrete-time linear time-invariant (LTI lant model is x(k +=Ax(k+Bu(k ( x(k R n, u(k R m There are random but bounded delays from the sensor to the controller The modedeendent switching state feedback control law is u(k =K rs(kx(k r s(k ( {r s(k} is a bounded random integer sequence with r s(k d s <, andd s is the finite delay bound If we augment the state variable x(k = [ x(k T x(k T x(k d s T ] T x(k R (ds+n, then the closed-loo system is x(k +=(Ã + BK rs(k C rs(k x(k (3 A I Ã = I I C rs(k = [ I ] B = and C rs(k has all elements being zero excet for the r s(kth block being an identity matrix Equation (3 corresonds to a discrete-time jum linear system Notice that these equations are in the form of an outut feedback control roblem, even if a state feedback control law ( was intended for the original system ( B

Plant x(k r= r= u(k Plant x(k Generalized Plant u(k Controller Random Delay and / or Package Loss x(k-r(k s Figure : Control over networks 3 3 3 r=3 r= 3 33 Figure : Markovian Jum States Random Delay and / or Package Loss Generalized Controller v(k z(k Controller Random Delay and / or Package Loss y(k Figure 3: Control over networks: the general case One of the difficulties with this aroach is how to model the r s(k sequence One way is to model the transitions of the random delays r s(k as a finite state Markov rocess [, 5] In this case we have Prob{r s(k +=j r s(k =i} = ij (4 i, j d s This model is quite general, communication ackage loss in the network can be included naturally as exlained below The assumtion here is that the controller will always use the most recent data Thus, if we have x(k r s(k at ste k, but there is no new information coming at ste k + (data could be lost or there is longer delay, then we at least have x(k r s(k available for feedback So in our model of the system in Figure, the delay r s(k can increase at most by each ste, and we constrain Prob{r s(k +>r s(k+} = However, the delay r s(k can decrease as many stes as ossible Decrement of r s(k models communication ackage loss in the network, or disregarding old data if we have newer data coming at the same time Hence the structured transition robability matrix is P s = (5 ds,d s d d d d3 dsd s ij and d s j= ij = (6 because each row reresents the transition robabilities from a fixed state to all the states The diagonal elements are the robabilities of data coming in sequence with equal delays The elements above the diagonal are the robabilities of encountering longer delays, and the elements below the diagonal indicate ackage loss or disregarding old data Figure shows a four state transition diagram with such a structure, which clearly shows that we can jum from r = to r = and from all other r s to r =, but we cannot jum directly from r =tor =orr = 3 We can use the mode-deendent switching controller if we know the delay stes on-line, and this is the case if we use time-stamed data in the network communication We will mention how to handle the situation the transition robabilities are uncertain in next section General model with dynamic outut feedback Next we consider a more general model with dynamic feedback controller (see Figure 3 z(k +=Fz(k+Gy(k (7 v(k =Hz(k+Jy(k In this case we use a mode-indeendent controller to simlify the notation More imortantly, because it is hard to redict the delays from the controller to the actuator at the time the control signal is calculated, the modeindeendent controller is robably the most relevant for this alication To roceed, augment the controller state variable z(k =[ z(k T v(k T v(k d a ] T,d a is the finite bound for the random delays r a(k fromthe controller to the actuator The generalized controller can be written as z(k += F z(k+ Gy(k u(k = H ra(k z(k+ K (8 ra(ky(k F G H J F = I G = I { [ ] H if r a(k = H ra(k = [ ] I if r a(k { J if ra(k = K ra(k = if r a(k Here we use the control signals v(k,,v(k d a generated at different stes for analysis and design urose After the controller (7 is designed using the generalized model, we need not to store them in real-time control alications The transition robability matrix P a has the same structure as P s in (5 Combined with the generalized lant model x(k +=Ã x(k+ Bu(k y(k =C C (9 rs(k x(k and using the state variable x =[ x T z T ] T,wecanwrite the generalized closed-loo system dynamics as x =(Ā + B K C ra(k rs(k x(k ( [ ] [ ] Ã B Ā = B = [ ] [ I ] I F G C rs(k = C C K rs(k ra(k = H ra(k K ra(k

This general system is also a jum linear system The Markovian jum arameter now becomes r(k = (r s(k, r a(k, and the transition robability matrix is P = P s P a, denotes the matrix Kronecker roduct [7] So, both static and dynamic feedback with random delays can be formulated as discrete-time jum system control roblems Thus all stability results and design algorithms in following sections aly to both cases 3 Jum System Control In this section we consider the general (closed-loo jum linear system x(k +=A r(k x(k ( x(k R n, the Markovian integer jum arameter r(k {,,d} is described by (4, and A r(k {A,A,,A d } Note this model can reresent (3 or ( in the revious section Define the mode indicator function { I, if r(k =i I i(k =, if r(k i There are two ways ([6], [] to define the mode-deendent covariance matrices (a Mi a (k = E[x(kx(k T I i(k] ( (b Mi b (k = E[x(kx(k T I i(k ] (3 They satisfy the linear recursions M a i (k += jia jmj a (ka T j (4 j= ( Mi b (k +=A i jimj b (k A T i (5 j= resectively All these equations hold for i =,,, d The mode-indeendent covariance matrix is M(k =E[x(kx(k T ]= Mi a (k = i= Mi b (k i= Thesystemismeansquarestableiflim k M(k =regardless of x( A necessary and sufficient condition is lim M i a (k = or k lim M i b (k =,i=,,,d k Theorem 3 [6] The mean square stability of system ( is equivalent to the existence of symmetric ositive definite matrices Q,Q,,Q d satisfying any one of the following 4 conditions: ( A i jiq j A T i <Q i, i =,,d (6 j= This theorem can be roven using the same technique as in [4] age 36-37 One interesting roerty is that stability of every mode is neither sufficient nor necessary for mean square stability of the jum system [9] If the transition robability matrix P is only known to belong to the olytoe Π = Co{P,P,,P t}, thenasufficient and necessary condition for the jum system to be mean square stable for every P Π is that Theorem 3 holds for every vertex P i of the olytoe [] The decay rate is defined as the largest β > such that lim k β k M(k = A lower bound of the decay rate β = /α must satisfy the inequalities (6 9 by relacing Q i or Q j on the right hand side by αq i or αq j 4 V K Iteration Now we consider the roblem of using outut feedback control to stabilize the jum system ( The closed-loo system dynamics are x(k += ( A r(k + B r(k K r(k C r(k x(k ( which can reresent the static feedback case (3 or the dynamic outut feedback case (, and is more general than these cases Since the four conditions in Theorem 3 are equivalent, we can use any one of them to describe our algorithm For examle, condition (9 with a decay rate β = /α for the closed-loo system ( becomes ji(a i + B ik ic i T Q i(a i + B ik ic i <αq j i= Using Schur comlements, it can be written in an equivalent form αq j (A +B K C T Q Q (A +B K C j (A d +B d K d C d T Q d Q Q d (A d +B d K d C d jd Q d > ( whichmustbetrueforj =,,,duseα =forthe stability roblem and α =/β for decay rate β Theseare couled bilinear matrix inequalities (BMI in the variables α, K i and Q i, i =,,,d, and every K i and Q i aear in all of the d + inequalities If we fix α and the K i s, then these equations are LMI s in the Q i s, and vice versa A lower bound for the decay rate β =/α can be found by solving the following otimization roblem A T j 3 4 ( jiq i A j <Q j, j =,,d (7 i= jia jq ja T j <Q i, i =,,d (8 j= jia T i Q ia i <Q j, j =,,d (9 i= minimize α subject to ( for j =,,,d and Q >,,Q d > ( If we fix the K i s, this corresonds to a generalized eigenvalue roblem, which is quasi-convex in α and the Q i s; if we fix the Q i s, it is a eigenvalue roblem, which is convex in α and the K i s Both of these roblems can be solved very efficiently by convex otimization [4]

Note that we want to design a controller for a secified transition robability matrix P However, it is often difficult to get an initial guess that works well for this P So we start with a simle P for which it is easy to design a stabilizing controller, and then change P in an outer-loo as we roceed through the design iteration Now, using a similar idea as in [3], we give the V -K iteration algorithm used to design a stabilizing controller for the system model develoed in Section Design a LQR (or LQG for dynamic outut feedback controller K for the lant ( without considering delays in the loo Let K i = K, fori =,,,d, and initialize the transition robability matrix, let P = P Design Iterations Reeat{ (a V -ste Given the controllers K i, i =,,d, solve the LMI feasibility roblem ( for all j =,,d with α = to find Q i,i=,,d to rove that the K i s stabilize the jum system (b K-ste Given Q i,i=,,d found in the V - ste, solve the eigenvalue roblem ( to find the K i,i=,,d which maximize the decay rate of the closed-loo jum system with resect to the Q i s (c -ste Perturb the transition robability matrix P by adding a small erturbation matrix ij: P P + ij } Until the desired transition robability matrix P is reached or the V -ste is not feasible Since the LQR (LQG controller corresonds to the no delay case, a reasonable initial transition robability matrix is dɛ ɛ ɛ dɛ ɛ ɛ P = dɛ ɛ ɛ To solve it numerically, we relace ji by a very small ositive number ɛ whenever it is zero in our structured transition robability matrix (5 At the same time, we must slightly change the matrix elements in the same row to kee the constraints (6 being satisfied For ɛ small enough, The first V -ste will always be feasible For each succeeding iteration ste we add a small erturbation matrix ij to the transition robability matrix, eg s s =, = s s s s =, s is a small number, say, for examle In the outer-loo of the design iterations, we will kee adding ij to P until ij is reached, and then start adding the next small erturbation matrix to P The small erturbation sequence will not be unique, and we can add small erturbations to several matrix elements at the same time (see the examle in Section 5 One general rincile is to erturb the system from shorter delays to longer delays, ie,, until the desired P is reached Remarks: A more aggressive initial transition robability matrix can be used in the design rocedure, as long as the initial controller is a feasible solution of the V - ste In the V -ste, given the controllers, instead of solving the LMI feasibility roblem (, we could solve the generalized eigenvalue roblem ( We can do more than one V -K iteration for every, ie, change ste (a and (b into an inner V -K iteration loo as used in [3] Using different forms in Theorem 3 or their mixtures can lead to different designs Notice that the outut feedback roblem ( has a secial structured C r(k as in (3, which allow us to use LQR (or LQG design to start the iteration For more general outut feedback control roblem, finding a feasible initial design is very difficult If the Markov jum arameters r(k is not known on-line, a non-switching controller can be designed by adding the constraint K = K = = K d to the K-ste 5 Examles Consider the cart and inverted endulum roblem in Figure 4(a This is a fourth order unstable system The state variables are [ x ẋ θ θ ] T The arameters used are: m =kg, m =5 kg, L =meter, and no friction surfaces The samling time is T s = second Assume the case random delays only exist from the sensor to the controller, and they are bounded by : r(k {,, } The controllers are designed using the linearized model, but the comuter simulation uses the nonlinear dynamics The initial condition for simulation is θ( = rad, andall other initial states are zero 5 State feedback controllers Given the exected transition robability matrix 5 5 P = 3 6 3 6 We want to design both switching and non-switching state feedback controllers for the jum system First we design an LQR state feedback controller using weighting matrices Q x = I 4 for the states and R u = for the control signal We get K = [ 5959 587 346 7349 ] Although this controller can stabilize the system when there is no delay in the feedback loo, it cannot stabilize the system in the mean square sense when random delays are

u θ m m, L x (a Cart and Pendulum u G G c (b Break the loo Figure 4: Cart and Inverted Pendulum Examle added in the loo This can be verified by the fact that K = K = K = K is not a feasible solution to the LMI s ( This fixed LQR controller also fails to stabilize the system in many of the simulation runs (using different random seeds, but not always Next we design a switching state feedback controller using the V -K iteration algorithm described in Section 4 The initial transition robability matrix and the small erturbation matrix used are 499 499 P = 48 5 48 5, = y The erturbation matrix was added in the design outer-loo until the ideal transition robability matrix P was reached There were design iterations and it took less than one hour running on a Sun SPARC work station The modedeendent switching controllers are K = [ 6439 99 393 7356 ] K = [ 5 7637 83 77458 ] K = [ 89 78869 4488 3379 ] Adding the constraint K = K = K to the K-ste in the design iteration, we obtained the mode-indeendent nonswitching controller K = K = K = [ 738 934 773 835 ] Notice that in our augmented state-sace model (3, both A + BK C and A + BK C with the above two controllers have eigenvalues located outside the unit circle, but the jum system is mean square stable Although the switching controller had a slightly better erformance than the nonswitching one in most simulations, these results show that the difference is small 5 Dynamic outut feedback controllers For another transion robability matrix 36 64 P = 36 3 3 (3 36 3 3 which corresonds to a longer average delay in stationary distribution, the above design iteration using state feedback Delays (Ts x and θ 5 5 5 5 x (meter θ (rad 5 5 Time (sec Figure 5: Random Delays and Initial Condition Resonse couldn t reach the new desired transion robability matrix But using the same design technique, we successfully designed a non-switching dynamic outut feedback controller for the system The outut information used is y =[x θ] T WestartedfromanLQGcontroller F = G = 99 67 54 33 585 35 3493 68 7 3 55 475 78 6384 69 835 33 9 6566 48 896 557 39675 H = [ 5969 587 346 7349 ] J = [ ] The initial transition robability matrix and the small erturbation matrix used are 98 P = 98, = 98 The iterative design rocedure rovides a controller which makes the closed-loo system mean square stable 866 673 3343 35 F = 348 58 957 5878 46 39 87 574 4985 66 546 6 G = 73 6876 333 386 48 398 565 774 H = [ 36345 84 63 7737 ] J = [ 38954 4699 ] (4 In this case, there were 3 design iterations and it took more than one day running on the same comuter Figure 5 shows one simulation run of the Markovian jum delays according to the given transition robability matrix,

Oen loo Gain (db 5 5 Final Design LQG design Critical Point Phase Delay and Phase Margin (deg 8 6 4 8 6 4 Phase Delay Phase Margin Phase Delay Phase Margin 9 85 8 75 7 65 6 Oen loo Phase (deg Figure 6: Phase Margins for Stability 3 4 5 6 7 8 9 Average Time Delay (sec Figure 7: Phase Margin versus Average Delay and the initial condition resonse of the closed-loo system using the controller (4 All other controllers designed before cannot stabilize the system To have a better understanding of the design algorithm, we can comare the average hase delay caused by the random time delays and the hase margin of the system To simlify the analysis, we break the loo at the control signal (see Figure 4(b so that a single-inut and single-outut oen-loo system is resulted and we can use intuitive classical control analysis tools Figure 6 shows the log-magnitude versus linear-hase lot of the oen loo system The curve encircles the the critical oint once (the lant has one unstable ole and has two hase margins to be considered for stability We can see that both hase margins and at the two cross over frequencies are increased through the V - K iteration Figure 7 shows the hase margins and average hase delays versus the average time delays corresonding to the transition robability matrices used in each iteration ste To kee the closed-loo system stable, intuitively, the hase margin curves must stay above the corresonding (at the same crossover frequency average hase delay curves, and this is recisely the case shown in Figure 6 An interesting henomenon is that, to kee the system stable, the algorithm has to give u the redundant hase margins at the low crossover frequency to maintain necessary hase margins at the high crossover frequency as the average time delay increases 6 Conclusions This aer rooses a V -K iteration algorithm to design stabilizing controllers for secial structured discrete-time jum linear systems, which are used to model control systems with random but bounded delays in the feedback loo This algorithm involves solving a convex otimization roblem constrained by LMI s at each ste The LMI s for each iteration are obtained by giving a small erturbation to the Markovian transition robability matrix An examle was included to demostrate the effectiveness of the aroach Current work is investigating how to include erformance criteria such as H and H norms into the V -K iteration design method References [] M Ait-Rami and L El Ghaoui: Robust State-feedback Stabilization of Jum Linear Systems via LMIs, Proceedings IFAC Symosium on Robust Control, Setember 994 [] M Ait-Rami and L El Ghaoui: H State-feedback Control of Jum Linear Systems, Proceedings IEEE Conference on Decision and Control, December 995 [3] D Banjerdongchai: Parametric Robust Controller Synthesis using Linear Matrix Inequalities PhD dissertation, Stanford University, 997 [4] S Boyd, L El Ghaoui, E Feron, V Balakrishnan: Linear Matrix Inequalities in System and Control Theory volume 5 of Studies in Alied Mathematics SIAM, Philadelhia, PA, June 994 [5] H Chan and Ü Özgüner: Otimal Control of Systems overacommunicationnetworkwithqueuesviaajum System Aroach In Proc IEEE Conference on Control Alications, 995 [6] O L V Costa and M D Fragoso: Stability Results for Discrete-Time Linear Systems with Markovian Juming Parameters Journal of Mathematical Analysis and Alications 79, 54-78, 993 [7] A Graham: Kronecker Products and Matrix Calculus: with Alications Ellis Horwoods Ltd, England, 98 [8] A Hassibi, S Boyd and J P How: Control of Asynchronous Dynamical Systems with Rate Constraints on Events In Proc IEEE Conf on Decision and Control, 999 [9] Y Ji and H J Chizeck: Jum Linear Quadratic Gaussian Control: Steady-State Solution and Testable Conditions Control Theory and Advanced Technology, Vol 6, No 3, 89-39, 99 [] R Krtolica, Ü Özgüner, H Chan, HGöktas, J Winkelman and M Liubakka: Stability of Linear Feedback Systems with Random Communication Delays Int Jour Control, Vol 59, No 4, 95-953, 994 [] Johan Nilson: Real-Time Control Systems with Delays PhD dissertation, Deartment of Automatic Control, Lund Institute of Technology, 998 [] Shao-Po Wu and S Boyd: SDPSOL: A Parser/Solver for Semidefinite Programming and Determinant Maximization Problems with Matrix Structure User s Guide Version Beta Stanford University, June 996