Using MATLAB in a Graduate Electrical Engineering Optimal Control Course

Using MATLAB in a Graduate Electrical Engineering Optimal Control Course Asad Azemi Department of Electrical Engineering Penn State University Great Valley Campus Malvern, PA 19355-1443 Abstract - Control system design packages like MATLAB, MATRIX X, Control C, SIMNON, etc. have become essential ingredients of both undergraduate and graduate courses in the systems and controls area. This work describes our experience, at the Great Valley Campus of the Pennsylvania State University and the University of Arkansas, with the use of one of these packages, namely MATLAB with its Control Systems, Robust Control and LMI toolboxes in an optimal control course. In this paper we will describe those standard functions of MATLAB and Control Systems Toolbox that are most appropriate for use in an optimal control course. Next, we will focus on some special features of MATLAB that can be used to produce interactive simulations, using user interface controls. With the help of interactive simulations instructors can effectively illustrate the change in system response due to parameter variations. Examples illustrating these features are included. A discussion of how MATLAB helps in reducing the amount of time spent in performing computational homework assignments and the advantages of using graphics user interface control will follow. Finally, the general positive student reaction will be reported. Introduction The Electrical and Engineering Departments at Penn State University and the University of Arkansas are incorporating computer aided engineering (CAE) and computer aided design (CAD) packages into their curricula. The intent of augmenting the curriculum with these packages is to enhance the students theoretical understanding of the material with hands on analysis and design experience. The benefits of CAE and CAD packages in the classroom have been realized by the authors and their co-workers before [1]-[6]. The benefits of using these packages in a university setting is also confirmed by the number of new textbooks, and revisions of previously printed textbooks incorporating new exercises and problems based on these packages, such as [7]-[15]. A summary of the advantages and disadvantages of incorporating these packages into our graduate curricula are presented below. The summary is followed by sections outlining the use of each package in Edwin Engin Yaz Department of Electrical Engineering University of Arkansas Fayetteville, AR 72701 each institution. A summary of the advantages and disadvantages of incorporating these packages into our curriculum are presented below. General Advantages The main advantages of using these tools are: the reinforcement of student understanding of theoretical principles by means of enhanced graphical aids and interactive simulations, analysis of more complex systems that can be treated by pencil and paper, and the instructors ability to assign fairly complex design problems that otherwise would have be unrealistic without the help of such software. Student response concerning the use of these packages is generally favorable. It is also worth mentioning that the use of many CAE packages, such as MATLAB [16], are no longer limited to a a specific filed. Early exposure to these packages will benefit the students. For a more detailed discussion of this topic readers can refer to our previous works [2]-[4]. General Disadvantages Three of the disadvantages of using these packages are the maintenance and operation of these packages on an accessible computer system, the extra work required by students (and instructors) to learn how to use CAE packages, and assuring that the packages are included in the baseline curriculum as part of the required course material. A more detailed discussion of this topic can be found in our previous works [2]-[4]. Optimal Control Systems at Penn State Great Valley Penn State Great Valley Campus, one of the eighteen campuses of the Penn State University, is a graduate center designed to address the educational need of the working engineers in Philadelphia area. Almost all of our students are working engineers, with a wide variety of backgrounds using simulation packages. The goals of this course are to expose the students to the mathematical tools of parametric

and dynamic optimization and their uses in designing optimally behaving dynamic systems. The nonlinear systems course covers the following topics: 1. Static Optimization 2. Optimal Control of Discrete-Time Systems 2.1 Linear Quadratic Regulator 2.2 Steady-State Closed-loop Control of Sub-Optimal Feedback 2.3 The Tracking Problem 2.4 Regulator with Function of Final State Fixed 3. Optimal Control of Continuous-Time Systems 3.1 Linear Quadratic Regulator 3.2 Steady-State Closed-loop Control of Sub-Optimal Feedback 3.3 The Tracking Problem 3.4 Regulator with Function of Final State Fixed 3.5 Final-Time-Free Problem 3.6 Constrained Input Problem 4. Dynamic Programming 4.1 Discrete-Time Systems [14] and [15] are used in teaching the course. Students are given weekly assignments that also include computer simulation/usage. Exams also include a take home part that have computer simulations. Student have access to student versions of MATLAB. Optimal Control Systems at the University of Arkansas This course is offered to advanced undergraduate and graduate students with adequate preparation in classical and state space control techniques. The goals of this course are to expose the students to the mathematical tools of parametric and dynamic optimization and their uses in designing optimally behaving dynamic systems. To accomplish these goals, the author (E. Yaz) has been updating the course materials since 1986 so that the following topics are typically covered in a semester: 1. Unconstrained and constrained optimization problems (1 class) 2. Cost function and parametric optimization (2 classes) 3. Necessary and sufficient conditions for optimality (3 classes) 4. Dynamic programming in continuous- and discrete-time (3 classes) 5. The minimum principle (1 class) 6. Applications of the minimum principle (3 classes) 7. Numerical techniques (2 classes) 8. The continuous- and discrete-time quadratic regulator (LQR) and its variants (6 classes) 9. Linear quadratic tracking and disturbance rejection (2 classes) 10. Robustness properties of LQR (2 classes) 11. Introduction to guaranteed cost, H, and linear matrix inequality methods (3 classes) 12. Exams (2 clauses) Our class is normally 80 minutes of lecture. Typically two in-class are given and the rest of the grading is based on weekly homework assignments and the student portfolio which is composed of the class notes that the student takes and the homework assignment corrections. The portfolios are collected twice, once in time to give mid-semester feedback and also, at the end of the semester, to measure the student s general understanding. The major use of the MATLAB software and its toolboxes is in homework solutions. The textbooks [14],[15], [17], and [18] have been used individually over the years as the recommended textbook for the course together with reference books [19]-[22]. The student responses that are mentioned in the conclusion are based on the standard student evaluations using the Purdue Cafeteria form and also an informal written feedback solicited by E. Yaz. Use of MATLAB in Optimal Control Systems Course In this section we will present those features of the MATLAB that we have found most useful in our classes, based on the aforementioned course outlines. The MATLAB and its Control Systems and Robust Control toolboxes are ideal for solving linear quadratic regulator (H 2 Control) and H control problems. The command lqr ( dqlr in the discrete-time case) used in homework assignments to solve the linear quadratic regulator problems. The syntax for this command is [K, p, ev] = lqr (A, B, Q, R, N) (1) where K is the optimal gain, P is the solution of the algebraic Riccati equation, and ev are the optimal closed loop eigenvalues. The quantities A, B, Q, R, N are respectively the system matrix input matrix, state weighing matrix, control weighting matrix and the weighting in the cross product of the state and control in the quadratic performance index. A general form of the algebraic Riccati equation can be solved by command X = are (A, B, C) (2) Where B = B T P 0 and C = C T. Moreover, since the application of the optimal constant gain feedback u = Kx results in a linear time-invariant system description, stability margins can be found by Bode plots ( bode ). Also, the Nyquist plot ( nyquist (A, B, C, D) )derived by Kalman for single input systems. For the special choice of weighting matrices Q = C T C and R = P T,! > 0 (3)

the Hamiltonian matrix A 1! BB T H = C T C A T is formed. The closed loop system eigenvalues are just the stable eigenvalues of H. Since the characteristic polynomial for H can be put in the root locus form for the parameter!, by using root-locus plots ( rlocus ), one can control the closed loop pole locations. In the worst case or minimax the solution of the standard state feedback H control problem is given by a special algebraic Riccati equation that can be solved by the command are. In other optima robust design of control systems x=(a+)x+bu (5) q where = w i D i E i, w2 minimization of the usual i=1 i > 1 infinite horizon quadratic cost criterion leads to the same type of H type algebraic Riccati equation. More recently, the MATLAB s LMI toolbox [23] became available, that makes the numerical solution of some problems that are not analytically traceable possible. One just example is minimizing an upper bound on the stochastic energy of the state of a multi-mode plant x= A i x + B i u (6) where i = 1,, N. The control of nonlinear systems with successive linearizations during the operation will lead to this kind of problems. Although, there is no known analysis solution, the LMI formulation allows a numerical solution to this convex optimization problem [24]. Interactive Simulations with MATLAB MATLAB is also capable of producing interactive simulations. This can enhance the quality of presentation with graphical simulations. With the help of interactive simulations instructors can effectively illustrate the change in system response due to parameter variations. This helps students gain a better understanding of the subject. Moreover, since there is no need for students to do any programming, this will allow students with limited or no knowledge of MATLAB programming to access features of MATLAB with little investments of time. This feature is essential in an interactive courseware development. Interactive simulations are produced with the help of graphical user interface (GUI) functions. The GUI is made up of graphical objects, such as menus, buttons, lists, and fields. These objects have meanings; when a user "chooses" an object there is an expectation that a certain kind of action will take place. In MATLAB the GUI is implemented using user interface (UI) controls. Figure 1 shows the application of user interface in minimum control energy for a scalar system with fixed final (4) state. The system model and performance index are given by x k+1 = ax k +bu k J 0 = 2 r n 1 k=0 u2 k (7) where x k and u k denote the state and control input, respectively. r is a positive scalar weighting factor. This program will produce plots of the state and the optimal input vs. time. The sliders will allow the user to change the system parameters and observe the resulting changes. Figure 2 illustrates the use of user interface control in a discrete-time fixed final state linear quadratic regulator. The system model and performance index are given by x k+1 = Ax k + Bu k, k > i J i = 1 2 x N T S N x N + 1 N 1 2 k=i x T k Qx k +u T k Ru k, N P0, Q P 0, R > 0 and the optimal control law equations given by K k = B T 1 S k+1 B + R B T S k+1 A, S N given S k = A T S k+1 (A BK k )+Q V K =(A BK k ) T V k+1, V N = C T P k = P k+1 V T k+1 B B T 1 S k+1 B + R B T V k+1, P N = 0 K k u = B T S k+1 B + R 1 B T u k = K k K u k V k+1 P 1 k VT k x k K u k V k+1 P 1 k r N, r N given The system parameters are specified in continuous time and are discretized based on the sampling period T. The program produces plots of output and control input for optimal and suboptimal regulator. The value of T and the final time, t(f), can be changed with the slider bars. The "A", "B", and "C" buttons allow the user to change the system parameters. The "system" button prints system parameters in continuous- and discrete-time, sampling period, poles of the system, and the final time. The "recalc" button updates the plots based on the parameter changes. Although, the GUI is very useful for presenting the changes in system response due to parameter variations, the implementation is very time consuming and requires an extensive knowledge of MATLAB. Conclusion In this paper, we have presented the use of MATLAB and some of its companion toolboxes in our graduate (8)

curriculum, at Penn State University and the University of Arkansas. Several of the advantages provided by computer simulation packages, such as MATLAB, include a reinforcement of student understanding of theoretical principles, allowing assignment of larger, and more complex designs, increased student attentiveness, and enhanced professional development. The main disadvantages of using computer simulation packages are the extra work required of students and instructors, the maintenance and operation of these packages on an accessible computer system, and assuring that the packages are inserted in the baseline curriculum as part of the required course material. Due to the variety of student backgrounds (at Penn State Great Valley Campus), some students have had no prior exposure to the MATLAB which made it difficult for them to finish all the assignments. Using user interface controls will reduce this problem but at the same time will place more burden on the instructor. The general student reaction to the use of and MATLAB has been very positive. [7] [8] [9] [10] [11] [12] Dorf, R., Bishop, R.H. Modern Control Systems, seventh edition, Addison-Wesley Publishing Company, 1995. Franklin, G., Powell, J., Workman, M. Digital Control of Dynamic Systems. Addison-Wesley Publishing Company, 1990. Strum, R., Kirk, D. Contemporary Linear Systems using MATLAB. PWS publishing company. 1994. Saadat, H. Computational Aids in Control Systems Using MATLAB. McGraw-Hill, 1993. Shahian, B., Hassul, M. Control System Design Using MATLAB. Prentice Hall, N.J., 1993. Leonard, N.E., Levine, W.S. Using MATLAB to Analyze and Design Control Systems. Benjamin/Cummings, 1992. [1] References Martin, T.W., Azemi, A., Hewett, D., Schneider, C.P., PSpice in Electrical Engineering Laboratories, Proceedings of the ASEE Annual Conference, 1992, pp. 1307-1308. [13] [14] Ogata, K. Solving Control Engineering Problems with MATLAB. Prentice Hall, N.J., 1994. Hanselman, D.C., Kuo, B.C. MATLAB Tools For Control System Analysis and Design. Prentice Hall, 2nd edition, 1995. [2] [3] Andrews, D., Azemi, A., Charlton, S., Yaz, E., Computer Simulation in Electrical Engineering Education, Proceedings of the ASEE Gulf-Southwest Section Meeting, 1994, pp. 77-82. Azemi, A., Yaz, E., PSpice and MATLAB in Undergraduate and Graduate Electrical Engineering Courses, Proceedings of the 24th Frontiers in Education Conference, 1994, pp. 456-459. [15] [14] [15] Biship, R.H. Modern Control Systems Analysis & Design Using MATLAB & SIMULINK. Addison-Wesley, 1997. Lewis, F.L., et al. Optimal Control. John Wiley & Sons, 2nd edition, 1995. Dorato, P., et al. Linear-Quadratic Control An Introduction. Prentice Hall, 1995. [4] [5] [6] Yaz, E., Azemi, A., Utilizing MATLAB in two Graduate Electrical Engineering Courses, Proceedings of the 25th Frontiers in Education Conference, 1995, pp. 2c6.1-2c6.4. Azemi, A., Stook, C., Utilizing MATLAB in Undergraduate Electric Circuits Courses, Proc. of the 26th Frontier in Education Conference, vol. 1, 1996, pp. 599-603, Salt Lake, UT. Azemi, A., Yaz, E., Utilizing SIMULINK and MATLAB in a Graduate Nonlinear Systems Analysis Course, Proc. of the 26th Frontier in Education Conference, vol. 1, 1996, pp. 595-599, Salt Lake, UT. [16] [17] [18] [19] [20] The MathWorks Inc., 21 Eliot St., South Natick, MA, 01760. Stengel, R.F. Stochastic Optimal Control. Wiley: New York, 1986. Lewis, F.L. Applied Optimal Control and Estimation. Prentice Hall, 1992. Cannon, M.D., et. al. Theory of Optimal Control and Mathematical Programming. McGraw Hill, 1990. Fortmann, T.F., Hitz, K.L. An Introduction to Linear Control Systems. Marcel Dekker, 1997.

[21] Bryson, A.E. Jr., Ho, Y-C. Applied Optimal Control. Hemisphere, 1975. [23] Gahinet, P., et. al. LMI Control Toolbox For Use with MATLAB. Math Works Inc., 1995. [22] Owens, D.H. Multivariable and Optimal Systems. Academic Press: London, 1981. [24] Boyd, S., et. al. LMIs in System and Control Theory. SIAM: Philadelphia, PA, 1994. Figure 1. User interface control for minimum control energy problem. Figure 2. User interface control for fixed final state LQ regulator.