Moel Embee Conrol: A Meho o Rapily Synhesize Conrollers in a Moeling Environmen Moel Embee Conrol: A Meho o Rapily Synhesize Conrollers in a Moeling Environmen E. D. Tae Michael Sasena Jesse Gohl Michael Tiller Hybri Powerrain Engineering, General Moors Corp. 187 Troy Tech Par, Troy, Michigan, 489 Emmesay, Inc, 47119 Five Mile Roa Plymouh, Michigan, 4817 e..ae@gm.com msasena@emmesay.com jbgohl@emmesay.com miller@emmesay.com Absrac One of he challenges in moeling complex sysems is he creaion of qualiy conrollers. In some projecs, he effor o evelop even a reasonable prooype conroller warfs he effor require o evelop a physical moel. For a limie class of problems, i is possible an racable o irecly synhesize a conroller from a mahemaical saemen of conrol objecives an a moel of he plan. To o his, a sysem moel is ecompose ino a conrols moel an a plan moel. The conrols moel is furher ecompose ino an opimizaion problem an a zero-ime plan moel. The zero-ime plan moel in he conroller is a copy or a reasonable represenaion of he real plan moel. I is use o evaluae he fuure impac of possible conrol acions. This ype of conroller is referre o as a Moel Embee Conroller (MEC) an can be use o realize conrollers esigne using Dynamic Programming (DP). To illusrae his approach, an approximaion o he problem of saring an engine is consiere. In his problem, an elecric machine wih a flywheel is connece o cran an slier wih a spring aache o he slier. The machine orque is consraine o a value which is insufficien o saically overcome he force of he spring. This consrain prevens he moor from achieving he esire spee from some iniial coniions if i only supplies maximal orque in he esire irecion of roaion. By using DP, a conrol sraegy ha achieves he esire spee from any iniial coniion is generae. This conroller is realize in he moel using MEC. The conroller for his example is creae by forming an opimizaion problem an calling an embee copy of he plan moel. Furhermore, his conroller is calibrae by conucing a large scale Design of Experimens (DOE). The experimens are processe o generae he calibraions for he conroller such ha i achieves is esign objecives when use for close loop conrol of he plan moel. I is well unersoo ha Moelica inclues many language feaures ha allow plan moels o be evelope quicly. As iscusse previously, he evelopmen of qualiy conrol sraegies generally remains a bolenec. In his paper we show how exising feaures along wih appropriae ool suppor an poenial language changes can mae a significan impac on he moel evelopmen process by supporing an auomae conrol synhesis process. Keywors: Conrol, Dynamic Programming, Moel Embee Conrol, Moel Base Conrol, Opimal Conrol 1 Inroucion The use of moeling is well esablishe in he evelopmen of complex proucs. Moern ools have significanly reuce he effor require o moel an une physical sysems. Acausal or opological moeling reuces he effor require o moel a sysem s physics. The use of opimizaion allows sysemaic uning of parameers o improve a esign. The combinaion of parameer opimizaion an The Moelica Associaion 493 Moelica 28, March 3 r 4 h, 28 1
E. Tae, M. Sasena, J. Gohl, M. Tiller rapi moeling allows a large se of poenial esigns o be quicly evaluae. However for sysems which inclue conrols, he evelopmen is, in general, a man-power inensive process subjec o large uncerainy in evelopmen ime an opimaliy. The opimizaion of boh conrols an esign mus be solve in many problems [1-3]. One way o aress his problem is o use numerical echniques o consruc conrollers. For cerain classes of problems, racable numerical echniques can be use o evelop an approximaely minimizing conroller [4]. A minimizing conroller is a conroller which achieves he bes possible performance from a sysem as measure agains an objecive. There may exis more han one conroller able o achieve his minimum, bu no conroller can perform beer han a minimizing conroller. For his wor, he erms minimizing conroller an opimal conroller are use inerchangeably. To consruc a minimizing conroller, an objec cos, J, is efine. This is a funcion which maps he sae an inpu rajecory of he sysem o a scalar: J C x, u. (1) Consier he special case of a plan escribe by orinary ifferenial equaions wih inpus ha are piecewise consan. These piecewise consan inpus are upae perioically a he ecision insances by a conroller a inervals of. The oal operaing cos is calculae as a sum over an infinie ime horizon. Furhermore, he sum of coss is iscoune by he erm which is greaer han zero an less han or equal o one. The oal cos is calculae by an aiive funcion ha operaes on he insananeous sae an he conrol inpus. This cos may ae a form similar o 1 J x c x, u. (2) The oal cos in (2) is a funcion of he iniial sae of he sysem. To simplify noaion, le he sae a he ecision insances be represene by con x x. (3) Le he iscree ime samples occur a. (4) Furhermore, le he coninuous-ime insananeous cos, c con, in (2) be represene in iscree ime noaion as an aiive cos over an inerval, 1 c x, u c x, u. (5) con Using he noaion evelope in (2) hrough (5), he coninuous-ime sysem s oal cos is expresse in iscree ime noaion as J x c x, u. (6) To simplify he coninuous-ime ynamics, le f x, u f, u, x. (7) Hence, x 1 f x, u. (8) An opimal conrol choice for each ime sep can be foun using he ynamic programming equaions, * u x arg min c x, u V f x, u. (9) u U x The funcion V x is nown as he value funcion. By using he ynamic programming (DP) equaions o fin he value funcion, a minimizing conroller is obaine. The DP equaions are V x min c x, u V f x, u, (1) where u U x U x u g x, u (11) efines he se of feasible acions, U x. For he case where he oal cos is consiere over an infinie horizon an u u x (see eq (9)), he * value funcion is he same as he oal cos funcion, J x V x. Equaion (1) can be solve hrough value ieraion, policy ieraion, or linear programming. See [5-23] for iscussion of soluion mehos. For iscussion of using DP o fin value funcions for auomoive conrol applicaion, see [24-29]. The formulaion of equaion (11) is chosen o simplify managemen of consrains hroughou he moel an o conform o a sanar form use in he opimizaion communiy, he negaive null form [3]. One problem wih solving (1) is ha when he sae space consiss of coninuous saes, V x is a funcion from one infinie se o anoher. Excep in special cases, his requires approximaion o solve. One common approach is o use linear bases o approximae he value funcion. Possible linear bases inclue he bases for muli-linear inerpolaion, he bases for barycenric inerpolaion, b-splines, an polynomials. See he appenices in [25] for a iscussion of linear bases for ynamic programming. In he case where V x is approximae by a linear basis, The Moelica Associaion 494 Moelica 28, March 3 r 4 h, 28 2
Moel Embee Conrol: A Meho o Rapily Synhesize Conrollers in a Moeling Environmen where T V x x w, (12) x f1 x f2 x fn x. (13) An approximae soluion o (1) is foun by fining he weighs, w, which solve T T x w min c x, u f x, u w. (14) u U x See [5-7] for a iscussion of using linear bases o form he value funcion. I is imporan o unersan ha his conroller is an opimal conroller for he iscree ime case only, when he conroller upaes every secons. In oher uses, he conroller will generally be subopimal. Aiionally, any evelopmen algorihm base on his mehoology will suffer from he curse of imensionaliy [31]. In oher wors, he ime o fin an opimal conroller will increase geomerically wih he size of he plan sae space. As a poin of reference, using a single commercially available PC from 25, a five sae conroller was foun in less han weny four hours. 2 Conroller Developmen To use equaions (2) hrough (14) o evelop a conroller, i is necessary o have a plan moel which inclues he ynamics ( f ), cos funcion ( c ), an consrains ( g ) all couple o an inegraor which can be invoe as a funcion call by a Conrol Design Algorihm (CDA). In aiion, he se of saes for he plan moel an he se of conroller acions mus be specifie o he CDA. For his wor, a cusom wrapper was evelope ha allowe baches of saes an acions o be efficienly evaluae. Each evaluaion reurne he sae a he nex inerval, he cos of operaion for he inerval, an he consrain aciviy over he inerval. To unersan he srucure of he equaions involve in his wor, consier a sysem consising of a plan an a conroller. Wihou loss of generaliy, assume he plan ynamics are escribe by orinary ifferenial equaions x f x, u, (15) where f is a funcion ha escribes he plan ynamics. For noaional simpliciy consier a coninuous ime conroller. Le he conroller be a full sae feebac conroller implemene as a saic mapping, M, from he sae, x, o he acion se, u : u M x. (16) Assuming only a single global minimum exiss, he ynamic programming equaions in (9) can be irecly use for he saic mapping (16). The auonomous ynamics of his sysem are hen escribe by he following equaion x f x,arg min c x, u V f x, u (17) u U x This equaion is hen inegrae o solve for x, x c x s, u (18) f x s,arg min s u U x V f x s, u where x x (19) efines he iniial coniions. To evaluae f from (7), a nese inegraor, which is inepenen of he primary simulaion inegraor, is require. This nese inegraor execues in zero-ime from he perspecive of he primary inegraor. We refer o his as an embee or nese simulaion. Because he nese inegraor is use insie a numeric opimizaion, i will poenially be calle muliple imes a each primary inegraor evaluaion. If f in (18) is expane using (7), he plan ynamics funcion, f, from (15) occurs in wo locaions in x c x s, u f x s,arg min s u U x V f, u, x s where (2) x x (21) The nese copy of he plan ynamics equaions, f, is referre o as he embee or nese moel. In he case where he conroller is moele as upaing perioically, raher han coninuously, he soluion o he opimizaion problem is hel consan beween conroller upaes. The equaion srucure in (2) an he reuse of he plan ynamics funcion, f, offer he abiliy o quicly synhesize conrollers using numerical echniques. However, exising ools mae he implemenaion of his ype of moel problemaic. There are wo primary issues in implemenaion. The firs is execuion efficiency. Few commercial ools have been evelope wih he goal of efficienly solving his class of equaions. Seconly, several The Moelica Associaion 495 Moelica 28, March 3 r 4 h, 28 3
E. Tae, M. Sasena, J. Gohl, M. Tiller commercial moeling environmens mae he efiniion an reuse of he plan moel cumbersome, requiring significan effors uring evelopmen an mainenance. Forunaely, he feaures of Moelica mae he efiniion an reuse of a plan moel manageable. The examples ha follow have been evelope in Dymola, however his general approach has also been use wih Simulin an AMESim. To sysemaically generae a sysem wih an opimal conroller, a moel of he plan is generae. This plan moel is wrappe wih an applicaion programming inerface (API) so a conrol esign algorihm can eermine he sae space, he acion space, he sae a he nex ime sep, he consrain aciviy, an he cos for a given sae an acion. This ineracion beween he plan moel, he API an he conrol esign algorihm is illusrae in Figure 1. The CDA queries he API o eermine he srucure of he sae an acion space. Given his srucure an he configuraion of he CDA, a sequence of DOEs is execue. The DOE aa are use o fin a soluion o (1). For his wor, he value funcion was moele using muli-linear inerpolaion an a soluion o (14) was foun. To simplify coing, value ieraion was use [5, 6] o fin V x. Conrol Design Algorihm (CDA) V x Sae Space Acion Space x,u f x, u g x, u c x, u Figure 1 - Plan Moel API API o expose funcions Plan Moel Once he value funcion is generae, he sysem moel is forme by one of wo mehos. The firs meho is by generaing a looup able ha maps he sae variables o an acion as in (16). The process of generaing a value funcion, fining a mapping equivalen o (9), an realizing a conroller as a mapping (or looup able) is referre o as Inirec Moel Embee Conrol (IMEC). This meho is appropriae for some sysems. Anoher approach, which is more compuaionally expensive, is referre o as Direc Moel Embee Conrol (DMEC). For DMEC, he conroller is realize by forming an opimizaion saemen aroun an embee copy of he plan moel. This srucure is illusrae in Figure 2. x x, uˆ x, uˆ g x, uˆ f Conroller Opimizer API o expose funcions Embee Plan Moel c x, uˆ * u Plan Moel Figure 2 - Direc Moel Embee Conroller Srucure To realize a Direc Moel Embee Conroller (DMEC), wo pieces are ae o he sysem moel. The firs piece is an opimizer which solves (9). This opimizer can be as simple as a Design of Experimens (DOEs) which consiers a fixe se of acions, an selecs one which minimizes (9). For more sophisicaion, if he naure of he problem permis i, a graien-base opimizer can be employe [3, 32, 33]. If he naure of he problem oes no allow soluion using hese ypes of approaches, global solvers can be use [34-36]. Ieally, an opimizaion library shoul suppor boh graien an non-graien mehos for consraine opimizaion problems. As par of his projec, libraries for performing boh DOEs an graien-base opimizaions were implemene enirely in Moelica. However, here are currenly no comparable commercial or public omain libraries available. The secon piece require o implemen a DMEC is he abiliy o invoe a funcion which efficienly iniializes an simulaes, over a shor ime horizon, a se of moels which are copies of he plan moel wih moifie parameers. Because of he srucure of he problem, each ime he conroller execues, muliple embee simulaions will execue. Depening on he naure of he acion se, he number of embee simulaions may vary from as few as wo embee simulaions o several housan embee simulaions. 3 Example Simple Engine Sar To illusrae how hese conceps are use o buil a conroller, consier he problem of saring an inernal combusion engine using an elecric machine The Moelica Associaion 496 Moelica 28, March 3 r 4 h, 28 4
Moel Embee Conrol: A Meho o Rapily Synhesize Conrollers in a Moeling Environmen wih insufficien orque o guaranee he engine complees a revoluion from all possible saionary saring poins. If he iniial posiion of he engine is in a range of angles, he elecric machine will sall. To simplify he moeling, le us assume he engine can be approximae using a cran slier connece o a spring. The sysem moel, shown in Figure 3, consiss of an elecrical moor connece o he cran which connecs hrough he cran slier mechanism o a pison which is subjec o amping from fricion. Ineria is presen in he moor roor, cranshaf an pison. The elecric machine is subjec o consrains on minimum an maximum orque. Figure 3 - Engine Saring Moel The objecive of he conrol sysem is o ensure he engine will overcome he iniial compression orque from any iniial sae an minimize engine sar ime. The oal cos of operaion (wha is being minimize) is expresse mahemaically as he oal ime aen o achieve a spee greaer han or equal o five hunre RPM. Once his spee is achieve, he conroller is eacivae an anoher scheme is use o manage he engine. The oal cos of operaion for his sysem is consiere over an infinie ime horizon an is compue as, 5 rpm J x. (22) 1,oherwise The insananeous cos for his sysem is, 5 rpm c x. (23) 1,oherwise This ype of cos generaes a shores-pah conroller. The conroller will minimize he oal ime o achieve 5 rpm. The oal cos in (22) is uniscoune. Therefore he iscouning facor,, which is visible in (2) is assigne a value of one an omie from he expression. While i is clear ha he sysem has exacly wo saes, hey can be selece somewha arbirarily. For his example, he engine angle an engine spee were selece. Wih hese variables as he saes, he conroller is represene as a saic map from he engine angle an engine spee o he elecric machine orque. u M, (24) The feasible acion se is a single real number, he moor orque, boune by he consrains on moor orque an power. The se of feasible acions is efine by U x u 1 u 1, 1 u 1. (25) The value funcion was represene using mulilinear inerpolaion, see equaion (12). The plan moel was implemene in Moelica. The Conroller Design Algorihm (CDA) was implemene in MATLAB. The CDA invoe funcion calls o a cusom API, similar o Figure 1, applie o he plan moel in Dymola. The CDA solve for he weighs, w, in he value funcion (equaion (12)). This value funcion was use o generae an Inirec Moel Embee Conroller (IMEC) an a Direc Moel Embee Conroller (DMEC). The value funcion generae by he CDA is shown in Figure 4..4.2 1 5 Engine Spee [rpm] Value funcion -5-1 35 Figure 4 - Value funcion 3 25 2 15 1 5 Engine Angle [eg] The IMEC was realize as a wo inpu looup able wih muli-linear inerpolaion on a regular gri. The gri poins in he able were foun by solving (9) using he value funcion generae by he CDA. This conroller was implemene using The Moelica Associaion 497 Moelica 28, March 3 r 4 h, 28 5
E. Tae, M. Sasena, J. Gohl, M. Tiller sanar Moelica componens. The acuaor commans for he IMEC conroller are shown in Figure 5 as a funcion of engine spee an angle. 1-1 1 5 Bes Conrol Choice - u* Engine Spee [rpm] -5-1 35 3 Figure 5 - IMEC conrol able 25 2 15 1 5 Engine Angle [eg] The DMEC was realize by wrapping a copy of he plan moel wih an API similar o he one use for he CDA. A DOE was use o search feasible acions. The resuling coe srucure is ienical o Figure 2. The opimal acion was chosen o minimize (9). For boh of hese conrollers, he problem of saring he engine from any iniial coniion was solve. The soluion involve he couner-inuiive approach of spinning he engine bacwars, hen reversing irecion o allow enough energy o be sore in he ineria o overcome he spring force. From a moel an a conrol objecive, an opimal conroller wih very complex behaviors was numerically generae in less han 1 minues on a single PC (3GHz, 2Gb RAM). Furhermore, a similar problem wih four saes was solve in less han hree hours. Of course he power of his approach can only be realize once a sufficien level of ool suppor is available so ha he ime require o se up he analysis is on he same orer as he soluion ime. Conrol Design Algorihm (CDA). The IMEC conroller was esigne by solving for he bes elecric machine orque for a se of engine angles an spees on a regular gri. For engine saes which occur off his gri, muli-linear inerpolaion was use o calculae he conrol acion. When he IMEC was use in an engine sar simulaion, if he opimal orque ransiione beween posiive an negaive, he inerpolaion cause a smooh change in he orque because of he coninuiy impose by inerpolaion. Alernaively, consier a Direc MEC. Because of he characerisics of he ynamic programming equaions an he value funcion, he opimal choices are eiher full posiive or full negaive orque. This resuls in an insananeous, nonconinuous change in orque. When ploe as in Figure 6, he ifference beween he conrol inpus an he sae evoluion of he sysem can be seen. The inerpolaion ue o he approximaion in he IMEC resuls in arifacs in he conrol acions an a sligh loss of performance in he sysem. Mahemaically his means ha more eail is require o resolve u * x, he funcion ha we are ulimaely rying o formulae, han o resolve V x. There are cases where an IMEC is superior o a DMEC approach (e.g. [26] illusraes jus such a case). In general, an IMEC implemenaion is superior when boh he acion se is coninuous an he opimal acions are coninuous. The DMEC approach is superior when eiher he acion se is iscree or he opimal acions are no coninuous wih respec o he sae. One example where DMEC is clearly superior is where he moor is conrolle by selecing he sae of a swich inverer. In his case, he acion se consiss of a finie se of choices for swich configuraion an he opimal acions are no coninuous wih respec o he sae. 3.1 Direc vs Inirec MEC Ieally, boh an IMEC an DMEC will resul in ienical behaviors. However, ifferences in approximaion schemes an inerpolaion can resuls in appreciable ifferences. In many cases, while Inirec MEC is simpler o realize in a moel, here are goo reasons o implemen a conroller wih he complexiy an compuaional cos of a Direc MEC. As an example, consier he previous problem. The value funcion, V(x), was foun using he Figure 6 - Comparison of IMEC an DMEC resuls The Moelica Associaion 498 Moelica 28, March 3 r 4 h, 28 6
Moel Embee Conrol: A Meho o Rapily Synhesize Conrollers in a Moeling Environmen 4 Implemenaion of opimizaion algorihms One of he challenges in Direc Moel Embee Conrol is he implemenaion of an opimizer. While his wor was performe using a Design of Experimens (DOE) o selec opimal acions, his approach becomes inracable when equaliy consrains an larger imensional acions ses are consiere. Towars he goal of supporing hese classes of problems, a graien-base opimizer was evelope. One of he goals in eveloping his opimizer was o fully implemen he opimizer in Moelica. By fully implemening in Moelica, all of he informaion use by he opimizer woul be accessible for spee improvemens by he compiler. Shoul naive suppor for moel embeing become available, all equaions associae wih a Direc MEC woul be accessible o he compiler for spee improvemen. Aiionally, since he embee simulaions in a DMEC can be compleely ecouple from each oher, simulaion ools coul easily exploi he coarse graine parallelism on muli-core CPUs by running several embee simulaions concurrenly when conucing searches in he opimizer (e.g. line searches an numerical graiens). The opimizer was evelope in Moelica o solve a consraine opimizaion problem which is generally sae in negaive null form [3] as min f s.. g h objecive inequaliies equaliies. (26) To implemen a graien opimizer, he opimizer funcionaliy was separae from he objecive funcion ( f objecive ), he inequaliy consrain funcions ( g inequaliies ), an he equaliy consrain funcions ( h equaliies ). The opimizer was esigne uner he assumpion ha he inequaliy consrain funcions are all in negaive null form: feasible inequaliy consrains are less han or equal o zero. The objecive funcion was assume o be a minimizaion objecive. Since Moelica oes no (ye) suppor he concep of mehos or passing of funcions as argumens, he opimizer was esigne o use saic inheriance. For his reason, he objecive an consrain funcions are replaceable funcions wihin an opimizer pacage. One feaure of his library, ha is no commonly available, is he abiliy o hanle funcions which are unefine over some region. The omain of he objecive an consrains may no be nown a priori. This occurs wih MEC applicaions because he objecive (e.g. equaion (9)) an consrain funcions (e.g. equaion (11)) are ypically evaluae using a solver. The solver may no fin a soluion. Hence, classical algorihms mus be moifie o recover from unefine evaluaions. Implemenaion of his capabiliy was problemaic because of he lac of numeric suppor for a real value which represens he concep of an unefine quaniy. Eiher a naive capabiliy similar o Malab s NaN, or operaor overloaing wih he abiliy o exen a class from real numbers woul have simplifie implemenaion. In his library, Moelica.Consans.inf was use o inicae ha a funcion call was unefine. However, he language specificaion oes no efine behavior for operaions (e.g. aiion, subracion, muliplicaion, ivision) on Moelica.Consans.inf. Therefore, all funcions an saemens which operae on variables ha migh be assigne a value of Moelica.Consans.inf require coniional expressions o ensure expece behavior. While his opimizaion library will no be publicly release, i is available for furher evelopmen. Conac he lea auhor for a copy. 5 Recommenaions While i is possible o realize boh IMEC an DMEC conrollers using Moelica 2.2, he aiion of a sanar opimizaion library an naive suppor for embee moel simulaion woul vasly simplify implemenaion an mainenance. Towars he goal of simplifying implemenaion of MEC, a recommene language improvemen is he aiion of a moel simulae funcion. The funcion woul accep argumens ha specify he moel o simulae, he parameer values o use in each simulaion, he oupus o reurn, an any solver specific seings. The solver shoul be able o be configure o solve boh iniializaion problems an simulaion problems. For efficiency in evaluaion, he funcion shoul suppor boh a scalar an vecor liss of parameers. In aiion o resuls which are associae wih he moel, here shoul be resuls associae wih he solver. These resuls shoul be sufficien o iagnose solver failures. A a minimum, hese shoul inclue he final ime in he evaluaion an an inicaion of wheher he simulaion successfully complee. A sample funcion efi- The Moelica Associaion 499 Moelica 28, March 3 r 4 h, 28 7
E. Tae, M. Sasena, J. Gohl, M. Tiller niion along wih an example invocaion are shown in Figure 7. funcion simulaemoel inpu Sring moelname; inpu Sring paramnames[:]; inpu Sring resulnames[:]; inpu Real paramvalues[:,size(paramnames,1)]; inpu SeingsRecor solverseings; oupu Real resuls[size(paramvalues,1), en simulaemoel; // example call size(resulnames,1)]; [angle, spee, exiconiion, exitime] = simulaemoel( moelname= Library.PlanMoel, paramnames{ w, hea, u }, resulnames= { w, hea, exiconiion, exitime }, paramvalues= [,, -1; 1,, -1; ; 2, 2*pi, 1], solverseings = ); SeingsRecor( soptime=1., fixesep=.1) Figure 7 - Moel evaluaion I is imporan o poin ou ha he goal is o be able o invoe such a funcion from wihin a running moel an no simply as a comman line analysis opion. As previously menione, he abiliy o irecly express such nese simulaion relaionships maes posing MEC problems much easier. If he MEC problem coul also irecly express he opimizaion problem associae wih MEC hen ools coul also bring he unerlying symbolic informaion o bear on efficien graien evaluaion as well. One remaining issue for DMEC problems is he iniializaion of sae variables in he embee moel. For DMEC problems we ypically wan he embee moel o sar a he curren sae of he paren simulaion. Sai anoher way, he curren values of he saes in he paren simulaion shoul be use as iniial coniions in he nese simulaion. Of course, i is possible using he funcion in Figure 7 o esablish such a mapping bu hopefully he language esign group will consier alernaives ha woul be less eious an error prone. 6 Conclusions I is racable o numerically synhesize near opimal (or approximaely minimal) conrollers for many sysems. While in mos cases he sae feebac require for he conrollers may mae hem impracical o eploy, hey can cerainly be use as prooype conrollers ha esablish performance limis for a given esign as well as provie insighs ino conrol laws for proucion conrollers. Furhermore, his approach can easily inegrae ino a combine plan-conroller opimizaion process. This can be one by maing he opimal conroller a funcion of he plan parameers. These opimal conrollers can be realize as looup ables (IMEC) or hrough he use of opimizaion an embee moels (DMEC). An algorihmic approach o conrols synhesis was presene. For his paper, he IMEC an DMEC approaches were applie o an engine saring problem o generae an opimal conroller in an auomae fashion. As his wor has shown, Moelica is a promising echnology for rapi prooyping of subsysem esigns an prooype conrollers. However, lac of suppor for moel embeing maes evelopmen an long erm mainenance problemaic because consierable wor mus be one o implemen his embeing. Lacing any language sanar, his wor will always be ool specific. Furhermore, implemenaion of conrollers which rely on opimizaion suffer from he lac of a sanar opimizaion library. While an opimizaion library was evelope for his wor, i isn pracical for mos users o mae such an invesmen. By aing boh language suppor o express he essenial aspecs of moel embeing an opimizaion iscusse in his paper, Moelica can evolve ino a powerful echnology for sysem evelopmen an opimizaion. References [1] H. K. Fahy,"Combine Plan an Conrol Opimaion: Theory, Sraegies, an Applicaions," Mechanical Engineering, Universiy of Michigan, Ann Arbor, 23. The Moelica Associaion 5 Moelica 28, March 3 r 4 h, 288
Moel Embee Conrol: A Meho o Rapily Synhesize Conrollers in a Moeling Environmen [2] H. K. Fahy, P. Y. Papalambros, A. G. Ulsoy, an D. Hrova, "Nese Plan/Conroller Opimizaion wih Applicaion o Combine Passive/Acive Auomoive Suspensions." [3] H. K. Fahy, J. A. Reyer, P. Y. Papalambros, an A. G. Ulsoy, "On he Coupling beween he Plan an Conroller Opimizaion Problems," in American Conrol Conference, Arlingon, Va, 21. [4] P. R. Kumar an P. Variaya, Sochasic Sysems: Esimaion, Ienificaion an Aapion. Englewoo Cliffs, New Jersey: Prenice Hall, 1986. [5] D. Berseas, Dynamic Programming an Opimal Conrol: Vol 2. Belmon, Mass: Ahena Scienific, 1995. [6] D. P. Berseas, Dynamic Programming an Opimal Conrol: Vol 1. Belmon, Mass: Ahena Scienific, 1995. [7] D. P. Berseas an J. N. Tsisilis, Neuro- Dynamic Programming. Belmon, Mass: Ahena Scienific, 1996. [8] M. A. Tric an S. E. Zin, "A Linear Programming Approach o Solving Sochasic Dynamic Programs," Carnegie Mellon Universiy 1993. [9] M. A. Tric an S. E. Zin, "Spline Approximaions o Value Funcions: A Linear Programming Approach," Macroeconomic Dynamics, pp. 255-277, 1997. [1] D. P. e Farias an B. Van Roy, "The Linear Programming Approach o Approximae Dynamic Programming," Operaions Research, vol. 51, pp. 85-865, November- December 23. [11] V. F. Farias an B. Van Roy, "Teris: Experimens wih he LP Approach o Approximae DP," 24. [12] D. P. e Farias,"The Linear Programming Approach o Approximae Dynamic Programming: Theory an Applicaion," Ph.D. Disseraion, Deparmen of Managemen Science an Engineering, Sanfor Universiy, Palo Alo, Ca, 22. [13] D. Dolgov an K. Labereaux, "Efficien Linear Approximaions o Sochasic Vehicular Collision-Avoiance Problems," in Proceeings of he Secon Inernaional Conference on Informaics in Conrol, Auomaion, an Roboics (ICINCO-5), 25. [14] G. J. Goron, "Sable Funcion Approximaion in Dynamic Programming," January 1995. [15] R. S. Suon an A. G. Baro, Reinforcemen Learning: An Inroucion. Cambrige, Mass: MIT Press, 1999. [16] R. Munos an A. Moore, "Barycenric Inerpolaors for Coninuous Space an Time Reinforcemen Learning," Avances in Neural Informaion Processing Sysems, vol. 11, pp. 124-13, 1998. [17] R. Munos an A. Moore, "Variable Resoluion Discreizaion in Opimal Conrol," Machine Learning, vol. 1, pp. 1-24, 1999. [18] J. M. Lee an J. H. Lee, "Approximae Dynamic Programming Sraegies an Their Applicabiliy for Process Conrol: A Review an Fuure Direcions," Inernaional Journal of Conrol, Auomaion, an Sysems, vol. 2, pp. 263-278, Sepember 24. [19] D. P. e Farias an B. Van Roy, "Approximae Value Ieraion wih Ranomize Policies," in 39h IEEE Conference on Decision an Conrol Suney, Ausralia, 2. [2] D. P. e Farias an B. Van Roy, "Approximae Value Ieraion an Temporal- Difference Learning," in IEEE 2 Aapive Sysems for Signal Processing, Communicaions an Conrol Symposium, 2, pp. 48-51. [21] B. Van Roy an J. N. Tsisilis, "Sable Linear Approximaions o Dynamic Programming for Sochasic Conrol Problems wih Local Transiions," Avances in Neural Informaion Processing Sysems, vol. 8, 1996. [22] P. W. Keller, S. Mannor, an D. Precup, "Auomaic Basis Funcion Consrucion for Approximae Dynamic Programming an Reinforcemen Learning." [23] V. C. P. Chen, D. Rupper, an C. A. Shoemaer, "Applying Experimenal Design an Regression Splines o High Dimensional Coninuous Sae Sochasic Dynamic Programming," Operaions Research, vol. 47, pp. 38-53, January-February 1999. [24] C.-C. Lin, H. Peng, an J. W. Grizzle, "A Sochasic Conrol Sraegy for Hybri Elecric Vehicles," in Proceeings of he 24 American Conrol Conference, 24, pp. 471-4715 vol. 5. [25] E. D. Tae,"Techniques of Hybri Elecic Vehicle Conroller Synhesis," Elecrical Engineering: Sysems, Universiy of Michigan, Ann Arbor, Michigan, 27. [26] E. Tae, J. Grizzle, an H. Peng, "Shores Pah Sochasic Conrol for Hybri Elecric Vehicles," Inernaion Journal of Robus an Nonlinear Conrol, 26. The Moelica Associaion 51 Moelica 28, March 3 r 4 h, 28 9
E. Tae, M. Sasena, J. Gohl, M. Tiller [27] I. Kolmanovsy, I. Siverguina, an B. Lygoe, "Opimizaion of Powerrain Operaing Policy for Feasibiliy Assessmen an Calibraion: Sochasic Dynamic Programming Approach," in Proceeings of he American Conrol Conference, Anchorage, AK, 22, pp. 1425-143. [28] J.-M. Kang, I. Kolmanovsy, an J. W. Grizzle, "Approximae Dynamic Programming Soluions for Lean Burn Engine Aferreamen," in Proceeings of he 38h Conference on Decision & Conrol, Phoenix, Arizona, 1999, pp. 173-178. [29] C.-C. Lin, H. Peng, J. W. Grizzle, an J.-M. Kang, "Power Managemen Sraegy for a Parallel Hybri Elecric Truc," IEEE Transacions on Conrol Sysems Technology, vol. 11, pp. 839-849, November 23. [3] P. Y. Papalambros an D. J. Wile, Principles of Opimal Design: Moels an Compuaion, 2 e. New Yor, New Yor: Cambrige Universiy Press, 2. [31] J. Rus, "Using Ranomizaion o Brea he Curse of Dimensionaliy," 1996. [32] S. Boy an L. Venenberghe, Convex Opimizaion. New Yor, N.Y.: Cambrige Universiy Press, 24. [33] P. E. Gill, W. Murray, an M. H. Wrigh, Pracical Opimizaion. New Yor, N.Y.: Acaemic Press, 1981. [34] D. R. Jones, C. D. Periunen, an B. E. Sucman, "Lipschizian Opimizaion wihou he Lipschiz Consan," Journal of Opimizaion Theory an Applicaions, vol. 79, pp. 157-181, 1993. [35] A. J. Booer, J. Dennis, J. E., P. D. Fran, D. B. Serafini, V. Torczon, an M. W. Trosse, "A Rigorous Framewor for Opimizaion of Expensive Funcions by Surrogaes." [36] M. J. Sasena,"Flexibiliy an Efficiency Enhancemens for Consraine Global Design Opimizaion wih Kriging Approximaions," Mechanical Engineering, Universiy of Michigan, Ann Arbor, 22. The Moelica Associaion 52 Moelica 28, March 3 r 4 h, 28 1