Stochastic Recruitment: A Limited-Feedback Control Policy for Large Ensemble Systems

Transcription

1 Sochasic Recruimen: A Limied-Feedback Conrol Policy for Large Ensemble Sysems Lael Odhner and Harry Asada Absrac This paper is abou sochasic recruimen, a conrol archiecure for cenrally conrolling he ensemble behavior of many idenical agens, in a manner similar o moor recruimen in skeleal muscles. Each agen has a finie se of behaviors, or saes, which can be swiched based on a broadcas command. By swiching randomly beween saes wih a cenrally deermined probabiliy, i is possible o designae he number of agens in each sae. This paper covers sochasic recruimen policies for he case when lile or no feedback is available from he sysem. Feed-forward conrol policies based on rae equilibria are presened, wih an analysis of he performance rade-offs inheren in he problem. Minimal feedback conrol laws are also discussed, and a policy is presened which minimizes he expeced convergence ime of he sysem given only he abiliy o hal he sysem when he desired oupu has been achieved. some inpu given o all agens in he sysem. The auhors are acively working on applying his conrol framework o he problems of endohelial cell migraion and arificial muscle acuaors. Figure 1 illusraes he basic sysem archiecure. I. INTRODUCTION Finding mehods for cenrally conrolling he collecive behavior of many idenical agens is an acive research opic across diverse communiies, including roboics, bioengineering, and conrol. Many differen sysems can be modeled as a swarm of idenical agens, such as baceria, cells, robos in a swarm, or compuers in a nework. In his paper, we will explore conrol of a class of finie sae or hybrid sae agens ha we believe o be applicable o a variey of hese problems. In paricular, we wish o regulae he ensemble disribuion of many agens over heir discree saes, or he fracion of agens having a paricular discree sae. The discree saes in quesion could be asks performed by swarm robos, cell migraion behaviors, or swiched modes governing he ime evoluion of some coninuous sae variable. One of he key componens of he hese regulaory mechanisms is sochasiciy. In biological sysems, sochasiciy is a ubiquious phenomenon ha can be observed in mechanochemical cell and molecular behaviors. For example, angiogenesis cell migraion and issue developmen are ofen reaed as direced random walks, where sochasic behaviors play key roles for developing muli-cellular srucures ha mee morphological and funcional requiremens [1], [2]. Naural sysems are buil upon random processes, and biological sysems in paricular exploi he sochasic naure of heir building blocks. In conrol engineering, randomness has been reaed as unwaned behavior ha should be filered ou or avoided. Excep for communicaion echnology and some of he sysem idenificaion echniques, randomness has no ye been fully exploied as a useful concep. In general, we assume ha he ransiion beween saes can be modeled as a Markov sae ransiion graph whose sae ransiion probabiliies are deermined by The auhors are wih he Mechanical Engineering Deparmen a he Massachuses Insiue of Technology, Cambridge, MA, USA, 2139 {lael, asada}@mi.edu Fig. 1. Many sysems in naure can be hough of as an ensemble of funcionally similar discree-sae agens ha each respond independenly o some global simulus. The ensemble oupu behavior of many such agens is he number of agens in each discree sae. In his paper, we discuss he performance of feed-forward conrol policies for conrolling he number of agens in each discree sae. If rich informaion usable for feedback conrol is available, hen many conrol echniques ha presume fullsae knowledge can be applied. The auhors have shown ha simple feedback policies can be formulaed for conrolling he ensemble sae disribuion based on linear feedback laws [3], expecaion-based conrol laws [4], and dynamic programming [5]. However, he oupu measuremens from real sysems ofen provide only uncerain esimaes of sysem sae disribuion. For example, a robo swarm migh have limied communicaion wih a cenral conrol saion. An arificial muscle composed of many individual sub-unis may (and usually will) exhibi some oupu delay or hyseresis, so ha some esimaing filer mus be used o predic he individual unis saes. Uncerainy of his form will lead o increased uncerainy in he closed-loop behavior of he sysem.we will show ha we can formulae open-loop conrol policies ha cause he sae disribuion o reach a sochasic equilibrium ha is close o any desired sae disribuion, having wellcharacerized variance. The performance of policies depending

2 o minimal or no feedback provide a good bounding case guidelines for examining he performance of closed-loop laws. Depending on he degree of uncerainy in he sae disribuion esimaed from measuremens, i may acually be advanageous o use an open-loop policy insead of a closed-loop policy. In his paper a sochasic cellular sysem wih limied oupu feedback will be formulaed in is simples form. Basic convergence properies and conrol policy will be discussed o gain insighs ino collecive behaviors of sochasic cellular sysems. The framing of he problem is based on cell and sysems biology and moivaed by needs in roboics and conrol. The paper, however, does no aim o apply he resuls direcly o a specific area; insead he objecive is o beer undersand he possible regulaory mechanism ha may govern a large populaion of cellular agens. II. INTRODUCING STOCHASTIC RECRUITMENT In an aemp o invesigae a regulaory mechanism ha works wih limied or no feedback, we look a biological conrol sysems, exrac key feaures from here, and formulae a simple, absrac problem for deailed analysis. One sriking difference from radiional engineered sysems is ha biological cells are living in a we environmen, where signals propagae hrough diffusion. Simuli o he process pervasively affec all he cells involved in he we environmen. I is no likely ha each cell receives a specific conrol signal from a cenral conroller. Raher, he conrol signal, if i exiss, is broadcas in naure. In response o simuli, an imporan propery of biological sysems is ha each cell s behavior is sochasic in naure. Alhough receiving he same simuli, he cell s response is randomly chosen, condiioned on he environmenal facors. Endohelial cell migraion, for example, each cell s movemen is a random process, swiching direcions sochasically [1]. Furhermore, he sae ransiion probabiliies ofen vary depending on he simuli ha individual cells receive. I is known ha biochemical kineics highly depend on emperaure and oher facors, resuling in changes o sae ransiion probabiliies. In he case of endohelial cell migraion, sae ransiion probabiliies are modulaed by he simuli each agen receives as well as he condiions of he we environmen o which i is exposed [2]. These sochasic behaviors of biological cells sugges a new conrol mehodology for engineered sysems; a cenral conroller broadcass signals ha modulae sae ransiion probabiliies o be used a individual agens, or direcly broadcass sae ransiion probabiliies across he cell populaion. Whichever he case, he cenral conroller does no dicae each agen o ake a specific ransiion, bu only specifies he probabiliies of ransiion. The acual conrol acion is up o each cellular agen, bu he collecive behavior may be effecively conrolled wih he cenral conroller. This broadcas conrol proocol will be useful for engineered sysems, since i requires very lile bandwidh, and allows each cellular agen o be compleely anonymous. The auhors have developed several ypes of sochasic broadcas conrol [3], [5], [6]. The major difference is ha he previous conrol archiecure explois feedback of aggregae oupu, while his paper addresses conrol issues wih no feedback or limied feedback. We call his conrol Sochasic Recruimen, which will be formally described nex. Fig. 2. Muscles are composed of many small moor unis, which are eiher relaxed or acivaed, producing force. The ne power and siffness of he muscle depend on he number of recruied moor unis. Fig. 3. A small auomaon having wo saes, ON and OF F, can be commanded o ransiion beween saes wih probabiliies p and q. The concep of recruimen is bes explained by examining he funcion of skeleal muscle. A muscle is no a homogenous collecion of individual cells; raher i is organized as a nework of many small moor unis, which respond auonomously o simulus from he spinal column, as shown in Figure 2. A command from he nervous sysem produces a varied level of force by affecing he percenage of moor unis which conrac. [7]. Each moor uni has a hreshold for response o he nervous exciaion, so ha varying he exciaion being sen o he muscle acs o modulae he force produced [8]. This archiecure can be replicaed in engineered sysems composed of a similar arrangemen of subunis, like an arificial muscle [3]. A cenral conroller broadcass some exciaion o all of he subunis, which change heir behavior a a rae based on he exciaion. The auhors have been using wo-sae agens of his kind o conrol shape memory alloy muscle acuaors made up of many binary unis [5]. Collecively, N small acuaors produce binary oupus which are modulaed so ha varied force and displacemen are produced as a funcion of he number of ON (force-producing) agens, N on. The probabiliy per uni ime of sae ransiions beween he ON and OFF saes are described by parameers p and q, as shown in Figure 3. A. The Dynamics of Randomized Recruimen A key appealing feaure of his sae ransiion framework is is similariy o saisical models for many real-world

3 phenomena, including he dynamics of gene regulaion [9], chemical reacions [1], and even swarms of insecs [11]. I is well undersood how kineic chemical reacion models yield well-described ransien and seady-sae behaviors. These dynamics can be exploied in engineered sysems, since an arificial sysem can be designed o arbirarily vary he sae ransiion probabiliies of is agens. To demonsrae his, we will consider he simples recruimen problem, a sysem made up of N small subsysems or agens having jus wo saes, ON and OFF. The conrol ask is o recrui a specific number of agens, N ref, ino he ON sae. These saes could represen wo differen modaliies or behaviors exhibied by each individual agen. To keep rack of he sae evoluion of he sysem, we will inroduce a discree disribuion variable, x, describing he probabiliy wih which each agen is ON a ime, x = P(sae = ON) (1) Of course, using x as a sufficien saisic for predicing sysem behavior does no guaranee ha N ON can be known exacly. Insead, he likelihood ha k unis are on is calculaed as a funcion of x using a binomial disribuion, ( ) P(N on N = k x ) = x k k (1 x ) N k (2) There are muliple reasons why using x makes more sense han considering N on, if lile feedback informaion is available. Firs and foremos, i is quie simple o predic he fuure behavior of x given an iniial condiion and a broadcas command. In he absence of oher informaion, such as a measured ensemble oupu, his will provide a good guess of long-erm behavior. Second, as N becomes large, he cenral limi heorem will guaranee ha N on will approach is expeced value, E(N on x ) = Nx (3) The evoluion of x can be wrien as a recursive sequence based on he Markov graph parameers p and q, x +1 = (1 q)x + p(1 x ) = (1 p q)x + p (4) The ime evoluion of x can also be wrien as a deerminisic sequence which saisfies (4), ( ) x +1 = x (1 p q) (5) p p + q + p p + q Here x is he iniial likelihood ha an arbirarily seleced uni is in he ON sae. Some physical meaning can be gleaned from (5). The ime-independen erm of he sequence corresponds o he fracion of ON agens a seady sae. This is equal o he probabilisic rae a which agens ransiion from OFF o ON, normalized by he sum of all ransiion raes beween saes, x ss = p p + q (6) The rae a which x exponenially approaches x ss from an iniial condiion x is is represened in he ransien erm, λ = 1 p q (7) These observaions are imporan in formulaing an openloop conrol policy. III. A NO-KNOWLEDGE CONTROL POLICY In previous work, he auhors formulaed closed-loop conrol laws by finding he sae ransiion graph parameers ha minimize he expeced fuure error, E(N+1 on p, q, Non ) = N ref (8) If he cenral conroller has no knowledge of he number of agens in each sae, hen he conrol policy mus produce feedforward dynamics ha move he sae disribuion oward he desired goal. Equaion (6) demonsraed ha he ime evoluion of x has a seady-sae componen. Insead of choosing p and q o minimize he one-sep ahead error condiioned on he presen sae disribuion, p and q could be chosen so ha some desired number of cells N ref is expeced in he seady sae according o (3), E(Nss on p, q) = Nx ss = N p p + q = Nref (9) Many policies saisfy his consrain for any given N ref. For example, seing p =.1 and q =.1 will drive he agens o a 5% likelihood of being in eiher sae. So will seing p = q =.3. In order o disinguish beween hese cases, a scaling analysis can be used o weigh he performance radeoffs beween hese policies. A. Convergence Rae and Seady Sae Disribuion are Independen. The conrol policy parameers p and q can be rewrien as βp and βq, where p +q = 1 and β is a scaling facor ha varies subjec o he consrain ha < p < 1 and < q < 1. When he ransiion probabiliies are scaled in his way, The seady-sae disribuion x ss is independen of β, x ss = βp β(p + q ) = p p + q (1) However, he rae of convergence sill depends on β, 1 p q = 1 β(p + q ) = 1 β (11) This means ha β is a free parameer wih which he convergence ime can be arbirarily varied while sill saisfying he condiion imposed in (9). In he mos exreme case, β is chosen o be 1, so ha λ =. In his case, x converges o p/(p + q) afer only one ime inerval. Figure 4 shows x converging o he same seady-sae behavior from he same iniial condiions, for several values of β.

4 B. Accuracy Varies Only as x ss and N. The oher poin of concern for his conrol sysem is he accuracy of he conrol sysem once i reached seady sae. Specifying x ss does no guaranee ha he number of recruied cells Nss on will converge. Insead, he disribuion from (2) will have some variance. When expressed as a variance normalized by he oal number of unis, his yields a measure of accuracy for recruimen, V ar (Nss on pq /N x ss ) = x ss (1 x ss )N = (p + q) 2 N (12) The β scaling argumen from (1) and (11) can be applied o he variance calculaion. The numeraor and denominaor of (12) boh vary by a facor of β 2, so he variance is independen of he rae a which he acuaor converges o is seady sae probabiliy disribuion, β 2 p q β 2 (p + q ) 2 N = p q (p + q ) 2 N (13) This is an imporan observaion; i means ha nohing is o be gained by aking baby seps, ha is, selecing very small values of p and q in hopes of improving he accuracy of recruimen in exchange for a slower rae of response. The only parameer ha can be varied o improve accuracy is N. number of ransiions can be calculaed condiioned on x, p and q, E(N rans x, p, q) = N(qx + p(1 x )) (14) In he seady sae (9) can be subsiued in, so ha (14) is a funcion of N, p and q, E(Nss rans p, q) = 2Npq p + q (15) Using he scaling argumen again, (15) can be rewrien in erms of βp and βq. This implies ha an increase in β implies more expeced ransiions per uni ime in he seady sae, E(N rans ss βp, βq ) = β2 2Np q β(p + q ) = β(2np q ) (16) The value of β minimizing he number of expeced ransiions is, naurally,, corresponding o he conrol policy ha allows no random ransiions beween sae. D. Generalizaion o Many Saes Boh closed-loop and open-loop conrol laws of his form are no limied o he wo-sae case. In general, he cenrallyspecified sae ransiion probabiliies for a k-sae graph could be represened as a marix M, p 11 p 21 p k1 p 12 p 22 p k2 M =..... (17). p 1k p 2k p kk Each column of M mus add up o 1. These k 2 k independen parameers could be chosen o minimize he expeced error condiioned on he presen knowledge for he number of agens in saes N N+1 k 1, as was done in (8), E(N i +1 M, N 1,... N k ) = N i,ref (18) Fig. 4. The expeced value of N on and probabiliy disribuion of N on a several poins in ime are shown for N = 5, p =.8, q =.2, and β =.2,.5 and 1. This plo illusraes he fac ha he variance of N on is independen of he rae of convergence. All hree cases approach he same probabiliy disribuion in N on. C. The Number of Transiions per Uni Time In a physical sysem, here is ofen a significan energy cos associaed wih swiching agens from one behavior o anoher. For example, a mobile robo swiching beween parolling wo differen areas will expend energy in driving from place o place. A shape memory alloy acuaor has significan laen hea associaed wih he phase ransiion used for acuaion, so spurious phase ransiions are cosly. As a consequence, i may be useful o consider he expeced number of sae ransiions per uni ime when formulaing a conrol policy. The expeced Similarly, (9) can be exended o muliple saes so ha he seady-sae expeced number of agens in each sae is equal o he reference, E(N i ss M) = Nv 1 (M) = N i,ref (19) Here v 1 (M) is he eigenvecor of M having eigenvalue 1. The second-larges eigenvalue will dominae he rae a which he feed-forward policy converges corresponding o he rae found in (7). Solving for hese wo condiions (he seadysae disribuion and he rae of convergence) can be done algebraically or numerically. The oher performance crieria also ranslae nicely o he muli-sae case. The variance of he number of agens in sae i is sill deermined only by he seady sae probabiliy of an agen being in ha sae, v 1i (M), and by he oal number of agens, N, V ar ( N i ss M) = v 1i (M)(1 v 1i (M))N (2)

5 Finally, he expeced number of ransiioning agens from (15) can be wrien for he k-sae case in erms of he seadysae disribuion, E(Nss rans ) = N k v 1i (1 p ii ) (21) i=1 The rade-offs for he muli-sae case are no as easy o quanify nicely. However, he general observaion is ha as he second-larges eigenvalue of M decreases, he convergence ime decreases and he expeced number of ransiions increases. E. The One-Sho Policy So far we have observed ha i is impossible o minimize boh convergence ime and he number of ransiions a seady sae. This performance rade-offs in he no-knowledge, consan policy recruimen problem can be addressed by varying p and q wih ime. Suppose ha we wan o recrui, as accuraely as possible, a specific number of agens, given no knowledge of N on or x, in minimal ime wih minimal seady-sae cos. I was demonsraed above ha he accuracy of recruimen depends only on N and x ss. As a consequence, he accuracy obained by he β = 1 policy will no improve for more han one round of sae ransiions. This moivaes he one-sho, ime-varying recruimen policy: 1) Compue p and q ha saisfies he seady sae condiion from (9), and ses he second-larges eigenvalue λ equal o. 2) Broadcas p and q o all agens. 3) Afer one round of sochasic sae ransiions, broadcas a command o all agens o se p = and q =, so ha all ransiions cease. This policy will guaranee ha he agens ge as close o he desired disribuion as possible, subjec o he variance of N on ss deermined by (13). IV. A MINIMAL KNOWLEDGE FEEDBACK POLICY Ofen, i may be useful o consider he case in which he cenral conroller has limied knowledge of he number of ON agens. Le y be a Boolean measuremen which les he conroller know if he curren disribuion has reached he desired disribuion, or is close enough. One definiion of his could be when exacly he desired number of agens are in he ON sae, { rue, N on = N y = ref false, N on N ref (22) The policy space o be searched is all policies for which a consan command (p, q) is broadcas, unil he minimal feedback measuremens deermine ha he desired sae has been reached. A his poin, all sae ransiions are commanded o cease, by seing p = q =. The broadcas command is assumed o be consan because, in he absence of addiional informaion, here is no good reason for changing he command. This problem will be posed as a sochasic shores pah problem. In his framework, he cos funcion o be minimized in formulaing a policy is he expeced ime ha he sysem akes o converge o he desired sae or oupu. We will formulae he cos J o be minimized as a funcion of he iniial probabiliy ha agens are ON, x, given he addiional knowledge ha he sysem has no converged a ime =. [ ] J = 1 + E g(y ) x (23) =1 Here g(y ) is he cos per sage of he sysem, equal o 1 when he sysem has no ye reached he arge, and when he sysem is already here: { 1, y = false g(y ) = (24),, y = rue A each poin in ime, Bellman s equaion can be used o express he runcaed cos J recursively forward in ime, J = E [g(y ) x ] + E [J +1 x ] (25) Keep in mind ha he sequence {x, x 1, x 2,..., x,...} is deermined by (5), as long as he broadcas command (p, q) is consan. The sochasic naure of he cos funcion arises from he uncerainy abou when he exac number of desired ON agens has been reached. Afer his poin, he cos funcion becomes zero. The pracical resul of his is ha he expecaion of he cos-o-go will be equal o he probabiliy of no converging a ime muliplied by he expeced cos-o-go assuming ha he desired sae has no ye been reached. The likelihood of reaching he desired sae can be described by hinking of he number of ON agens as a Bernoulli variable (he sum of many binary random variables), H = P(y x ) = ( N N ref ) x Nref (1 x ) N Nref (26) Similarly, he likelihood of no reaching he desired sae will be defined as H, H = P(ȳ x ) = 1 H (27) Using his, he expecaion in (25) can be evaluaed, J = H + H J +1 = H [1 + J +1 ] (28) The benefi of rewriing he series represenaion of J recursively is ha each erm in he series carries a muliplicaive erm from he previous ime sep, J = H + H H+1 + H m 1 H+1 H H k J m k= The recursive form of his expression can be used o derive he opimal policy. In order o demonsrae ha a policy locally minimizes his cos funcion, we mus firs find a policy for which he gradien of J wih respec o he policy parameers is zero, indicaing ha he policy lies a an exremum of J in parameer space. As in he no-knowledge case, nohing is known abou he number of ON agens unil he arge

6 disribuion is achieved, so a policy wih consan p and q mus be pursued unil y is rue. The opimal policy o pursue is no hard o imagine; Essenially, i is o re-run he one-sho policy again and again unil y is rue. This can be proved analyically. ( (p, q) = N ref N, 1 Nref N ) (29) The firs condiion is ha he gradien of J wih respec o p and q is zero, for any iniial probabiliy x. J can be expanded recursively forward as a recursive series from any poin in ime using (28), J p = H p [1 + J +1] + H J +1 p (3) The sign of each erm in his series is deermined by he sign of he parial derivaive of H a each poin in ime. The oher erms in he expression are probabiliies, which are posiive, or runcaed cos funcions, which mus also be posiive. The derivaive of H wih respec o x reduces o an expression in erms of H, G = ( Nx N ref) ( p x ) p + q x q (35) As above, he only erms in he series defining he inner produc which can be negaive are wihin G. G can be evaluaed for p = βn ref /N and q = β(1 N ref /N), o obain he expression: G = N(x p ) 2 (1 β) 2 (36) This is greaer han zero for β 1, so he cos funcion is also increasing in ha direcion. G can be shown o be posiive for any small perurbaion in p and q, bu he expressions involved are lenghy and will no be repeaed here. Conour plo of J, N=5, N ref =2 H N ref Nx = H x x (1 x ) (31) p.4 The parial derivaive of H wih respec o p is: H p = H Nx N ref x x (1 x ) p (32) One muliplicaive erm wihin his expression, Nx N ref, is paricularly ineresing. Using he allegedly opimal policy from (29), he value of x can be found for all using (5): β x = { x, = N ref /N, > (33) For all >, Nx N ref is equal o N N ref /N N ref =. Consequenly H / p = for >, and by exension every erm in he series defining J / p. Furhermore, his line of reasoning also works o show ha J / q =, because each erm of he series defining i will also conain a facor of N ref Nx. The ask remaining is o show ha J is increasing as he policy parameers deviae from he opimal policy. I will no be shown here, bu i is sraighforward o demonsrae ha J increases wih even infiniesimal perurbaions in p. As Figure 5, shows, however, he increase in J as β varies is very gradual, and second derivaive ess do no suffice. One approach is o show ha he gradien of J a some finie disance away from he criical poin is always poining away from he opimal policy. This is shown by making sure ha he inner produc beween he vecor disance o he opimal policy and he gradien a ha poin is posiive: ( p, q) J = Z = H [1 + J +1 ] x (1 x ) G + H z +1 (34) The sub-expression G deermines he sign of each erm in he series, Fig. 5. The conour lines of he cos funcion for N = 5, N ref = 2 show he shallowness of he minimum wih respec o he scaling facor β. The cos seeply increases as p is varied for a fixed N ref. In conras, changes in β resul in only modes increases in J. V. COMPUTATIONAL RESULTS A swarm of 5 wo-sae agens was simulaed wih he goal of recruiing 2 agens ino he ON sae. The resuls illusrae he behaviors described above. Simple feed-forward conrol policies were compued, one seing β =.1 and he oherβ = 1. As prediced, he number of ON agens approaches 2 in boh cases. The recruimen behavior of he β =.1 policy seems less random; however, his is he resul of he auo-covariance of he sysem, no a difference in he acual variance of N on. The cumulaive disribuion of values of N on in he seady-sae regime of boh simulaions shows ha he overall disribuion of boh processes is nearly idenical, as shown in Fig. 7 using he cumulaive disribuion funcion of N on for each policy. However, he number of sae ransiions per uni ime, shown in red on Figure 6, is much lower for he β =.1 policy, as prediced by (15). The one-sho policy was simulaed using he same reference as he consan policies, N ref = 2. Figure 8 shows he resul of 2 simulaions. This conrol policy produces consan

7 seady-sae errors, unlike he consan policies. However, he disribuion of hese errors is exacly equal o hose ploed in Fig. 7. Figure 9 demonsraes his using he resul of 1 simulaions. The minimal feedback policy inroduced in Secion IV was simulaed for N ref = 2, showing he ypical convergence behavior. Figure 1 shows a simulaed resul, as well as he expeced he expeced convergence ime o he desired disribuion, 27.5 ime inervals. The auhors previous work found ha he expeced convergence ime under hese same condiions, bu wih full knowledge of N on was abou 4.5 ime inervals [5]. This gives a sense scale when considering he value of sae informaion for feedback in recruimen problems. N on 2 1 Consan Feed Forward Policy, N on Number of ON cells Simulaion of One Sho Policy Time Fig. 8. Tweny simulaion runs of he one-sho policy. This plo illusraes he rade-off made in his conroller. In exchange for a seady-sae error having he same variance as he consan policy, he agens make no sae ransiions a seady sae. N rans Time Seps Consan Feed Forward Policy, N rans β = 1 β = Time Seps Fracion of Simulaions CDF of One Sho Policy Seady Sae Error Half of Simulaed Resuls N ref Seady Sae Error Fig. 6. A simulaion of he consan policy recruimen algorihm. The number of ON cells for N ref = 2 is ploed (blue) for β =.1 and β = 1, wih he number of ransiions per uni ime (red). As prediced, reducing β by a facor of 1 reduces he number of ransiions per uni ime by a facor of 1. CDF 1.5 Cumulaive Disribuion of N on N on β=1 β=.1 Fig. 7. The cumulaive disribuion of N on in seady sae, for he wo simulaions shown in Fig. 6. Despie he difference in convergence rae and smoohness of he ime evoluion, any value of β produces he same cumulaive disribuion. VI. CONCLUSION This paper has demonsraed ha recruiing finie sae agens ino specified saes in specified numbers is possible Fig. 9. A cumulaive disribuion funcion for he seady-sae errors in he one-sho disribuion. This disribuion will approach exacly he same binomial limi disribuion as he consan conrol policies. Number of ON cells Minimal Feedback Policy, N ref =2 Expeced Convergence Time Time Fig. 1. The minimal feedback policy consiss of using he β = 1 consan policy unil some measuremen confirms ha he desired number of agens are recruied; The cenral conroller hen commands all agens o cease any sae ransiions.

8 using sable, feed-forward conrol policies as long as he conroller can find inpus ha saisfy a few eigenvecor and eigenvalue consrains on he sae evoluion equaion. Unlike he previously described closed-loop policies, hese feedforward conrol laws canno cause he sysem o converge. However, he random disribuion of agens among saes will be ighly grouped abou he desired disribuion wih a well-characerized variance. Because he variance is wellcharacerized and relaively independen of he oher performance measures for he sysem, i is sraighforward o assess wheher a closed-loop law or a feed-forward law provides greaer accuracy simply by comparing he variance in he response of he closed-loop policy wih uncerain informaion o he variance of he feed-forward law. This analysis is resriced o very limied knowledge of he sae of agens in he swarm; Pas work on recruimen policies having full knowledge of has yielded good resuls in finding numerically opimal conrol policies. Finding policies for sysems in which parial knowledge is obainable in he form of an esimaing filer, or for sysems wih a limied range of conrol inpus boh remain ineresing fuure direcions. N on REFERENCES [1] N. Manzaris, S. Webb, and H. Ohmer, Mahemaical modeling of umor-induced angiogenesis, Mahemaical Biology, Vol. 49, pp , 24 [2] S. McDougall, R.A. Anderson, and M. Chaplain, Mahemaical modeling of dynamic adapive umor-induced angiogenesis: Clinical implicaions and herapeuic argeing sraegies, Journal of Theoreical Biology, Elsevier, 26. [3] J. Ueda, L. Odhner, S. Kim, H. Asada, Disribued Sochasic Conrol of MEMS-PZT Cellular Acuaors wih Broadcas Feedback, in 26 IEEE-RAS Inernaional Conference on Biomedical Roboics and Biomecharonics, 2-22 February 26, p [4] L.Odhner, J. Ueda, H. Asada, Feedback Conrol of Sochasic Cellular Acuaors, in The 1h Inernaional Symposium on Experimenal Roboics, 26, p [5] L. Odhner, J. Ueda, H. Asada, Sochasic Opimal Conrol Laws for Cellular Arificial Muscles, in 27 Inernaional Conference on Roboics and Auomaion, 1-14 April 27, p [6] J. Ueda, L. Odhner, H. Asada, Broadcas Feedback of Sochasic Cellular Acuaors Inspired by Biological Muscle Conrol, Inernaional J. of Roboics Research, v.26, n.11-12, pp [7] Zajac, Felix, Muscle and Tendon: Properies, Models, Scaling and Applicaion o Biomechanics and Moor Conrol, Cri. Reviews in Biomedical Engineering, v.17, n.4, 1989, pp [8] E. Henneman, G. Somjen, and D.O. Carpener, Funcional significance of cell size in spinal mooneurons, J Neurophysiol, 1965, v. 28, p [9] A. Julius, A. Halasz, V. Kumar, G. Pappas, Conrolling biological sysems: he lacose regulaion sysem of Escherichia coli, in 27 American Conrol Conference, 9-13 July 27 p [1] D. Gillespie, Exac Sochasic Simulaion of Coupled Chemical Reacions, The Journal of Physical Chemisry, Vol, 8 1, No. 25, 1977 [11] S. Berman, A. Halasz, V. Kumar, S. Pra, Bio-Inspired Group Behaviors for he Deploymen of a Swarm of Robos o Muliple Desinaions, in 27 Inernaional Conference on Roboics and Auomaion, 1-14 April 27, p