Generating Timed Trajectories for Autonomous Robotic Platforms: A Non-Linear Dynamical Systems Approach

Transcription

1 Generatng Tmed Trajectores for Autonomous Robotc Platforms: A Non-Lnear Dynamcal Systems Approach Crstna Manuela Pexoto dos Santos Open Access Database Introducton Over the last years, there have been consderable efforts to enable robots to perform autonomous tasks n the unpredctable envronments that characterze many potental applcatons. An autonomous system must exhbt flexble behavour that ncludes multple qualtatvely dfferent types of actons and conforms to multple constrants. Classcally, task plannng, path plannng and trajectory control are addressed separately n autonomous robots. Ths separaton between plannng and control mples that space and tme constrants on robot moton must be known before hand wth the hgh degree of precson typcally requred for non-autonomous robot operaton, makng t very dffcult to work n unknown or natural envronments. Moreover, such systems reman nflexble, cannot correct plans onlne, and thus fal both n non-statc envronments such as those n whch robots nteract wth humans, and n dynamc tasks or tme-varyng envronments whch are not hghly controlled and may change overtme, such as those nvolvng ntercepton, mpact or complance. The overall result s a robot system wth lack of responsveness and lmted real-tme capabltes. A reasonable requrement s that robust behavour must be generated n face of uncertan sensors, a dynamcally changng envronment, where there s a contnuous onlne couplng to sensory nformaton. Ths requrement s especally mportant wth the advent of humanod robots, whch consst of bodes wth a hgh number of degrees-of-freedom (DOFs ). Behavor-based approaches to autonomous robotcs were developed to produce tmely robotc responses n dynamc and non-engneered worlds n whch lnkage between percepton and acton s attempted at low levels of sensory nformaton (Arkn, 998). In (Khatb, 986), some of ths plannng s made on-lne n face of the varyng sensoral nformaton. Most current demonstratons of behavor-based robotcs do not address tmng: The tme when a partcular acton s ntated and termnated s not a controlled varable, and s not stablzed aganst perturbatons. When a vehcle, for nstance, takes longer to arrve at a goal because t needed to crcumnavgate an obstacle, ths change of tmng s not compensated for by acceleratng the vehcle along ts path. Tmed actons, by contrast, nvolve stable temporal relatonshps. Stable tmng s mportant when partcular events must be acheved n tme-varyng envronments such as httng or catchng movng Degree-of-freedom (DOF) s the number of dmensons (varables) requred to defne the state of the system. Source: Cuttng Edge Robotcs, ISBN , pp. 784, ARS/plV, Germany, July 5 Edted by: Kordc, V.; Laznca, A. & Merdan, M. 55

2 objects, avodng movng obstacles, or coordnatng multple robots. Moreover, tmng s crtcal n tasks nvolvng sequentally structured actons, n whch subsequent actons must be ntated only once prevous actons have termnated or reached a partcular phase. Ths chapter addresses the problem of generatng tmed trajectores and sequences of movements for robotc manpulators and autonomous vehcles when relatvely low-level, nosy sensoral nformaton s used to ntate and steer acton. The developed archtectures are fully formulated n terms of nonlnear dynamcal systems whch lead to a flexble tmed behavour stably adapted to changng onlne sensory nformaton. The generated trajectores have controlled and stable tmng (lmt cycle type solutons). Incouplng of sensory nformaton enables sensor drven ntaton and termnaton of movement. Specfcally, we address each of the followng questons: (a) Is ths approach suffcently versatle such that a whole varety of rcher forms of behavour, ncludng both rhythmc and dscrete tasks, can be generated through lmt cycle attractors? (b) Can the generated tmed trajectores be compatble wth the requrement of onlne couplng to nosy sensoral nformaton? (c) Is t possble to flexbly generate tmed trajectores comprsng sequence generaton and stably and robust mplement them both n robot arms and n vehcles wth modest computatonal resources? Flexblty means here that f the sensoral context changes such that the prevously generated sequence s no longer approprated a new sequence of behavours, adequate to the current stuaton, emerges. (d) Can the temporal coordnaton between dfferent end-effectors be appled to the robotcs doman such that a tendency to synchronze among two robot arms s acheved? Can the dynamcal systems approach provde a theoretcally based way of tunng the movement parameters? (e) Can the proposed tmng archtecture be ntegrated wth other dynamcal archtectures whch do not explctly parameterze tmng requrements? These questons are answered n postve and shown n a wde varety of experments. We llustrate two stuatons n exemplary smulatons. In one, a smple robot arm ntercepts a movng object and returns to a reference poston thereafter. A second smulaton llustrates two PUMA arms perform straght lne moton n the 3D Cartesan space such that temporal coordnaton of the two arms s acheved. As an mplementaton of the approach, the capacty of a low level vehcle to navgate n a non-structured envronment whle beng capable of reachng a target n an approxmately constant tme s chosen. The evaluaton results llustrate the stablty and flexblty propertes of the tmng archtecture as well as the robustness of the decson-makng mechansm mplemented. Ths chapter wll gve a revew of the state of the art of modellng control systems wth nonlnear dynamc systems, wth a focus on arm and moble robots. Comments on the relatonshp between ths work, smlar approaches and more tradtonal control methods wll be presented and the contrbutons of ths chapter are hghlghted. It wll provde an overvew of the theoretcal concepts requred to extend the use of nonlnear dynamcal systems to temporally dscrete movements and dscuss theoretcal as well as practcal advantages and lmtatons.. Background and Related Work The state of the art descrbed n ths chapter addresses the work of the most relevant peers developng research n bologcally motvated approaches for achevng movement generaton and also addresses a reasonable number of demonstratons n the robotc doman whch use dynamc systems for movement generaton. 56

3 In ths chapter, we descrbe a dynamcal system archtecture to autonomously generate tmed trajectores and sequences of movements as attractor solutons of dynamc systems. The proposed approach s nspred by analoges wth nervous systems, n partcular, by the way rhythmc and dscrete movement patterns are generated n vertebrate anmals (Beer et al., 99; Clark et al., ). The vertebrate motor system has only lttle changed durng evoluton despte the large varety of dfferent morphologes and types of locomoton. These regulartes or nvarants seem to ndcate some fundamental organzatonal prncples n the central nervous systems (Schaal, ). For nstance, the basc movements of anmals, such as walkng, swmmng, breathng, and feedng consst of reproducble and representatve movements of several physcal parts of the body whch are nfluenced by the rhythmc pattern produced n the nervous system. Further, n the presence of a varable envronment, anmals show adaptve behavours whch requre coordnaton of the rhythms of all physcal parts nvolved, whch s mportant to acheve smooth locomoton. Thus, the envronmental changes adjust the dynamcs of locomoton pattern generaton. The tmng of rhythmc actvtes n nervous systems s typcally based on the autonomous generaton of rhythms n specalzed neural networks located n the spnal cord, called central pattern generators (CPGs). Electrcal stmulaton of the bran stem of decerebrated anmals have shown that CPGs requre only very smple sgnals n order to nduce locomoton and even changes of gat patterns (Shk and Orlosky, 966). The dynamc approach offers concepts wth whch ths tmng problem can be addressed. It provdes the theoretcal concepts to ntegrate n a sngle model a theory of movement ntaton, of trajectory generaton over tme and also provdes for ther control. These deas have been formulated and tested as models of bologcal motor control n (Schoner, 994) by mathematcally descrbng CPGs as nonlnear dynamcal systems wth stable lmt cycle (perodc) solutons. Coordnaton among lmbs can be modelled through mutual couplng of such nonlnear oscllators (Schoner & Kelso, 988). Couplng oscllators to generate multple phase-locked oscllaton patterns has snce long tme beng used to mathematcally model anmal behavour n locomoton (Collns, Rchmond, 994), and to formulate mathematcal models of adaptaton to perodc perturbaton n quadruped locomoton (Ito et al, 998). Ths framework s also deal to acheve locomoton by creatng systems that autonomously bfurcate to the dfferent types of gat patterns. The on-lne lnkage to sensory nformaton can be understood through the couplng of these oscllators to tme-varyng sensory nformaton (Schoner, 994). Lmted attempts to extend these theoretcal deas to temporally dscrete movements (e.g., reachng) have been made (Schoner, 99). In ths chapter, these deas are further extended to the autonomous generaton of dscrete movement patterns (Santos, 3). Ths tmng problem s also addressable at the robotcs doman. Whle tme schedules can be developed wthn classcal approaches (e.g., through confguraton-tme space representatons), tmng s more dffcult to control when t must be compatble wth contnuous on-lne couplng to low level and often nosy sensory nformaton whch s used to ntate and steer acton. One type of soluton s to generate tme structure at the level of control. In the Dynamcal Systems approach to autonomous robotcs (Schoner & Dose, 99; Stenhage & Schoner, 998, Large et al, 999, Bcho et al, ), plans are generated from stable states of nonlnear dynamcal systems, nto whch sensory nformaton s fed. Intellgent choce of plannng varables makes t possble to obtan complex trajectores and acton sequences from statonary stable states, whch shft and may even go through 57

4 nstabltes as sensory nformaton changes. Heren, an extenson of ths approach s presented to the tmng of motor acts, and an attractor based two-layer dynamcs s proposed that autonomously generates tmed movement and sequences (Schoner & Santos, ). Ths work s further extended to acheve temporal coordnaton among two DOFs as descrbed n (Santos, 3). Ths coordnaton s acheved by couplng the dynamcs of each DOF. The dea of usng dynamc systems for movement generaton s not new and recent work n the dynamc systems approach n psychology has emphaszed the usefulness of autonomous nonlnear dfferental equatons to descrbe movement behavour. In (Rabert, 986), for nstance, rhythmc acton s generated by nsertng nto dynamc control model terms that stablzed oscllatory solutons. Smlarly, (Schaal & Atkeson, 993) generated rhythmc movements n a robot arm that supported jugglng of a ball by nsertng nto the control system a model of the bouncng ball together wth terms that stablzed stable lmt cycles. Earler, (Buhler et al., 994) obtaned jugglng n a smple manpulator by nsertng nto the control laws terms that endowed the complete system wth a lmt cycle attractor. (Clark et al., ) descrbes a nonlnear oscllator scheme to control autonomous moble robots whch coordnates a sequence of basc behavours n the robot to produce the hgher behavour of foragng for lght. (Wllamson, 998) explots the propertes of a smple oscllator crcut to obtan robust rhythmc robot moton control n a wde varety of tasks. More generally, the nonlnear control approach to locomoton poneered by (Rabert, 986) amounts to usng lmt cycle attractors that emerge from the couplng of a nonlnear dynamcal control system wth the physcal envronment of the robot. A lmtaton of such approaches s that they essentally generate a sngle motor act n rhythmc fashon, and reman lmted wth respect to the ntegraton of multple constrants, and plannng was not performed n the fuller sense. The flexble actvaton of dfferent motor acts n response to user demands or sensed envronmental condtons s more dffcult to acheve from the control level. However, (Schaal & Sternad, ) has been able to generate temporally dscrete movement as well. However, there are very few mplementatons of oscllators for arm and vehcles control. The work presented n ths chapter extends the use of oscllators to tasks both on an arm and on a wheeled vehcle. It also dffers from most of the lterature n that t s mplemented on a real robot. In the feld of robotcs, the proposed approach holds the potental to become a much more powerful strategy for generatng complex movement behavor for systems wth several DOFs than classcal approaches. The nherent autonomy of the appled approach helps to synchronze systems and thus reduces the computatonal requrements for generatng coordnated movement. Ths type of control scheme has a great potental for generatng robust locomoton and movement controllers for robots. The work proposed s novel because t sgnfcantly facltates movement generaton and sequences of movements. Fnally, the approach shows up several appealng propertes, such as percepton-acton couplng and reusablty of the prmtves. The techncal motvaton s that ths framework fnds a great number of applcatons n servce tasks (e.g. replacement of humans n ndustral tasks and unsafe areas, collaboratve work wth a human/robot operator) and wll permt to advance towards better rehabltaton of movement n amputees (e.g. ntellgent and more human lke prostheses). In summary, t s expected that n the long run one can potentally ncrease the applcaton of autonomous robots n tasks for helpng the common ctzen. 58

5 3. The Dynamcal Systems Trajectory Generator In ths secton, we develop an approach to generate rhythmc and dscrete movements. Ths work s nnovatve n the manner how t formalzes and uses movement prmtves, both n the context of bologcal and robotcs research. We apply autonomous dfferental equatons to model the manner how behavours related to locomoton are programmed n the oscllatory feedback systems of central pattern generators n the nervous systems (Schoner, 994). The desred movement s to start at tme t nt n an ntal postural state, x tasknt, and move to a new fnal postural state, x taskfnal, wthn a desred movement tme and keepng that tme stable under varable condtons. Such behavor s what we consder tmed dscrete movement. Fgure llustrates such movement along the X-axs: At tme t nt, the system begns ts tmed movement between x tasknt, and x taskfnal. After a certan movement tme, denomnated MT, the movement stops and the system remans at the x taskfnal poston. In order to understand the concept of dscrete movement, consder a two dof robot arm movng n a plane from an ntal rest poston to a fnal folded one. The mappng between the robot s movement and the dscrete movement s accomplshed through smple coordnate transformatons n whch the value of the tmng varable x s updated. The ntal postural state of dscrete movement s mapped onto the ntal rest poston of the arm. The x oscllatory movement durng movement tme, MT, corresponds to the arm movng from the ntal rest poston to the folded one. The fnal postural poston of the dscrete movement corresponds to the arm n ts fnal folded poston (see Fgure ). We use the dynamc systems approach to model the desred behavour, dscrete movement, as a tme course of the behavoural varables. These varables are generated by dynamcal systems. Fgure. A dscrete movement along the ˆX axs (as ndcated by the black arrow). Mappng of the tmng varable x onto the movement of a two dof robot arm The state of the movement s represented by a sngle varable, x, whch s not drectly related to the spatal poston of the correspondng effector, but rather represents the effector temporal poston wthn the dscrete movement cycle. Ths temporal poston x s then converted by smple coordnate transformatons nto the correspondent spatal poston (Fgure ). Generatng oscllatory solutons requres at least two dynamcal dof. A postural state s a statonary state n whch there s no movement. 59

6 Thus, although only the varable x wll be used to control moton of a relevant robotc task varable, a second auxlary varable, y, s needed to enable the system to undergo perodc moton. We could have select x (nd order), but nstead we select y whch provdes smplcty to equatons that can be easly solved analytcally. The entre dscrete trajectory s sub-dvded nto three dfferent behavours or task constrants: an ntal postural state, a stable perodc movement and a fnal postural state. The mpled nstablty n the swtch between these states does not allow to use local bfurcaton theory and s dffcult to buld a general dynamcal model whch generates ths movement as a stable soluton (Schoner,99). The adopted soluton s to expresse each constrant as behavoural nformaton (Schoner & Dose, 99) captured as ndvdual contrbutons to the dynamcal system. Therefore, the trajectores are generated as stable solutons of the followng dynamcal system, whch conssts on the combned addton of the contrbutons of the ndvdual task constrants, x = unt fnt + uhopf f hopf + u fnal f fnal + gwn () y that can operate n three dynamc regmes controlled by the three neurons u ( = nt; hopf; fnal). These neurons can go on (= ) or off (=), and are also governed by dynamcal systems, descrbed later on. The f nt and f fnal contrbutons are descrbed by dynamcal systems whose solutons are stable fxed pont attractors (postural states), and the f hopf contrbuton generates a lmt cycle attractor soluton. The tmng dynamcs are augmented by a Gaussan whte nose term, gwn, that guarantees escape from unstable states and assures robustness to the system. Postural contrbutons are modelled as attractors of the behavoral dynamcs x x xpost = α post () y y where x s the current x poston of the movement, x post s the x postural poston and y s the y postural poston of the movement (set to zero snce t s an auxlary varable requred to enable the system to undergo a perodc moton). These states are characterzed by a tme scale of. 6 τ post =. α = post The Hopf contrbuton generates the lmt cycle soluton: the perodc stable movement. We use a well-known mathematcal expresson, a normal form of the Hopf bfurcaton (Perko, 99), to model the oscllatory movement between two x values: x αh ω x x = γ ( x + y ) y ω αh y y Ths smple polynomal equaton contans a bfurcaton from a fxed pont to a lmt cycle. We use t because t can be completely solved analytcally, provdng complete control over ts stable states. The lmt cycle soluton s a perodc oscllaton wth cycle tme π αh T = and fnte ampltude, A =. Relaxaton to ths stable soluton occurs at a ω tme scale of: τ osc = α h λ. Further detals regardng the Hopf normal form can be found n (Perko, 99). An advantage of our specfc formulaton s the fact that our system s analytcally treatable to a large extent, whch facltates the specfcaton of parameters such as

7 movement tme, movement extent, or maxmal velocty. Ths analytcal specfcaton s also an nnovatve aspect of our work.. Neural Dynamcs The neuronal dynamcs u ( = nt; fnal; hopf) swtches the tmng dynamcs from the fxed pont regmes nto the oscllatory regme and back. Thus, a sngle dscrete movement act s generated by startng out wth neuron u nt = actvated, the other neurons deactvated ( u hopf = u fnal = ), so that the system s n a postural state. The oscllatory soluton s then stablzed ( u nt = ; u hopf = ). Ths oscllatory soluton s deactvated agan when the effector reaches ts target state, after approxmately a half-cycle of the oscllaton, turnng on the fnal postural state nstead ( u hopf = ; u fnal = ). These varous swtches are generated by the followng compettve dynamcs: ( u fnal + uhopf ) unt gwn v( unt + unt ) uhopf gwn v( u + u ) u gwn 3 αu nt = µ ntunt µ nt unt v + (3) 3 αu hopf = µ hopf uhopf µ hopf uhopf + (4) 3 αu fnal = µ fnalu fnal µ fnal u fnal nt hopf fnal + (5) The frst two terms of each equaton represent the normal form of a degenerate ptchfork bfurcaton. A sngle attractor at u fp = for negatve µ becomes unstable for postve µ, and two new attractors appear at u fp = and u fp = -. We use the absolute value of u as a weght factor n the tmng dynamcs, so that + and - are equvalent on states of a neuron, whle u = s the off state. The thrd term n each equaton s a compettve term, whch destablzes any attractors n whch more than one neuron s on. The neurons, u, are coupled through the parameter υ, named compettve nteracton. For postve µ, all attractors of ths compettve dynamcs have one neuron n an on state, and the other two neurons n the off state. In the compettve case, the parameter µ determnes the compettve advantage of the correspondent behavoral varable. That s, among the u varables, the one wth the largest µ wns (u = ) and s turned on, whle the others competng varables are turned off (u = ). However, for suffcently small dfferences between the dfferent µ values multple outcomes are possble (the system s multstable)( Large et al., 999).. Sequental Actvaton Crtera We desgn functonal forms for parameters µ and υ such that the compettve dynamcs approprately bfurcates to the dfferent types of behavour n any gven stuaton. These bfurcatons happen for certan values of parameters µ and υ, whch are n turn dependent on the envronmental stuaton tself. As the envronmental stuaton changes, the neuronal parameters reflect by desgn these changes causng bfurcatons n the compettve level. To control swtchng, the parameters, µ (compettve advantages) are therefore defned as functons of user commands, sensory events, or nternal states (Stenhage & Schoner, 998). Here, we make sure that one neuron s always on by varyng the µ -parameters between the values.5 and 3.5, µ =.5 + b,where b are quas-boolean factors takng on values between and (wth a tendency to have values ether close to or close to ). These quas-booleans express logcal or sensory condtons controllng the sequental actvaton of the dfferent neurons (see (Stenhage & Schoner, 998; Santos, 3), for a general framework for sequence generaton based on these deas): 6

8 . b nt may be controlled by user nput: the command move sets b nt from the default value to to destablze the ntal posture. In a frst nstance b nt s controlled by an ntal tme set by the user. Thus, ts value changes from to when tme exceeds t nt : b nt ( t t) ( t) = σ (6) Heren, σ(.) s a sgmod functon that ranges from for negatve argument to for postve argument, selected as nt [ tanh( ) ] / σ ( x) = x + (7) although any other functonal form wll work as well. b nt may also be controlled by sensory nput, such that, for nstance, b nt changes from to when a partcular sensory event s detected. Below we demonstrate how the tme-tocontact of an approachng object computed from sensory nformaton can be used to ntate movement n ths manner.. b hopf s set from to under the same condtons. Ths term s multpled, however, wth a second factor b has not reached target (x) = σ(x crt - x) that resets b hopf to zero when the effector has reached ts fnal state. The factor, b has not reached target (x) has values close to one whle the tmng varable x s below x crt =.7 and swtches to values close to zero when x comes wthn.3 of the target state (x = ). Multplyng two quas-booleans means connectng the correspondng logcal condtons wth an and operaton. Thus, as soon as the tmng varable has come wthn the vcnty of the fnal state, t autonomously turns the oscllatory state off. In actual mplementaton, ths swtch can be drven from the sensed actual poston of an effector rather than from the tmng dynamcs. The fnal expresson for b hopf s: b = σ t t b ( x (8) hopf ( ) ) nt has not reached t arg et 3. b fnal s, conversely, set from to when the tmng varable comes nto the vcnty of the target: b fnal = - b has not reached target. The tme scale of the neuronal dynamcs s gven by τ u = and s set to a relaxaton µ υ tme of τ u =., ten tmes faster than the relaxaton tme of the tmng varables. Ths dfference n tme scale guarantee that the analyss of the attractor structure of the neural dynamcs s unaffected by the dependence of ts parameters, µ on the tmng varable, x, whch s a dynamcal varable as well. Strctly speakng, the neural and tmng dynamcs are thus mutually coupled. The dfference n tme scale makes t possble to treat x as a parameter n the neural dynamcs (adabatc varables). Conversely, the neural weghts can be assumed to have relaxed to ther correspondng fxed ponts when analyzng the tmng dynamcs (adabatc elmnaton). The adabatc elmnaton of fast behavoral varables reduces the complexty of a complcated behavoural system bult up by couplng many dynamcal systems (Stenhage & Schöner, 998; Santos, 3). By usng dfferent tme scales one can desgn the several dynamcal systems separately..3 An Example: A Tmed Temporally Dscrete Movement Act Perodc movement can be trvally generated from the tmng and neural dynamcs by selectng u hopf on through the correspondng quas-booleans. A tmed, but temporally dscrete movement act, s autonomously generated by these two coupled levels of nonlnear dynamcs through a sequence of neural swtches, such that an oscllatory state exsts durng an approprate tme nterval of about a half-cycle. Ths s llustrated n Fgure 6

9 . The tmng varable, x, whch s used to generate effector movement, s ntally n a postural state at -, the correspondng neuron u nt beng on. When the user ntates movement, the quas-booleans, b nt and b hopf exchange values, whch leads, after a short delay, to the actvaton of the hopf neuron. Ths swtch ntates movement, wth x evolvng along a harmonc trajectory, untl t approaches the fnal state at +. At that pont, the quas-boolean b fnal goes to one, whle b hopf changes to zero. The neurons swtch accordngly, actvatng the fnal postural state, so that x relaxes to ts termnal level x =. The movement tme s approxmately a half cycle tme, here MT =.. Smulaton of a Two Dof Arm Interceptng a Ball As a toy example of how the dynamcal systems approach to tmng can be put to use to solve robotc problems, consder a two dof robot arm movng n a plane (Fgure 3). The task s to generate a tmed movement from an ntal posture to ntercept an approachng ball. Movement wth a fxed movement tme (reflectng manpulator constrants) must be ntated n tme to reach the ball before t arrves n the plane n whch the arm moves. Factors such as reachablty and approach path of the ball are tmng varables x y u nt neurons u hopf u fnal tme quas booleans b nt b hopf b fnal tme ntaton of movement tme Fgure. Smulaton of a user ntated temporally dscrete movement represented by the tmng varable, x, whch s plotted together wth the auxlary varable, y, n the top panel. The tme courses of the three neural actvaton varables, u nt, u hopf, and u fnal, whch control the tmng dynamcs, are shown n the mddle panel. The quas-boolean parameters, b nt, b hopf, and b fnal, plotted on bottom, determne the compettve advantage of each neuron contnuously montored, leadng to a return of the arm to the restng poston when ntercepton becomes mpossble (e.g., because the ball hts outsde the workspace of the arm, the ball s no longer vsble, or ball contact s no longer expected wthn a crteron tme-to-contact). After the ball ntercepton, the arm moves back to ts restng poston, ready to ntate a new movement whenever approprate sensory nformaton arrves. 63

10 In order to formulate ths task usng the nonlnear dynamcal systems approach, three relevant coordnate systems are defned: a) The tmng varable coordnate system, {P} 3, descrbes the tmng varable poston along a straght path from the ntal to the fnal postural poston. Ths s a conceptual frame, n whch temporal movement s planned. b) The task reference coordnate system (unversal coordnate system) descrbes the endeffector poston, (x, y, z), of the arm along a straght path from the ntal poston (ntal posture) to the target poston (computed coordnates of pont of nterceptance) and the ball poston. c) The base coordnate system {R} s attached to the robot's base and the arm knematcs s descrbed by two jont angles n ths frame. Fgure 3. A two dof arm ntercepts an approachng ball. Correspondng ball and arm postons are llustrated by usng the same grey-scale. The frst poston (lght grey) s close to the crtcal tme-to-contact, where arm moton starts. The last poston (dark grey) s close to actual contact. The black arrows ndcate the ball's movement The tmng varable coordnate system {P} s postoned such that P x coordnates vary between - and +, wth P y and P z coordnates equal to zero. P x s scaled to the desred ampltude, A, dependent on the predcted pont of nterceptance. Frame (a) and (b) are lnked through straghtforward formulae, whch depend on the predcted pont of nterceptance, (x(τ tc ); y(τ tc );) n the task reference frame. Frame (b) and (c) are lnked through the knematc model of the robot arm and ts nverse (whch s exact). Durng movement executon, the tmng varables are contnuously transformed nto task frame (b), from whch jont angles are computed through the nverse knematc transformaton. The solutons of the appled autonomous dfferental equatons are converted by smple coordnate transformatons, usng model-based control theory, nto motor commands. 3 However, n the text, the tmng varables P x and P y are referred to as tmng varable x, y wthout superscrpt. 64

11 . Couplng to Sensoral Informaton In order to ntercept an approachng ball t s necessary to be at the rght locaton at the rght tme. We use the vsual stmulus as the percepton channel to our system. In these smulatons we have extracted from a smulated ball trajectory two measures: the tme-tocontact, τ tc, and the pont-of-contact. The robot arm ntersects the ball on the plane of the camera (mounted on ts base) such that the ball's movement crosses the observer (or mage) plane (z B = ). The tme t takes to the ball to ntersect the arm at ths pont n space, that s, the tme-to-contact, s extracted from segmented vsual nformaton wthout havng estmated the full cartesan trajectory of the ball (Lee, 976). We consder the ball has a lnear trajectory n the 3D cartesan space wth a constant approach constant velocty. The pont of contact can be computed along smlar lnes f the ball sze s assumed to be known and can be measured n the mage. To smulate sensor nose (whch can be substantal f such optcal measures are extracted from mage sequences), we added ether whte or coloured nose to the estmated tme-tocontact. Here we show smulatons that used coloured nose, ξ, generated from ζ = ζ + Q gwn (9) τ corr where gwn s gaussan whte nose wth zero mean and unt varance, so that Q = 5 s the effectve varance. The correlaton tme, τ corr, was chosen as. sec. The smulated tme-to-contact was thus. Behavor Specfcatons τ = true tme to contact + ζ ( ) () t c t These two measures, tme-to-contact and pont-of-contact, fully control the neural dynamcs through the quas-boolean parameters. A sequence of neural swtches s generated by translatng sensory condtons and logcal constrants nto values for these parameters. For nstance, the parameter, b nt, controllng the compettve advantage of the ntal postural state must be on (= ) when the tmng varable x s close to the ntal state -, and ether of the followng s true: a) Ball not approachng or not vsble (τ tc ). b) Ball contact not yet wthn a crteron tme-to-contact (τ tc > τ crt ). c) Ball s approachng wthn crteron tme-to-contact but s not reachable ( <τ tc < τ crt ; b reachable = ). These logcal condtons can be expressed through ths mathematcal functon: b nt [ σ ( τ τ ) + σ ( τ ) σ ( τ τ ) σ ( b ) + σ ( τ )] = σ ( x x) () crt t c crt t c crt t c reacchable t c where σ(.) s the threshold-functon used earler (Equaton 7). The or s realzed by summng terms whch are never smultaneously dfferent from zero. In other cases, the or s expressed wth the help of the not (subtractng from ) and the and. Ths s used n the followng expressons for b hopf and b fnal whch can be derved from a smlar analyss: [ σ ( xcrt x) σ ( τ t c) σ ( τ crt τ t c) σ ( breacchable) ]) ){ σ ( b ) + σ ( τ ) + σ ( τ τ ) + σ ( x x) } bhopf = ( () ( [ σ ( x + xcrt reacchable t c t c crt crt ]) b = σ τ ) σ ( τ τ ) σ ( b ) + σ ( x x ) (3) fnal ( t c crt t c reacchable crt.3 Propertes of the Generated Tmed Trajectory Fgure 3 shows how ths two dof arm ntercepts an approachng ball. The detaled tme courses of the relevant varables and parameters are shown n Fgure 4. As the ball 65

12 approaches, the current tme-to-contact becomes smaller than a crtcal value (here 3), at whch tme the quas-boolean for moton, b hopf becomes one, trggerng actvaton of the correspondng neuron, u hopf, and movement ntaton. Movement s completed (x reaches the fnal state of +) well before actual ball contact s made. The arm wats n the target posture. In ths smulaton the ball s reflected upon contact. The negatve tme-to-contact observed then leads to autonomous ntaton of the backward movement to the arm restng poston. The fact that tmed movement s generated from attractor solutons of a nonlnear dynamcal system leads to a number of propertes of ths system, that are potentally useful to real-world mplementatons of ths form of autonomy. The smulaton shown n Fgure 4 llustrates how the generaton of the tmng sequence ressts aganst sensor nose: the nosy tme-to-contact data led to strongly fluctuatng quas-booleans (nose beng amplfed by the threshold functons). The neural and tmng dynamcs, by contrast, are not strongly affected by sensor nose so that the tmng sequence s performed as requred. When smulatons wth ths level of sensor nose are repeated, falure s never observed, although there are nstances of mssng the ball at even larger nose levels. By smulatng strong sensor nose we demonstrate that the approach s robust. Note how the autonomous sensor-drven ntaton of movement s stablzed by the hysteress propertes of the compettve neural dynamcs, so that small fluctuatons of the nput sgnal back above threshold do not stop the movement once t has been ntated (Schoner & Dose, 99; Schoner & Santos, ). The desgn of the quas-boolean parameters of the compettve dynamcs guarantees that flexblty s fulflled: f the sensoral context changes such that the prevously generated sequence s no longer adequate, the plan s changed and a new sequence of events emerges. tmng varables y x neurons u nt u hopf u fnal quas booleans b nt b hopf b fnal tme to contact 4 ball trajectory tme to contact reaches threshold ball contact and reflecton Fgure 4. Trajectores of varables and parameters n autonomous ball ntercepton and return to restng poston. The top three panels represent tmng varables, neural varables and quas-booleans. The bottom panel shows the tme-to-contact, whch crosses a threshold at about :5 tme unts. When contact s made, the ball s assumed to be reflected, leadng to negatve tme-to-contact tme 66

13 When sensory condtons change an approprate new sequence of events emerges. When one of the sensory condtons for ball ntercepton s nvald (e.g., ball becomes nvsble, unreachable, or no longer approaches wth approprate tme-to-contact), then one of the followng happens dependng on the pont wthn the sequence of events at whch the change occurs: ) If the change occurs durng the ntal postural stage, the system stays n that postural state. ) If the change occurs durng the movement, then the system contnues on ts trajectory, now gong around a full cycle to return to the reference posture. 3) When the change occurs durng posture n the target poston, a dscrete movement s ntated that takes the arm back to ts restng poston. The decson s dependent on local nformaton avalable at the system's current poston: on the current locaton of the tmng varable, x, on the tme-to-contact and pont-ofcontact nformaton currently avalable. The non-local sequence of events s generated through local nformaton wthout needng symbolc representatons of the behavours. Ths s acheved by obeyng the prncples of the Dynamc Approach and llustrates the power of our approach: the behavour of the system tself leads to the changng sensor nformaton whch controls the change and persstence of a rch set of behavors. Consder the ball s suddenly shfted away from the arm at about.9 tme unts, leadng to much larger tme-to-contact, well beyond threshold for movement ntaton. In Fgure 5, tme-to-contact becomes suddenly larger than the crtcal value when the arm s n ts moton stage: u hopf neuron s actvated and the other neurons are deactvated. The u hopf neuron rests actvate whle the arm contnues ts movement a full cycle. At the tme the x tmng varable s captured by the ntal postural state (x = -), the quas-boolean b nt becomes one, trggerng the actvaton of the neuron, u nt, and b hopf becomes zero, deactvatng the correspondng neuron u hopf. The arm rests n the reference poston. Ths behavour emerges from the sensory condtons controllng the neuronal dynamcs. tmng varables y x neurons u u nt hopf.5 u fnal quas booleans b nt b hopf b fnal tme to contact ball s perturbed ncreasng 3 4 tme to contact tme to contact beyond threshold 5 6 tme reaches threshold Fgure 5. Smlar to Fgure 4, but the ball s suddenly shfted at about.9 tme unts leadng to a tme to contact larger than the threshold value (3) requred for movement ntaton 67

14 3. Couplng among Two Tmng Systems Another task llustratng uses of the dynamcal systems approach to tmng s the temporal coordnaton of dfferent dofs. In robotcs, the control of two dof s generally acheved by consderng the dofs are completely ndependent. Therefore, ths problem reduces to the one of controllng two robot arms nstead of one. However, n motor control of bologcal systems where there are numerous dofs, ths ndependence s not verfed. Movement coordnaton requres some form of plannng: every dof needs to be suppled wth approprate motor commands at every moment n tme. However, there exst actually an nfnte number of possble movement plans for any gven task. A rch area of research has been evolvng to study the computatonal prncples and mplct constrants n the coordnaton of multple dofs, specfcally the queston whether or not there are specfc prncples n the organzaton of central nervous systems, that coordnate the movements of ndvdual dofs. Ths research has been manly drected towards coordnaton of rhythmc movement. In rhythmc movements, behavoural characterstcs show off n that the oscllatons verfed n these dofs reman coupled n-phase, and also because the dofs show a tendency to become coupled n a determned way (Schaal, ). In reachng movements, behavoural characterstcs reveal n the synchronzaton and/or sequencng of movements wth dfferent on-set tmes. The coordnaton of rhythmc movements has been addressed wthn the dynamc theoretcal approach (Schoner,99). These dynamc concepts can be generalzed to understand the coordnaton of dscrete movement. Temporal coordnaton of dscrete movements s enabled through the couplng among the dynamcs of several such systems such that n the perodc regme the two forms of movement, stable n-phase and antphase, are recovered. Ths s acheved by ntroducng a dscrete movement for each dof (end-effector), and two dofs are consdered. The dea s to couple the two dscrete movements wth the same frequency n a specfc way, determned by well establshed parameters, such that wthn a certan tme the two movements become locked at a gven relatve phase. The couplng term s multpled wth the neuronal actvaton of the other system's Hopf state such that couplng s effectve only when both components are n movement state. Ths s acheved by modfyng the "Hopf" contrbuton to the tmng dynamcs as follows x x x =... + uhopf, fhopf,( x, y) + uhopf, cr( θ )... (4) y y y x x x =... + uhopf, fhopf,( x, y) + uhopf, cr( θ )... (5) y y y where ndex =, refers to the dof and, respectvely, and θ s the desred relatve phase. Ths coordnaton through couplng approaches the generaton of coordnated patterns of actvaton n locomotory behavour of nervous bologcal systems. 3. An Example: Two 6 Dof Arms Coupled Consder two PUMA arms performng a straght lne moton n 3D Cartesan space (Schoner & Santos, ; Santos, 3). In the smulatons, the nverse knematcs of the PUMA arms were based on the exact soluton. Each robot arm ntates ts tmed movement from an ntal posture to a fnal one. Movement parameters such as ntal posture, movement ntaton, ampltude and movement tme are set for each arm ndvdually. 68

15 Each arm s drven by a complete system of tmng and neural dynamcs. Further, the two tmng dynamcs are coupled as descrbed by Equatons 4 and 5. The specfed relatve phase s θ = º. In dscrete motor acts, a couplng of ths form tends to synchronze movement n the two components, a tendency captured n terms of relatve tmng of the movements of both components. The two robot arms are temporally coordnated: f movement parameters such as movement on-sets or movement tmes are not dentcal, the control level coordnates the two components such that the two movements termnate approxmately smultaneously. Ths couplng among two tmng systems helps synchronze systems and reduces the computatonal requrements for determnng dentcal movement parameters across such components. Even f there s a dscrepancy n the MT programmed by the parameter ω of the tmng dynamcs, couplng generates dentcal effectves MTs. Ths dscrete analogue of frequency lockng s llustrated n left panel of Fgure 6. x c x c x u x u coupled x c uncoupled uncoupled x u x u x c coupled Tme Tme Fgure 6. a) Coordnaton between two tmng dynamcs through couplng leads to synchronzaton when movement tmes dffer ( vs. 3). b) Movement ntaton s slghtly asynchronous t nt = and t nt =.4s ( t nt = 5 % of MT), MT = MT = s and c = ) Ths tendency to synchronze s also verfed when both movements exhbt equal movement tmes but the on-sets are not perfectly synchronzed (rght panel of Fgure 6). In ths case, the effect n the delayed component s to move faster, agan n the drecton of restorng synchronzaton. In case we set a relatve phase of θ = 8º we verfy ant-phase lockng. In the case of dscrete movement, ant-phase lockng leads to a tendency to perform movements sequentally. Thus, f movement ntaton s asynchronous, the movement tme of the delayed movement ncreases such that the movements occur wth less temporal overlap. 4. Integraton of Dfferent Dynamcal Archtectures As an mplementaton of the approach, the capacty of a low level vehcle to navgate n a non-structured envronment whle beng capable of reachng a target n an approxmately constant tme s chosen. 4. Attractor Dynamcs for Headng Drecton The robot acton of turnng s generated by varyng the robot's headng drecton, φ h, measured relatve to an allocentrc coordnate system, as a soluton of a dynamcal system 69

16 (Schöner and Dose, 99). Ths behavoural varable s governed by a nonlnear vector feld n whch task constrants contrbute ndependently by modellng desred behavours (target acquston) as attractors and undesred behavours (obstacle avodance) as repellers of the overall behavoural dynamcs (Bcho, ). Target locaton, (x B, y B ), s contnuously extracted from vsual segmented nformaton acqured from the camera mounted on the top of the robot and facng n the drecton of the drvng speed. The angle φ h of the target's drecton as seen from the robot s: y y R = arctan φ = arctan x x R B R yb φ (6) tar tar B R where (x R, y R ) s the current robot poston n the allocentrc coordnate system as gven by the dead-reckonng mechansm. Integraton of the target acquston (behavour f tar ) and obstacle avodance (behavour f obs ) contrbutons s acheved by addng each of them to the vector feld that governs headng drecton dynamcs d( φh ) = Fobs ( φh ) + ftar ( φh ) + f stoch( φh ) (7) dt We add a stochastc component force, F stoch, to ensure escape from unstable states wthn a lmted tme. The complete behavoural dynamcs for headng drecton has been mplemented and evaluated n detal on a physcal moble robot (Bcho, ). 4. The Dynamcal Systems of Drvng Speed Robot velocty s controlled such that the vehcle has a fxed tme to reach the target. Thus, f the vehcle takes longer to arrve at the target because t needed to crcumnavgate an obstacle, ths change of tmng must be compensated for by acceleratng the vehcle along ts path. The path velocty, v, of the vehcle s controlled through a dynamcal system archtecture that generates tmed trajectores for the vehcle. We set two spatally fxed coordnates frames both centred on the ntal posture, whch s the orgn of the allocentrc coordnate system: one for the x and the other for the y spatal coordnates of robot movement. A complete system of tmng and neural dynamcs s defned for each of these fxed coordnate frames. Each model conssts of a tmng layer (Schoner & Santos, ), whch generate both stable oscllatons (contrbuton f hopf ) and two statonary states (contrbutons nt and fnal ). x = a u nt, f nt, + u hopf, f hopf, + u fnal, xb f fnal, + gwn where the ndex = x, y refers to tmng dynamcs of x and y spatal coordnates of robot movement. A neural dynamcs controls the swtchng between the three regmes through three neurons, u j, (j = nt, hopf, fnal) (Equaton ). The nt and fnal contrbutons generate stable statonary solutons at x = for nt and A c for fnal wth a = for both. These states are characterzed by a tme scale of τ = /5 =.. The Hopf contrbuton to the tmng dynamcs s defned as follows: (8) 7

17 A α c h ω A x A c x f = + hopf, γ x a ω α h a a 4α h where λ = defnes ampltude of Hopf contrbuton. A c c (9) 4.3 Behavoural Specfcatons The neuronal dynamcs of u j, [-; ] (j = nt, hopf, fnal) swtches each tmng dynamcs from the ntal and fnal postural states nto the oscllatory regme and back, and are gven by 3 α u = µ u µ u v u u + gwn u j, j, j, j, j, a, j, () We assure that one neuron s always on by varyng the µ -parameters between the values.5 and 3.5: µ =.5 + b, where b are the quas-boolean factors. The compettve advantage of the ntal postural state s controlled by the parameter b nt. Ths parameter must be on (= ) when ether of the followng s true: () tme, t, s bellow the ntal tme, t nt, set by the user (t < t nt ); () tmng varable x s close to the ntal state (b x close xnt (x)); and tme exceeds t nt (t > t nt ); and target has not been reached. We consder that the target has not been reached when the dstance, d tar, from the actual robot poston (as nternally calculated through dead-reckonng) and the (x target, y target ) poston s hgher than a specfed value, d margn. Ths logcal condton s expressed by the quas-boolean factor, b x has not reached target (d tar ) = σ(d tar -d margn ), where σ(.) s the sgmod functon explaned before (Equaton 7). Note that ths swtch s drven from the sensed actual poston of the robot. The factor b x close xnt (x ) = σ( x crt -x ) has values close to one whle the tmng varable x s bellow.5a c and swtches to values close to zero elsewhere. These logcal condtons are expressed through the mathematcal functon: b nt a j [ ( )]} { d ( t t ) b ( x )( t t ) b ( ) = () nt x close x A smlar analyss derves the b hopf and b fnal parameters: b fnal nt nt x has not reached t arg et bhopf ( t tnt ) bx ( ) ( ) ( ) not close x x fnal bx has not reached t arg et dtar σ bupdate Ac ( t t ) b ( x ) + b ( d ) + b ( x ) + ( σ ( b ) nt = () [ x not close x x reached t et tar x not close x update A ] = arg (3) fnal We algorthmcally turn off the update of the tmed target locaton, T x target or T y target, once ths changes sgn relatvely to the prevous update and the correspondng tmng level s n the ntal postural state. The factor b x not close xfnal (x ) = σ(d swtch - d crt ) s specfed based on absolute values, where d swtch represents the dstance between the tmng varable x and the fnal postural state, A c and d crt s tuned emprcally. The compettve dynamcs are the faster dynamcs of the all system. Its relaxaton tme, τ u, s set ten tmes faster than the relaxaton tme of the tmng varables (τ u =.). The system s desgned such that the plannng varable s n or near a resultng attractor of the dynamcal system most of the tme. If we control the drvng velocty, v, of the vehcle, the system s able to track the movng attractor. Robot velocty depends whether or not obstacles are detected for the current headng drecton value. Ths velocty depends on fnal c 7

18 the behavour exhbted by the robot and s mposed by a dynamcs equal to that descrbed by (Bcho et al, ) dv dt ( v V ) ( v V = cobs obs ( ) ( ) t mn g v V obs exp c t mn g v Vt mn g exp σ v σ v V obs s computed as a functon of dstance and s actvated when an obstacle s detected. V tmng s specfed by the temporal level as V t mn g x y ) (4) = x + x (5) For further detals regardng ths dynamcs refer to (Bcho, ). The followng herarchy of relaxaton rates ensures that the system relaxes to the stable solutons, obstacle avodance has precedence over target acquston and target achevement s performed n tme τ v, obs << τ v, obs, τ v, t mn g << τ tar, τ obs << τ tar (6) Suppose that at t = s the robot s restng at an ntal fxed poston, the same as the orgn of the allocentrc coordnate system. The robot rotates n the spot n order to orent towards the target drecton. At tme t nt, the quas-boolean for moton, b hopf, becomes one, trggerng actvaton of the correspondng neuron, u hopf, and movement ntaton. Movement ntaton s accomplshed by settng the drvng speed, v, dfferent from zero. Durng perodc movement, the target locaton n tme s updated each tme step based on error, x R - T x x, such that T x T ( x x ) t arg et = xt arg et R x (7) where T x x s the current tmng varable x x, x r s the x robot poston and s the tmng varable x x. The perodc moton's ampltude, A xc, s set as the dstance between T x target and the orgn of the allocentrc reference frame (whch s concdent wth the x robot poston prevously to movement ntaton), such that A xc T ( t) x ( t) = (8) t arget The perodc soluton s deactvated agan when the vehcle comes nto the vcnty of the x tmed target, and the fnal postural state s turned on nstead (neurons u hopf = ; u fnal = ). At ths moment n tme, the x tmed target locaton s no longer updated n the tmng dynamcs level. The same behavour apples for the tmng level defned for the y spatal coordnate. 4.4 Expermental Results The dynamc archtecture was mplemented and evaluated on an autonomous wheeled vehcle (Santos, 4). The dynamcs of headng drecton, tmng, compettve neural, path velocty and dead-reckonng equatons are numercally ntegrated usng the Euler method wth fxed tme step. Image processng has been smplfed by workng n a structured envronment, where a red ball les at coordnates (xb; yb) = (-.8; 3.) m (on the allocentrc coordnate system) on the top of a table at approxmately.9m tall. The ntal headng drecton s 9 degrees. The sensed obstacles do not block vson. An mage s acqured only every sensoral cycles such that the cycle tme s 7 ms, whch yelds a movement tme (MT 4 ) of 4s. Forward 4 Specfed tme for the robot to meet the ball after the forward movement s ntated. 7

19 movement ntaton s trggered by an ntal tme set by the user and not from sensed sensoral nformaton. Forward movement only starts for tnt = 3s. The rotaton speeds of both wheels are computed from the angular velocty, w, of the robot and the path velocty, v. The former s obtaned from the dynamcs of headng drecton. The later, as obtaned from the velocty dynamcs s specfed ether by obstacle avodance contrbuton or the tmng dynamcs. By smple knematcs, these veloctes are translated nto the rotaton speeds of both wheels and sent to the velocty servos of the two motors. A. Propertes of the Generated Tmed Trajectory The sequence of vdeo mages shown n Fgure 7 llustrates the robot moton n a very smple scenaro: durng ts path towards the target, the robot faces two obstacles separated of.7m, whch s a dstance larger enough for the robot to pass n between. The tme courses of the relevant varables and parameters are shown n Fgure 8. At tme t = 7. s obstructons are detected and the velocty dynamcs are domnated by the obstacle constrants (bottom panel of Fgure 8). Due to obstructons crcumnavgaton, the robot poston dffers from what t should be accordng to the tmng layer (at tme t =.5s). The x robot poston s advanced relatvely to the T x x tmng dynamcs specfcatons and A xc s decreased relatvely to x target (Fgure 8). Fgure 7. Robot moton when the robot faces two objects separated of.7m durng ts path. The robot successfully passes through the narrow passage towards the target and comes to rest at a dstance of.9m near the red ball at t = 6.56s. The effectve movement tme s 3.56 s Therefore, the robot velocty s de-accelerated n ths coordnate. Conversely, the y robot poston lags the T x y tmng varable and robot velocty s accelerated n ths coordnate 73

20 (thrd panel of Fgure 8). Fnally, the target s reached and the robot comes to rest at a dstance of.9m near the red ball. The overall generated tmed trajectory takes t = s to reach the target (forward movement started at tme t = 3s). Ths trajectory dsplays a number of propertes of dynamcal decson makng. Fgure 8 shows how the hysteress property allows for a specal knd of behavoral stablty. At t = 5s the quas-boolean parameter b x,hopf becomes zero but the u x,hopf neuron remans actvated untl the neuron u x,fnal s more stable, what happens around t = 6.s. At ths tme, the x perodc moton s turned off. Thus, hysteress leads to a smple knd of memory whch determnes system performance dependng on ts past hstory. Fgure 9 shows the robot trajectory as recorded by the dead-reckonng mechansm when the dstance between the two obstacles s smaller than the vehcle's sze (.3m). The path followed by the robot s qualtatvely dfferent. In case tmng dynamcs stablze the velocty dynamcs the robot s strongly accelerated n order to compensate for the object crcumnavgaton. Lght crosses on the robot trajectory ndcate robot postons where vson was not acqured because the robot could not see the ball. The ball poston as calculated by the vsual system slght dffers from the real robot poston (ndcated by a dark crcle). B. Trajectores Generated wth and wthout Tmng Control Table surveys the tme the robot takes to reach the target lyng at coordnates (-.8, 3.4) m for several confguratons when path velocty, v, s controlled wth and wthout tmng control. In the latter, path velocty s specfed dfferently: when no obstructons are detected the robot velocty s stablzed by an attractor, whch s set proportonal to the dstance to the target (Bcho, ). Note that forward movement starts mmedately. Conversely, forward movement only happens at t = 3 s when there s tmng control. The specfed movement tme s 4s. tmng varables.5 y x x target A xc x x neurons 5 5 u x,nt u x,hopf tmng varables y target A yc x y neurons 5 5 u y,nt u y,hopf y y b x,hopf u x,fnal quas booleans 5 5 b x,nt b x,fnal 5 5 u y,fnal quas booleans 5 5 b y,nt b y,hopf b y,fnal 5 5 d swtch start d target d swtch start d target 5 5 tme (s) u y,hopf 5 5 u x,hopf U(φ) tme (s) 5 5 T x xx R x target A xc 5 5 y target T xy 5 5 A yc y R. v tmng v v obs 5 5 tme (s) Fgure 8. Tme courses of varables and parameters for the robot trajectory depcted n Fgure 7. Top panels depct tmng and neural dynamcs (x and y coordnate n left and rght panels, respectvely). Bottom panel depcts neural and tmng varables, robot trajectores, real target locatons, perodc moton ampltudes and velocty varables. 74