Synthesized Articulated Behavior using Space-temporal On-line Principal Component Analysis

Transcription

1 Sytheszed Artculated Behavor usg Space-temporal O-le Prcpal Compoet Aalyss YUICHI MOAI Uversty of Vermot, USA, Abstract hs paper presets a ew framework to sythesze humaod behavor by learg ad mtatg the behavor of a artculated body usg moto capture. he vdeo-based moto capturg method has bee developed maly for aalyss of huma movemet, but s very rarely used to teach or mtate the behavor of a artculated body to a vrtual aget a o-le maer. Usg our proposed applcatos, ew behavors of oe aget ca be smultaeously aalyzed ad used to tra or mtate aother wth a ovel vsual learg methodology. I the o-le learg phase, we have developed a ew framework of prcpal compoet aalyss (PCA) usg a cremetal techque. For tradtoal off-le PCA, t s ecessary to store all the prevous ad ew staces order to recostruct the egespace. Our proposed o-le PCA ca add umerous ew trag staces whle matag reasoable egespace dmesos. he proposed methodology s well matched to both behavoral trag ad sythess, sce t s automatcally carred out as a o-le log-term learg of humaod behavors wthout the overhead of a expadg learg space. he fal outcome of these methodologes s to sythesze multple humaod behavors for the geerato of arbtrary behavors. he expermetal results usg a humaod fgure ad a vrtual robot demostrate the feasblty ad merts of ths method. Keywords: Imtatve robot, artculated behavor sythess, vsual moto capture system, learg from observato, huma behavor model. INRODUCION Sythess of humaod/robotc behavor s a computatoally demadg process. I the advaced robotcs commuty, autoomous creato of moto,.e., behavor for a vrtual avatar, s a very mportat research topc. Although the ultmate research goal s to establsh a autoomous aget wth ay artculated behavors, we are far from achevg ths deal level. hus, here we cocetrate our efforts towards makg t possble for a robot to mtate or drectly lear from a teacher some comprehesve behavors, or to set up a framework to trasfer behavors by observg actual - - Advaced Robotcs (RSJ)

2 behavors the real world. I may stuatos, teachg artculated behavor captured by a vdeo camera s possble for humaod robots. he use of vso-based moto aalyss s ot ew; rather, t s stadard computatoal o-cotact type measuremet systems. For example, some exstg 3D moto aalyss systems have bee already developed for varous dustral applcatos, such as Vco Moto Systems [] A.P.A.S. [2], Moto Aalyss [25], ad [0,, 2] academa. he cotrbuto for ths paper s that we have address the developmet of a ew framework for behavor mtato, whch dyamc behavor acqusto ca be acheved by sytheszg prcpal behavors. Usg moto captured from vdeo sequeces, we have mplemeted a o-le learg framework whch observable staces are cremeted. Wth prevous methods, the feature space explodes whe the degree of behavoral freedom s creased. Our proposed ew method prevets ths from happeg by lmtg the amout of data that must be stored. For ths latest method, we exteded prcple compoet aalyss (PCA) to classfy the trag behavors the teratve maer. More specfcally, usg a cremetal learg techque, we attempt to replcate the best or optmal behavor of a partcular aget that has bee gve drect structo. Certa behavors may follow a commo patter, ad we ca use formato from ths kematcs moto to buld a commo or prcpal task. We wll expla how the prcpal compoets of artculated motos ca be obtaed ad used together wth classfcato ad sythess. At the begg of the moto capture ad behavor trag process, the huma--the-loop approach s developed whe regsterg the behavors the form of basc moto descrpto. A huma operator tally eeds to supervse the learg system by labelg each behavor usg the proposed graphcal user terface (GUI). hs uque moto edtg process elmates a umber of problems classfyg the behavors that wll be automatcally carred out a o-le maer. he orgazato of the rest of ths paper s as follows: Secto 2 descrbes related studes o learg by observato for humaod robot. Secto 3 llustrates the overall vsual observato system to capture artculated movemets for a humaod robot. Our proposed methodologes for o-le learg usg cremetal PCA are preseted Secto 4, ad evaluated expermetally Secto 5. Secto 6 reports some terestg expermetal results to demostrate the performace of sythess. Fally, the sytheszed performace usg ege behavors s evaluated Secto 7. Cocludg remarks are gve Secto RELAED SUDIES he feld of vsual mtato usg moto capture s very actve, ad has evolved ot oly the advaced robotc commuty, but also bomechacs ad computer graphcs. Obvously, those felds have some terdscplary overlap; deed, our project descrbed ths paper has bee spred by ad spas these felds. I the area of computer graphcs, vrtual agets are realzed through avatars. he captured moto data are drectly used for determg avatar behavor by Advaced Robotcs (RSJ)

3 followg typcal bomechacs. Sce they do ot volve the actual mechacal structures of the robots, t seems smpler to apply forward kematcs to the graphcal model of the avatar the vrtual world. It s, however, more complcated f the structure of the avatar s ot detcal to the real object, or f the ew behavor must be sytheszed [3]. For example, patters of each jot are selected ad optmzed uder physcal costrats [4]. Aother exteso s [5], whch behavors of the avatar ca be trasferred from adults male/females to chldre by PD cotrol wth the fxed relevat parameters. hose extesos amog avatars are retargeted f the physcal models are clearly specfed. o mtate behavors from the real object a vrtual avatar, may further problems develop:.e. t becomes ecessary to estmate modfed behavors for ew or dfferet codtos [6]. For example, the case of a modfed ru ad walk behavor, the avatar ca slowly ru by relocatg the two dstgushed behavors (ru ad walk). I [7], walkg behavors are sytheszed by terpolatg termedate maagemets. Popov c ad hs colleagues [32] developed a teractve system that gave the amator fe cotrol over the moto of a sgle rgd body. Popov c ad Wtk [3] showed that sgfcat chages to moto capture data ca be made by maually reducg the character to the degrees of freedom for the task. he optmzed moto was the mapped back up to the full character. Humaod moto wth may degrees of freedom ca be optmzed whe the amator provdes closely spaced key frames wthout exact tmg formato [29]. Short segmets of moto ca be computed for characters wth may degrees of freedom as Rose ad hs colleagues [33] demostrated whe they computed optmal trastos betwee humaod moto segmets that bega ad eded wth dfferet but smlar poses. Lu ad Popov c [30] showed that some dyamc effects ca be preserved by eforcg patters of lear ad agular mometum that do ot requre the computato of such dyamc parameters as cotact forces ad jot torques. Outsde the area of characterzg vrtual robots, reduced order models of dyamcs have bee explored extesvely computer graphcs ad other felds [26,27,28,3,34]. I addto to the area of robotcs, artfcal tellgece, ad physology, umercal approaches of learg by observato robotcs have bee examed [3,4,5,6,7,8]. For those the computatoal vso ad mache learg commutes, the task-level of learg eeds to be costructed a pror based o the captured sesory data of the trajectores. he approaches descrbed ths paper are the le of those. 3. VISUAL OBSERVAION SYSEM USING IMIAIVE FRAMEWORK HOUGH MOION CAPURE We select artculated objects as targets sce we wsh to establsh effcet ad effectve teracto amog the actual target objects, both amog vrtual avatars ad betwee the vrtual ad actual targets. Our overall learg system wll be acheved by developg the followg phases: ) Presetato: he targeted artculated aget/huma performs the desred behavoral tasks to the system from begg Advaced Robotcs (RSJ)

4 to ed wthout pausg. 2) Italzato: he huma--the-loop module specfes the abstract class of targeted objects ad edts hgher structures such as a goal herarchy ad depedecy amog the behavor tasks. 3) Recogto: he capturg system observes the performace by a color camera ad costructs a spato-temporal cotrol-pot descrpto of the behavor/task. Recogto must be automatc ad o-le order to keep up wth the cremets of the behavor/task performace. 4) rag: For a gve target evromet, the system recogzes the tal cofgurato of the objects, ad classfes ad statates the behavor/task descrpto. 5) Sythess: he traed system s executed order to geerate the sytheszed behavors/tasks. o realze these phases, the followg requred mmal protocols are establshed: Frst, humaod movemets are observed by a moto capturg devce usg a camera wth a ladmark marker (alteratvely a gyroscope, or a dataglove). he, usg a uquely developed GUI, the moto s regstered the computer cremetally. he regstered behavor s retargeted to a vrtual humaod aget from the real world moto. he fal step s to sythesze the moto the aget by specfyg how may behavors are combed. I ths proposed scearo, rather tha regsterg the lmted jot descrptos, the system lears the hgher level behavor of each artculated object part. Moreover, determg optmal values of moto parameters such as trajectory ad velocty of the mapulator deped o the cofgurato of the workspace ad the structure of the target object. However, those descrptos requred a labelg procedure the learg phase. he exhaustve sample-labelg that s eeded makes expesve huma resources ecessary ad s ofte urealstc. For example, vsually-based automatc gesture trag, labelg data s a tme cosumg ad dffcult task. For other applcatos such as remote sesg or medcal magg, labelg data may requre very expesve tests ad so oly a small set of labeled data may be avalable. hus, oly a few postve samples are avalable, whle ulabeled examples are pletful. I all these cases, we eed to fd ways to releve the aotato burde for the users. herefore, we develop here a ew framework of trag behavor wth tal labeled staces usg cremetal learg methodologes, whch the rest of ulabeled trag behavors are cremetally executed the o-le maer. I comparso to other approaches of cremetal learg usg each statc mage [9,20,2,22,23], our proposed method s ew sce we cosder a sequece of mages as a sgle ut of sesory data for the postos of markers. Our cotrbuto ths system s that we have developed PCA to ft vdeo sequeces by cremetally updatg ege behavors a ole maer. he system automatcally recostructs the egespace by updatg a scatter matrx wth the ew ulabeled staces. We call these sequeces egespace ege behavors. Aother cotrbuto of the system that we have developed s a uque sytheszed process for geeratg autoomous behavors based o the ege behavors. Please ote that durg the tal PCA supervsed learg (before the ole cremetal PCA), the computer stll eeds to acqure the label of the tal behavors. o label each behavor, we wll descrbe a GUI developed as follows Advaced Robotcs (RSJ)

5 3.. Graphcal terface behavor edtor for artculated objects I the learg phase of behavor acqusto, the abstract models of Huma Fgure ad Humaod Robot have bee prepared. We have developed a Ope-GL based GUI edtor for the regstrato process. A humaod maequ or robot Kodo KHR- s used for artculated objects the real world. Each physcal model, however, s ot specfed detal. he umber of jots s assged by choosg from the meu the behavor-learg edtor. Fgure llustrates the regstrato process usg our developed GUI edtor. A huma operator specfes the umber of the artculated parts, whch correspods to the umber of the hghlghted ladmarks. he humaod structure follows Kodo KHR- specfcatos, smlar to the stadard of the Iteratoal Orgazato for Stadardzato (ISO) ad the Iteratoal Electrotechcal Commsso (IEC) as FCD 9774 Humaod vrtual robot [8]. (a) (b) (c) Fgure : Learg edtor (b) ad (c) for a humaod robot Kodo KHR- (a). he moto of the artculated objects s captured usg the camera sesor. 4. ONLINE BEHAVIOR LEARNING USING INCREMENAL PCA FOR SYNHESIS Our proposed learg system has several uque characterstcs for capturg the behavors of artculated agets. I order to lmt the umber of artculated behavor scearos, the robotc aget s oly allowed to lear a few typcal operatos wth represetatve postural tasks, such as pckg up a object. hs provdes examples for cosderg a small umber of sequetal artculated behavors. Ultmately we would lke to create effcet supervsed classfers, wth the goal of requrg a mmal amout of supervso. As llustrated Fgure 2, tal behavor staces, typcally umberg 40, are labeled for classfcato by a huma operator usg the GUI descrbed the prevous secto. hs frst step s called off-le learg, whch trag behavors are classfed usg a stadard PCA method (Secto 4.). I the ext step we develop a o-le learg methodology usg a structured learg etty. I our learg frame, we tally apply supervsed learg for small staces usg PCA, ad the we apply a ew automated framework usg a cremetal learg techque (Secto 4.2). Fally we wll dscuss the sythess module by utlzg the PCA framework. (Secto 4.3) Advaced Robotcs (RSJ)

6 Fgure 2: Overall learg procedure. 4.. Off-le PCA for tme sequetal data he represetato of the leared behavors plays a very mportat role determg how the learg algorthm works. Although we prepare the trag stace by settg the begg ad edg of behavor sequeces, each frame of the sequece s aalyzed for the mages from a camera sesor or from each data samplg of a data glove. We exted the data frames the followg way. ) Rather tha usg the etre set of 2D mage pxels to get a egemage (as [26]), here the data are oly extracted from the marker pots. hese pots are correlated to the geometrcal model of the artculated object. Let us deote a 2D mage. x ) l l l = ( u, v where l s the umber of the marker posto 2) For each kd of behavor, we wll take a sequece of mages q to completely classfy p types of behavors. More specfcally, we recostruct ths measuremet sequece by defg X to deote the -th sequece X [ x q ] l x l 2... x l =, = p, where x l j meas the j-th mage the -th behavor sequece. hus we have a matrx we calculate a 2l*q dmesoal sample mea matrx m by computg the scatter matrx S of mage sequeces X wth dmesos of 2l rows by q colums. Itally X as S = = m = X =. We defe ( X m)( X m). Usg the egevectors e j ( j= k) of Se j = λ je j, the p types of behavors X are decomposed by X = m + Ea, where E = [e e 2 e k ] ad = p. he prcpal compoet or scalar vector Advaced Robotcs (RSJ)

7 a s determed by a = E ( X m). I order to track the movemet of the humaod fgure precsely ad prevet terferece from the evromet, we put a color stcker o each jot of the humaod. For example, our expermet we have total of color stckers; each color stcker has a vertcal ad horzotal posto. herefore, the dmesoalty of each mage has bee reduced from 256*256 to 22*. Sce we have p types of behavors ad q mage sequece for each behavor, whe p ad q become large, we stll requre a lot of memory to store the data. By usg PCA, we ca reduce the dmesoalty of each mage, therefore reducg the amout of storage ecessary Icremetal o-le PCA for tme-sequetal data Durg the trag phase, we tra each behavor based o extracted sesory data. I the phase where ew, ulabeled sequeces are acqured, the system wll update the egevectors ad fd the closest behavor whch best represet the ew sequece. We wll apply a ew, cremetal PCA to the behavor classfcato. radtoal PCA uses batch computato. hat meas the etre set of trag mage sequeces are eeded to compute the kowledge represetato. Whe a ew mage sequece s to be corporated to the represetato, we must dscard the old represetato ad compute the + mage sequeces to get the ew represetato. herefore, order to process ew mages durg learg, the etre set of orgal trag mage sequeces must be stored. If the trag mages are very large, such a method wll cosume the storage of the aget. Istead, the use of cremetal PCA to represet the trag behavoral scees allows the reteto of oly the most mportat features. We ca update the kowledge by combg the old represetato of trag ad the ew mage sequeces. I ths way, we ca dscard the orgal mage sequeces oce they have bee used updatg. Sce we oly keep the reduced represetato of the mage sequeces, the storage effcecy s creased. We ca assume we have obtaed a ege subspace E = [e e 2 e k ] from mage sequeces X, =. he correspodg k largest egevalues are λ j, the prcpal compoet scalar matrx s a = a a... a ], =, ad the sample mea s m. Now, suppose a ew mage [ 2 k sequece X + s foud; we wll update the kowledgebase to take to accout ths ew mage sequece. Frst, we update the sample mea: m = ( m + X + ) () + We project the ew mage sequece to the old subspace E: a+ = E ( X + m) (2) Advaced Robotcs (RSJ)

8 he updated scatter matrx ca be obtaed: S = S + ( X + m)( X + m) (3) + I order to reflect the ew data from mage sequece X +, we must update the egevectors by solvg S e = λ e. he updatg process ca be summarzed as follows. Whe a ew mage sequece X + s receved, we compute the ew sample mea m ad scalar vector a + for the old subspace E, the costruct a updated subspace E, but wth o mage represetato. Let us deote that X () represets mage sequeces X the old subspace of E ad superscrpt meas that the mage sequeces are reduced by the egevectors. Also X (+) deotes the represetato of prevous mage sequeces X ad the ew mage data X + the updated subspace E. he: X ( ) Ea + m = ( ) = (4) I the ew subspace: X ( ) = E a( + ) + + m = + (5) We calculate the updated scalar value a ( +) by a + ) = ( E ) ( X ) = + (6) ( m Please ote that we do ot store the prevous mage sequece ew prcpal compoet Eq. (6) s represeted by: X, =, thus usg Eq. (4), the a E Ea + m m = ( ) ( ( ) )... ( + ) = ( E ) ( X m) = + (7) After updatg the kowledgebase, we eed to represet all the sesory data of mage sequeces aga the ew subspace, so that we do ot eed to keep the orgal mage sesory data X () of mages memory. Sce we oly store the reduced sesory data of mage sequeces, t s mportat to update these mages ad also keep the approxmatos provded by Eq. (7). I order to reduce the sze of the represetato, we attempt to lmt the umber of stored dmesos to k, but oly f ths umber does ot reduce the classfcato accuracy. hese k ew egevectors are sorted by decreasg order of the egevalues. We wll defe Icremetal O-le PCA as the codto whe the egespace k s expaded to k+ whe the ew egespace s computed wth ew stace X +. We wll desgate the classfcato codto as No-cremetal O-le PCA f the egespace matas the prevous dmeso after updatg. We ow troduce some crtera to judge whe the dmeso of the ege subspace should be expaded from k to k+ order to mata the desred balace betwee storage ad accuracy. wo Advaced Robotcs (RSJ)

9 crtera are evaluated by comparg thresholds to determe whether or ot the egespace eeds to be exteded. Crtero : he ew sesory data of mage sequeces at =+ caot be represeted wth the old subspace. hs occurs whe the error betwee the orgal mage ad the reduced mage X + X ( ) has exceeded a pre-set threshold. Crtero 2: If the ew sesory data of mage ( ) sequeces ca be represeted wth the old subspace, but the overall sum of error for + sesory data + from the mages X ( ) X ( ) has exceeded the threshold, we eed to crease the = dmeso of the subspace. herefore f oe of the error calculatos exceeds ts threshold, the ege space s exteded, applyg Icremetal O-le PCA Behavor sythess usg PCA-based trag for tme-sequetal data he system sytheszes a ew or bleded behavor usg the traed behavors, whch are labeled =. here are a umber of ways to geerate such a behavor. As a straghtforward approach, we wsh to establsh a methodology based o the reduced PCA space. As show the prevous subsecto, we have computed a scatter matrx ad a mea matrx over the etre set of trag data: S = ( X m)( X m), m = X (8) = = We decompose the scatter matrx to wth-class matrx S wc (scatterg each behavor at =,2,..c) ad betwee-classes matrx S B (scatterg across multple behavors). hat s S = S wc + S B, ad those are defed the form: S wc = c = S = c = j= where s the umber of data the -th class S B = c = ( X m )( X m ) (9) j j ( m m)( m m) (0) where m = X j ad a = EB ( m m) () j= he decomposto of scatter matrx s ot ew, t s a stadard method lear dscrmat aalyss (LDA). We mport ths dea for sytheszg a ew behavor by computg ege behavor of betwee-class scatter matrx. Suppose we have total c behavor classes (=,2,..c), the sytheszed behavor ca be recostructed usg betwee-class scatter matrx S B by solvg S e B j = λ e. Let j j re-deote E B = [e e 2 e c- ] for the egevectors. Now the system may geerate the ew behavor X sy = E a + m. Note that sce the behavor data format s the same as the ( ) B Advaced Robotcs (RSJ)

10 prevous secto of X [ x q ] l x l 2... x l =, the spato-temporal posto of ladmarks are already specfed ad ca used for the sytheszed behavors. 5. LEARNING PERFORMANCE OF AUOMAED ON-LINE PCA RAINING USING SEQUENIAL DAA I ths expermet, we have used color-pot markers for all movable jots of a humaod object show Fgure 3. A Pulx CCD color camera was used for capturg the color markers. We chose 6 represetatve huma fgure postos for partal behavoral sequetal tasks. he task starts from a flat stadg state, ad completes the locomoto by oe of the state chose from 6 fgure structures descrbed below. hese behavors are descrbed as M: rase the rght had, M2: rase the left had, M3: rase both hads, M4: rase the rght leg, M5: rase the left leg, ad M6: rase both legs. Each behavor was captured durg 4 secods for 6 frames. From the color mage, the 2D posto l l of each marker pot x l = ( u, v ) s extracted, where u,v s a 2D pot at each ceter posto of the -th jot (from = to ). hus, the tal off-le PCA trag space, the sesory mage sequece X has 22 rows ad 6 colums. Usg the 6 behavor staces, the overall sze of matrx [ X, X 2,.. X ] s 22*96. he dmeso of the correspodg scatter matrx S s 22*22, ad the ege behavor s at most 22, based o the matrx formato. I the cremetal o-le PCA trag phase, we may add ay addtoal sesory sequece X. he computato tme used for the cremetal PCA s maly due to computato of egevalues/vectors, ad s short eough to ot terfere wth capturg the ew mage data frame. Fgure 3: Sx behavoral trag tasks of a huma fgure, captured usg a camera sesor. Behavor tasks are added to the trag sequece for cremetal learg. radtoal off-le PCA was frst coducted usg the tal 6*5=30 trag behavors; the we appled the followg fve dfferet methods to hadle the ew behavors: Advaced Robotcs (RSJ)

11 ) the New rag method, whch volves classfyg the ew behavor usg the old egevectors. 2) the No-cremetal O-le method, whch updates the egevectors usg the ew behavor ad old recostructed behavors, matag the same egespace dmesos. 3) the Icremetal O-le method, whch updates the egevectors usg both the ew behavor ad the old recostructed behavors ad expads the egespace. 4) the No-cremetal Off-le PCA, whch volves addg the ew behavor to the orgal trag behavors ad updatg the egevectors, but keepg the same egespace dmeso. 5) the Icremetal Off-le PCA, whch ot oly updates the egevectors usg the ew ad old orgal behavors, but also expads the egespace. Fgure 4: otal error trasto wth respect to egespace dmeso across several PCA methods: New rag, No-cremetal O-le, Icremetal O-le, No-cremetal Off-le, ad Icremetal Off-le. he fttg measure was evaluated by comparg the measured dstace betwee the raw measuremet value X ad the recostructed value X () from Eq. (4) usg tradtoal O-le PCA at -th. Also, whle the ew data were acqured, the fttg measure was evaluated usg X (+) from Eq. (5) for O-le PCA at the +-th sequece ether cremetally or o-cremetally. o determe the ew dmesoalty of the ege subspace, we appled crtero ad 2 descrbed Secto 4.2. I other words, the error X ( ) X or X ( + ) X was computed wth respected to the umber of put egevalues (as demostrated o the X-axs Fgure 4). Whe the error was small, the recostructed value was close to that of the stace, whch dcated that the egespace expaded well to represet the sample stace. Fgure 4 compares the errors of all fve of the PCA methods descrbed above (New ra, No-cremetal O-le, Icremetal O-le, - - Advaced Robotcs (RSJ)

12 No-cremetal Off-le, Icremetal Off-le), across a umber of egespace dmesos. he absolute total error for O-le PCA was larger tha that for Off-le PCA for egespaces 8-22, but the dfferece betwee them was ot large. he dfferece of absolute total error betwee the No-cremetal ad Icremetal O-le PCA methods s early zero for egespaces However, for a small umber of egespaces (-8), the cremetal methods perform better terms of error reducto, both for O-le ad Off-le PCA performs better. Ulke Off-le PCA, our proposed method does ot store the prevous staces, oly the egespace; the space s recostructed usg the ew stace. he O-le PCA allows the learg system to dscard the acqured measuremet data mmedately after the update, reducg storage requremets. Fgure 4 llustrates that our O-le PCA method matas the recostructo accuracy of tradtoal Off-le PCA. 6. SYNHESIS PERFORMANCE OF AUOMAED ON-LINE PCA RAINING FOR HUMANOID MOIONS I ths expermet, the system sytheszed a ew or bleded behavor usg the traed ege behavors as descrbed Secto 4.3. We captured 50 mage frames for each sequece ad repeated 50 tmes for each category of behavor. he sytheszed performace was evaluated by comparg the correspodg actual behavor X actual wth the ew sytheszed behavor X sy = E a + m ). Please ote that the format for the spato-temporal posto of ( B ladmarks was detcal for the actual ad the sytheszed behavors, therefore the degrees of freedom ad graphcal models of the target humaod stayed the same. he evaluato crtero was defed by the absolute total accuracy ( X X sy / X ). actual actual Fgure 5: Overall accuracy whe actual ad sytheszed behavors are compared. For example, Fgure 5 shows that = was the closest to X actual X sy ( X sy(2) + X sy(5) ) the accuracy degree close to.0. hs result dcates that the chose behavors are dssmlar each Advaced Robotcs (RSJ)

13 other; therefore the sytheszed combato behavor would be expected to geerate a ew behavor. Note that our sythess method decomposes scatter matrx S to S wc +S B. he extracto of dscrmat factors betwee ege behavor 2 ad 5 are geerated by mmzg (subtractg) the wth-class porto ad thus maxmzg (ehacg) the betwee-class porto. As show Fgure 6, the output sytheszed behavor for the combato of M2 ad M5 cofrmed the accuracy of the sythess usg our ege behavor approach sce we ca easly compare t to the actual moto capture data. Fgure 6: Sytheszed vrtual robot sequece for combg ege behavors usg M2 ad M5. 7. PERFORMANCE OF SYNHESIS FOR AUOMAED ON-LINE PCA RAINING FOR HUMANOID MOIONS As a exteso of combg two behavors, we set a arbtrary terpolato by sytheszg across multple (more tha 2) ege behavors, that s X sy ( = E a + m ), where p, the umber of behavors to be sytheszed, wll be determed by a huma operator/desger. I our sythess framework, a arbtrary operato s restrcted so that the betwee-classes ca be maxmzed ad the wth-classes ca be mmzed. he crtera that we shall cosder are the trace ad the determat of the scatter matrces descrbed Secto 4.3. he trace crtero was chose ad represeted by maxmzg the followg crtero, called the relatve trace of the betwee-class p B scatter matrx: arg & p tr [ S tr [ S B p ] ] p = = p = j = ( m ( X j m )( m m )( X m ) j m ) (2) hs ew crtero measures the relatve square of the radus betwee ege behavors wth respect to the choce of p (umber of behavors to sythesze) ad (whch ege behavors). Please ote that S p = S wc + S B, ad the trace s proportoal to the sum of the varaces the coordate drectos. he proposed crtero s maxmzed order to evaluate the degree of dssmlarty betwee the chose ege behavors Advaced Robotcs (RSJ)

14 Fgure 7: Relatve race SB for comparso betwee chose combatos of ege behavors. Fgure 7 llustrates that the relatve trace was largest whe the most dssmlar ege behavors were chose. It s obvous that farly well separated behavors are expected. hs measure of separato dcates whch behavors may be bleded to sythesze atural sythess. Such a measure s possble because hghly correlated behavors caot be used to effcetly sythesze arbtrary behavors. Fgure 8 llustrates the sytheszed results usg 4 behavors from 6 ege behavors. For example, M-M2-M4-M5 behavors are chose amog C 5 cases because of 6 4 = the relatve trace crtero. Sce the actual behavor meets all requred verse kematcs for jots, the above results would cofrm that the sytheszed behavor follows the related costrats across the jots. I comparso to other studes [3, 25], our sytheszed method s very effcet terms of both computatoal tme ad data space requremets, because we do ot solve the teratve costrat optmzato problem, whch always suffers from local mmum ad covergece ssues. Future work would clude the quattatve comparso of our method wth other methods that utlze kematc costrat-based optmzato. Also, the behavor sythess resultg from our method (. e. the example show Fgure 8) should be evaluated through a subjectve ratg by huma operators. Fgure 8: Sytheszed vrtual robot sequece for combg ege behavors usg M-M2-M4-M Advaced Robotcs (RSJ)

15 8. CONCLUSION I ths paper, we have developed a o-le learg method for mtatg artculated behavors ad utlzed these behavors to help sythesze ew behavors. We have appled a cremetal learg framework by updatg the classfers from tal supervsed PCA output. ypcally, tradtoal PCA has prmarly bee used a off-le maer. I cotrast to off-le PCA method, whch eeds to store all the orgal prevous staces ad ew staces order to recostruct the egespace, our proposed o-le method does ot eed to keep prevous staces, oly the egespace. he space s the recostructed usg each ew put stace. Sce vdeo-based motos are our measuremet data, the ew staces arrve as a cotuous stream that s too expesve to store. I our o-le PCA, we ca add may ew trag staces whle matag reasoable egespace dmesos. We have developed automated cremetal learg methodologes usg the postos of lad markers each mage sequece as a ut of moto capture data. Each set of lad marker data s used as a trag behavor ad cremetally executed as ege behavors a o-le maer. he expermetal results demostrated the feasblty ad merts of the reducto of the learg dmeso usg our o-le cremetal PCA. We observed that a major factor the desg of the autoomous behavor s the avalablty of a tal, prcple set of ege behavors, ad the ablty to expad the set by bledg dssmlar (ucorrelated) ege behavors. hus we set a framework to aalyze the scatter matrx for ege behavors to betwee-classes ad wth-classes, ad to sythesze the betwee-classes ege behavors. We troduced the crtero of the relatve trace of betwee-class scatter matrx to determe whch ege behavors should be chose. he greatest advatage of our proposed sytheszed approach s that t s extremely effcet both computatoal tme ad data space requremets, whle the other methods requre solvg costrat-based teratve optmzato problems, whch always suffers local mmum ad covergece ssues. As a remag ssue, the o-le trag ad sythess performace usg the ege behavor should to be evaluated a log-term trag phase to acqure a larger database for trag ad testg. ACKNOWLEDGEMENS he author ackowledges J. Brooks Zur for proof readg ad commets. REFERENCES [] Vco Moto Systems, Oxford Metrcs Ltd. [O-le] [2] A.P.A.S, Arel Dyamcs, [O-le] [3] Z. Popovc ad A. Wtk, Physcally Based Moto rasformato, Proc. of SIGGRAPH, pp. -20 (999) Advaced Robotcs (RSJ)

16 [4] J. K. Hodgs ad N.S. Pollard, Adaptg Smulated Behavors For New Characters, Proc. of SIGGRAPH, pp , (997) [5] A. Bruderl ad L. Wllams, Moto Sgal Processg, Proc. of SIGGRAPH, pp (995) [6] A. Wtk ad Z. Popovc, Moto Warpg, Proc. of SIGGRAPH, pp (995) [7] H.C. Su ad D. N. Metaxas, Automatg gat geerato, Proc. of SIGGRAPH, pp (200) [8] [O-le] [9] [O-le] [0] C. Stauffer ad W.E.L. Grmso, Learg patters of actvty usg real-tme trackg, IEEE rasactos o Patter Aalyss ad Mache Itellgece, 22 (8) pp (2000) [] C.R.Wre, A. Azarbayeja,. Darrell, A.P. Petlad, Pfder: real-tme trackg of the huma body, IEEE rasactos o Patter Aalyss ad Mache Itellgece, 9 (7) pp (997) [2] I. Hartaoglu, D. Harwood, L.S. Davs, W 4 : real-tme survellace of people ad ther actvtes, IEEE rasactos o Patter Aalyss ad Mache Itellgece, 22 (8) pp (2000) [3] A. Fod, M. Matarc, ad O. C. Jeks, Automated Dervato of Prmtves for Movemet Classfcato, Autoomous Robots, 2 () pp (2002) [4] O. C. Jeks, M. J. Matarc, ad S. Weber, Prmtve-Based Movemet Classfcato for Humaod Imtato, Proc. of Iteratoal Coferece of Humaod Robotcs (2000) [5] Moto Aalyss Ic, [O-le] [6] P. Gausser, S. Moga, J.P. Baquet, M. Quoy, N, CREARE. From percepto-acto loops to mtato processes: A bottom-up approach of learg by mtato, Iteratoal Joural of Appled Artfcal Itellgece, 2, pp (998) [7] S. Schaal, Is mtato learg the route to humaod robots, reds Cogtve Sceces, 3 (6), pp (999) [8] J. Demrs, G. Hayes, Imtatve learg mechasms robots ad humas, Proc. of Europea Workshop o Learg Robots, pp. 9-6 (996) [9] Y. L, L. Q. Xu,J. Morphett, R. Jacobs, A tegrated algorthm of cremetal ad robust PCA, Proc. of Iteratoal Coferece o Image Processg, pp (2003) [20] M. Artac, M. Joga, A. Leoards, Icremetal PCA for o-le vsual learg ad recogto, Proc. of Iteratoal Coferece o Patter Recogto, pp (2002) [2] M. Artac, M. Joga, A. Leoards, Moble robot localzato usg a cremetal egespace model, Proc. of IEEE Iteratoal Coferece o Robotcs ad Automato, pp (2002) [22] P. Hall, D. Marshall, ad R. Mart, Icremetal egeaalyss for classfcato, Proc. of Brtsh Mache Vso Coferece, pp (998) Advaced Robotcs (RSJ)

17 [23] S. L. Nayar, S. A. Nee, ad H. Murase, Subspace methods for robot vso, IEEE rasactos o Robotcs ad Automato, 2 (5), pp (996) [24] P. Maes,. Darrell, B. Blumberg, A. Petlad, he ALIVE System: Wreless, Full-body Iteracto wth Autoomous Agets, ACM Multmeda Systems, 5 (2), pp (997) [25] A. Safoova, J. K. Hodgs, N. S. Pollard, Sytheszg physcally realstc huma moto low-dmesoal, behavor-specfc spaces, Proc. of SIGGRAPH, pp (2004) [26] F. De La orre, ad M. J. Black, A framework for robust subspace learg, Iteratoal Joural of Computer Vso, 54, pp (2003) [27] O. C. Jeks ad M. J. Matarc, Dervg acto ad behavor prmtves from huma moto data, Proc. of IEEE/RSJ Iteratoal Coferece o Itellget Robots ad Systems, pp (2002) [28] Y. L,. Wag, ad H.-Y. Shum, Moto texture: a two-level statstcal model for character moto sythess, ACM ras. o Graphcs, pp (2002) [29] Z. Lu, ad M. Cohe, Decomposto of lked gure moto: Dvg, Proc. of Eurographcs Workshop o Amato ad Smulato (994) [30] C. K. Lu, ad Z. Popovc, Sythess of complex dyamc character moto from smple amatos, ACM rasactos o Graphcs, 2 (3), pp (2002) [3] A. Petlad, ad J. Wllams, Good vbratos: Modal dyamcs for graphcs ad amato, Proc. of SIGGRAPH, 23, pp (989) [32] J. Popvc, S. M. Setz, M. Erdma, Z. Popovc, ad A. P. Wtk, Iteractve mapulato of rgd body smulatos, Proc. of SIGGRAPH, pp (2000) [33] C. F. Rose, B. Gueter, B. Bodehemer, ad M. F. Cohe, Effcet geerato of moto trastos usg spacetme costrats, Proc. of SIGGRAPH, pp (996) [34] M. Satelo, M. Fladers, ad J. F. Soechtg, Patters of had moto durg graspg ad the uece of sesory gudace, Joural of Neuroscece, 22, pp (2002) Advaced Robotcs (RSJ)