As-Rigid-As-Possible Image Registration for Hand-drawn Cartoon Animations

Size: px
Start display at page:

Download "As-Rigid-As-Possible Image Registration for Hand-drawn Cartoon Animations"

Transcription

1 As-Rgd-As-Possble Image Regstraton for Hand-drawn Cartoon Anmatons Danel Sýkora Trnty College Dubln John Dnglana Trnty College Dubln Steven Collns Trnty College Dubln source target our approach [Papenberg et al. 2007] [Glocker et al. 2008] Fgure 1: Compared wth the state-of-the-art n deformable mage regstraton, our novel approach reaches plausble results even for challengng confguratons undergong large amounts of free-form deformaton and notable changes n appearance. Abstract We present a new approach to deformable mage regstraton sutable for artculated mages such as hand-drawn cartoon characters and human postures. For such type of data state-of-the-art technques typcally yeld undesrable results. We propose a novel geometrcally motvated teratve scheme where pont movements are decoupled from shape consstency. By combnng locally optmal block matchng wth as-rgd-as-possble shape regularzaton, our algorthm allows us to regster mages undergong large free-form deformatons and appearance varatons. We demonstrate ts practcal usablty n varous challengng tasks performed n the cartoon anmaton producton ppelne ncludng unsupervsed nbetweenng, example-based shape deformaton, auto-pantng, edtng, and moton retargetng. CR Categores: I.4.3 [Image Processng and Computer Vson]: Enhancement Regstraton; I.3.4 [Computer Graphcs]: Graphcs Utltes Graphcs edtors; J.5 [Computer Applcatons]: Arts and Humantes Fne arts Keywords: deformable mage regstraton, as-rgd-as-possble deformaton, nteractve shape manpulaton 1 Introducton In a tradtonal cartoon anmaton each anmaton frame s drawn by hand so that the correspondences between them are unknown. Such a drawback consderably lmts the usage of tradtonal approaches n recent computer-based anmaton systems, where the knowledge of correspondences between ndvdual key-frames plays an mportant role. e-mal: Obtanng correspondences automatcally s a challengng task snce each hand-drawn mage s unque and typcally undergos a large amount of free-form deformaton and notable change n appearance. In ths context popular computer vson technques based on local smlarty [Lowe 2004] or global contexts [Belonge et al. 2002] fal snce they rely on unque local features or stable global confguratons. Although such features are typcal for real world photographs they are rare n hand-made drawngs. Moreover, the aforementoned technques provde only solated pont correspondences and do not consder global consstency. Thus, they can easly lead to spatally nconsstent mappng. A more powerful approach to ths problem s deformable mage regstraton [Mantz and Vergever 1998; Moderstzk 2004; Gholpour et al. 2007] whch allows the retreval of dense correspondences between mages and smultaneously mantans spatal consstency of the resultng mappng. It s typcally formulated as a nonlnear optmzaton problem where a predefned energy functon s mnmzed through some establshed numercal optmzaton technque [Klen et al. 2007]. However, there are two key dffcultes whch make the soluton challengng: (1) non-convexty of the energy functon and (2) senstvty to outlers (.e. appearance varatons that do not ft the selected deformaton model). To overcome these, an ntal guess close to the global mnma s requred. It can be obtaned through varous heurstcs such as the popular mult-scale approach [Lucas and Kanade 1981] or by usng a herarchy of deformaton models [Bergen et al. 1992]. Unfortunately, for large dsplacements or appearance varatons, even these heurstcs yeld erroneous results. Ths fundamental problem has been recently addressed by technques that attempt to mnmze the energy not through the teratve numercal optmzaton, but drectly va dscrete labellng [Glocker et al. 2008; Shekhovtsov et al. 2008]. These are bult upon recent advances n algorthms for nference from random felds [Szelsk et al. 2008] whch allow fast approxmate solutons to non-lnear problems wth effectve avodance of local mnma. Nevertheless, they stll do not a guarantee global optmum (snce the problem s NP-hard) and become computatonally ntractable for large dsplacements due to sgnfcantly ncreasng number of labels. In ths paper we develop a new approach to deformable mage regstraton whch addresses the ssue of local mnma and s able to reach a desred pose even from large ntal dsplacements or no-

2 Fgure 2: Deformable mage regstraton n progress we want to regster a straght strpe (flled wth transparent color and embedded n a square lattce) wth ts S-shaped counterpart flled wth lght blue color (column 0). In each teraton (columns 1 4) two steps are repeated: ponts are frst pushed towards locatons wth mnmal vsual dfference wthout consderng shape consstency (left) and then the shape s regularzed usng a varant of as-rgd-as-possble shape matchng algorthm (rght). Note, how the shape gradually approaches the desred confguraton. table changes n appearance. It s nspred by the success of recent work n real-tme smulaton of deformable objects [Müller et al. 2005; Rvers and James 2007], where ponts are pushed drectly towards desred postons and then as-rgd-as-possble shape regularzaton s used to ensure consstency of the orgnal shape. In our case, ponts are not nfluenced by gravty or nertal forces, but nstead attracted to locatons wth mnmal vsual dfference. A key beneft of ths new approach les n the fact that the aforementoned shfts can be arbtrary and only the shape regularzaton ensures consstency. Ths s the fundamental dfference to numercal optmzaton where ncremental shfts are predcted by mnmzng locally lnearzed verson of the energy functon, whch typcally leads to an napproprate local mnma. Our new technque s closer to the approaches based on dscrete optmzaton n the sense t can recover from napproprate local mnma and lead to more plausble results. However, the key dfference s that we do not mnmze an energy functon but nstead use a geometrcally motvated shape regularzaton scheme whch preserves local rgdty and does not requre computatonally demandng dscrete optmzaton napplcable to large ntal dsplacements. Snce our work s manly motvated by the needs of the cartoon anmaton producton ppelne, we demonstrate the practcal usablty of our new algorthm n ths context. We show how t can reduce the amount of manual work n tasks such as nbetweenng, pantng, retargetng, shape deformaton, and reusng tradtonal anmaton. We beleve that these examples demonstrate the practcal potental of our new technque and wll motvate developers of recent professonal cartoon anmaton systems to ncorporate our technque as a versatle buldng block applcable to varous practcal scenaros. The rest of the paper s organzed as follows. Frst we brefly overvew related work and analyze key drawbacks whch motvated us to develop a new approach. Then we descrbe the proposed algorthm n more detal and dscuss ts strengths and lmtatons. Fnally we demonstrate ts practcal usablty n the context of the cartoon anmaton producton ppelne and conclude wth several deas for future work. 2 Related work Obtanng correspondences between hand-drawn mages s a challengng task that has captured the attenton of many researches wthn the last two decades. Xe [1995] used smple affne transformatons to match two lne drawngs and perform automatc nbetweenng. Madera et al. [1996] poneered a regon-centered approach where drawngs are frst sub-dvded nto regons and then matched usng shape smlarty and topologcal relatons. Ths approach has been later mproved by several authors who assume addtonal semantc nformaton about the mage [Kort 2002], cater specfcally for black-and-whte cartoons [Sýkora et al. 2005], rely on a herarchy of regons [Qu et al. 2005] or employ skeleton matchng [Qu et al. 2008]. Ther common lmtaton s that they are applcable only to specfc easy-to-analyze drawng styles and do not provde dense correspondences. Bregler et al. [2002] presented a more general approach n ther famous framework for cartoon moton capture. They overcome the problem of deformable regstraton by samplng the space of possble deformatons usng as-rgd-as-possble nterpolaton [Alexa et al. 2000] and then nfer optmal lnear combnatons of these samples to ft the target pose. Although ths approach works n the context of moton retargetng, t s not applcable to our problem snce t does not drectly provde dense correspondences between two anmaton frames. De Juan and Bodenhemer [2006] utlze dense correspondences n ther framework for re-usng tradtonal anmaton. However, they completely rely on manual ntalzaton and refne dense mappng usng an exstng algorthm [Wrtz et al. 2004] based on numercal optmzaton. Recently, Xu et al. [2008] proposed a system for anmatng moton from several snapshots captured n a sngle mage. However, snce they rely on shape contexts [Belonge et al. 2002], unstable under large free-form deformatons, extensve manual nterventon s needed to dentfy stable features. 3 Our approach Our novel approach to deformable mage regstraton stems from the successful workflow recently used for dynamc smulaton of deformable objects by Müller et al. [2005] and later extended by Rvers and James [2007]. A core dea of ths technque s to decouple pont movements from shape consstency so that the physcal smulaton can be appled drectly on ponts, treatng them as partcles wthout consderng ther mutual connectvty. To keep the shape consstent, a geometrcally motvated shape matchng phase s then used to regularze pont locatons. A key observaton s that such a workflow can brng sgnfcant beneft also to deformable mage regstraton snce t allows retreval of locally optmal shfts and stll keeps the shape consstent. Ths s n contrast to energy-based approaches where shfts are lmted by the deformaton model whch does not allow temporary ncrease of the overall energy and so typcally leads to undesrable local mnma.

3 target push 1 regularze 2 output source Fgure 3: A schematc overvew of the proposed algorthm the am s to regster a lght blob (source) wth ts darker counterpart (target). The lght blob s embedded n a square lattce (partly vsble). The algorthm terates two man phases: push (yellow part) & regularze (green part). In the pushng phase block matchng s used to move lattce ponts towards locatons where a sum of absolute dfferences over a local neghborhood (red square) s mnmal. Then n the shape regularzaton phase two steps are terated: (1) optmal rgd transformaton s computed for each lattce square and then (2) lattce ponts are moved to the centrod of ther shared nstances n all connected squares. The only necessary modfcaton as compared to the orgnal concept s to replace physcally motvated forces by an attracton to locatons wth vsually smlar neghborhood. Based on ths setup, the resultng algorthm s as follows. Smlarly to [Rvers and James 2007] we embed the nput mage nto a regular square lattce respectng ts artculated shape and then terate two followng steps untl a stable confguraton s reached: 1. Push all ponts to locatons wth mnmal vsual dfference (Secton 3.1). 2. Regularze the pont locatons to keep the shape consstent (Secton 3.2). These steps are llustrated n Fgure 3 and a practcal example s presented n Fgure 2. Note how n each teraton, ponts are frst pushed arbtrarly towards desred locatons and how the overall shape becomes messy. Nevertheless, after regularzaton the shape s consstent and better fts the target pose. In the followng sectons we descrbe these two steps n more detal, dscuss mplementaton ssues, stoppng crtera, and possble lmtatons. 3.1 Push The am of the pushng phase s to fnd a new locaton for each pont on the embeddng lattce that mnmzes vsual dfference n ts local neghborhood. Snce we are not lmted by shape consstency we can utlze smple block matchng whch guarantees globally optmal shft wthn a predefned search area (see Fgure 3, yellow part). Formally, the am s to fnd a shft vector t from search area M that mnmzes the sum of absolute dfferences over a neghborhood N,.e.: t = arg mn S(p + t) T(p) (1) t M p N where S denotes the source and T the target mage. Note that n spte of shft optmzaton, the overall algorthm s not lmted to pure translaton, snce S s slghtly deformed n each teraton local neghborhoods of ponts gradually adapt to more complcated deformatons. The mportant parameters of the block matchng phase are the sze of the neghborhood N and the sze of search area M. In general N should be large enough to contan substantal nformaton but also small enough to preserve localty, whereas M should allow attracton to further locatons but also avod ambguty. The other lmtng factor s the computatonal overhead whch can ncrease dramatcally due to the worst case complexty of the block matchng algorthm O( N M ). If we consder that n each teraton typcally hundreds of block matchng operatons are executed, the complexty can be very hgh even for small neghborhoods and search areas. However, due to the fact that optmal block postons typcally reman constant durng most teratons and all other shfts have much hgher sums of absolute dfferences, the early termnaton heurstc [L and Salar 1995] can be used to gan consderable speed up. Based on ths observaton, we set the wdth of N to 16 pxels and the wdth of M to 48 pxels (provdng that mages are n PAL resoluton). Ths settng yelds good results both n robustness and computatonal overhead n all examples shown n ths paper. 3.2 Regularze The second step of our algorthm s a geometrcally motvated routne that teratvely regularzes the pont locatons so that local rgdty of the shape s preserved. Ths s another mportant dfference from state-of-the-art technques, whch typcally use elastc models that do not preserve rgdty and produce undesrable deformatons when the ntal dsplacements are large or when there s a notable varaton n appearance (see Fgure 1). In Müller s orgnal algorthm, the am was to fnd an optmal rgd transformaton (rotaton R and translaton t ) that moves ponts of the orgnal shape p P so that the sum of squared dstances to the desred pose q s mnmzed: (R, t ) = arg mn R,t R p + t q 2 (2) In 3D the computaton of R s non-lnear, thus polar decomposton s requred to solve ths problem. However, as shown by Schaefer et al. [2006], a smple closed form soluton exsts n 2D. It can be obtaned when we compute centrods p c and q c of the source and target pose and then substtute ˆp = p p c and ˆq = q q c : where R = 1 µ ( ˆp ˆp ) ( ˆq T ˆq T ), (3) ( ) 2 ( ) 2 µ = ˆq ˆp T + ˆq ˆp T, (4) T denotes transposton, and the operator denotes the perpendcular vector,.e.: (x, y) = (y, x). Once the rotaton matrx R s known, the translaton vector t can be computed drectly: t = p c R q c (5)

4 Our embeddng lattce conssts of several connected squares. In ths case local rgd transformatons are computed ndvdually for each square and then the global smoothng step s used to ensure consstency. Ths smple extenson enables more flexble deformatons and stll preserves local rgdty of the orgnal shape (see Fgure 4). Fgure 4: An example of as-rgd-as-possble mage deformaton the orgnal mage embedded n a square lattce (left) and ts deformed counterpart (rght). A smlar mechansm s also used n the context of nteractve shape deformaton [Igarash et al. 2005; Sumner et al. 2007; Botsch et al. 2007; Sorkne and Alexa 2007], however, the key dfference s that n these technques user-specfed pont locatons are treated as hard constrants and the am s to fnd an optmal deformaton to satsfy them. In our case we do not have hard constrants. What we want s to smooth out pont locatons so that the shape becomes consstent. To perform ths smoothng we explot a very smple teratve approach nspred by recent work n nteractve shape deformaton [Wang et al. 2008]. It produces smlar results to [Rvers and James 2007] but allows smooth control over shape rgdty (see Fgure 3, green part): 1. For each square on the embeddng lattce, use equatons (3), (4) and (5) to obtan (R, t ) and use ths to transform ts ponts. 2. Move each pont on the embeddng lattce to the centrod of ts transformed nstances n all connected squares. The only dfference to the orgnal shape deformaton technque s that Wang et al. addtonally smulate hard constrants by settng very large weghts to ponts that represent manpulaton handles. In fact ther algorthm s nearly dentcal to [Sorkne and Alexa 2007], where nstead of computng centrods a sparse lnear system s solved. Ths modfcaton clarfes the aforementoned dfference between shape regularzaton and nteractve shape deformaton. In our case no ponts are fxed therefore after suffcent number of teratons the shape wll return to the orgnal confguraton up to some global rgd body transformaton. Such behavor s depcted n Fgure 5 where one pont s fxed at a dfferent locaton and then the evoluton of the deformaton s captured durng several shape regularzaton teratons. Intally the shape s flexble but wth an ncreasng number of teratons, global rgdty s enforced so that the deformaton gradually reduces to pure translaton. Ths s caused by the dffusve nature of the averagng phase whch gradually propagates rgdty through the whole shape. Ths gradual dffuson of rgdty has several practcal applcatons. By changng the number of shape regularzaton teratons we can smoothly vary between rgd and flexble deformaton. It allows us to mplement a smooth analogy to a herarchy of deformaton models [Bergen et al. 1992] (see Secton 3.4) and also one-pont nteractve shape deformaton (see Secton 4). 3.3 Stoppng crtera Although our method does not explctly mnmze predefned energy functon we can stll estmate plausblty of the resultng regstraton by computng the average sum of absolute dfferences over all blocks durng the block matchng phase. In Fgure 9 there are several graph plots of ths average (blue curve) measured durng a hundred teratons for dfferent regstraton tasks. In most cases ths value decreases monotoncally and after several teratons the change s neglgble. However, when a part of the shape undergoes a large non-overlappng deformaton the change can be neglgble for several teratons (see Fgure 9b) and after that perod the algorthm suddenly approaches new confguratons wth much lower value. Ths s caused by the fact that although a part of the mage moves towards the desred pose t stll remans n an area wth no overlap where the sum of absolute dfferences s nearly constant. To overcome such ambguty we nstead montor the average dstance to the ntal rest pose (red curve n Fgure 9): d avg = 1 P p q (6) Ths value nforms us whether the control ponts on the embeddng lattce are movng or not. We stop push-regularze teratons when d avg has not changed consderably n the last 20 teratons. Fgure 5: The evoluton of the shape deformaton through several shape regularzaton teratons one pont s fxed at dfferent locaton (left). Durng the frst teratons the shape s flexble but when the number of teratons ncreases the deformaton gradually approaches pure translaton (from left to rght). 3.4 Lmtatons Although our approach produces good results n most practcal scenaros, there are some lmtng factors whch should be taken nto account snce they can lead to unexpected behavor. In ths secton we dscuss these n more detal and address how they can be dealt wth. Lmted resoluton. Snce we embed the mage nto a coarse lattce we cannot drectly obtan pxel or even sub-pxel precson. Although a mult-scale extenson s possble, ncreasng the number of squares makes the overall teratve process neffectve. Ths s caused manly by an ncreasng number of block matchng nstances and shape regularzaton teratons. However, the coarse approxmaton produced by our algorthm s typcally close enough to the desred pose so that classcal energy-based approaches can be utlzed to refne the regstraton to sub-pxel precson (we use a publcly avalable mplementaton of [Glocker et al. 2008]). Occluson and topology. The presence of occluders and topology varatons n 2D projectons of 3D artculated objects s a longstandng and challengng computer vson problem. It also lmts

5 1 st phase 2 nd phase 3 rd phase example-based deformaton + + Fgure 6: Example-based shape deformaton by regsterng several consecutve anmaton phases (left) a smooth sequence of ntermedate frames can be generated. Ths can be utlzed for a synthess of new poses satsfyng a user-gven postonal constrant (rght): the current poston of the dragged pont (red dot) s projected (blue dot) on ts key-frame trajectory (red curve) to retreve the correspondng ntermedate frame whch s subsequently deformed to match the actual poston of the dragged pont. the usage of our algorthm snce occluded parts move together wth ther occluders and topology varatons mpose false connectvty. A possble soluton to ths problem s to reconstruct a layered representaton of the mage where each layer has ts own depth nformaton and shares common control ponts wth other layers. Usng ths structure one can perform the pushng step only between layers havng equal depth and then use the shape regularzaton phase to propagate these movements to other connected layers. Scalng and shearng. As our method explots the as-rgd-aspossble deformaton model t s not able to handle deformatons whch do not preserve local rgdty (such as scalng or 3D rotaton). Ths lmtaton can be partally reduced by explotng an approach analogous to the use of a herarchy of deformaton models. Intally we can treat the mage as more rgd and perform a hgher number of rgd shape matchng teratons. After that the number s gradually decreased to allow more flexble deformatons. However, even usng ths extenson, sgnfcant changes n scale and/or shearng are stll ntractable. In cases when such deformatons are requred we can swtch to a dfferent local deformaton model allowng smlarty or even affne transformatons. Accordng to [Schaefer et al. 2006] for smlarty ths can be done by replacng (4) wth: µ = ˆp ˆp T (7) and for affne transform by replacng (3) and (5) wth a full affne matrx: ( ) 1 A = ˆp T ˆp ˆp T j ˆq j (8) However, snce smlarty and affne models do not tend to preserve area they are not as stable as the orgnal as-rgd-as-possble model therefore are sutable only for fnal refnement when the source and target pose are close enough, otherwse they can dstort the mage consderably and lead to unacceptable results. Insuffcent overlap. A key feature of our technque s the ablty of the block matchng phase to overcome confguratons whch correspond to local mnma n energy-based technques. However, to avod matchng ambguty, the sze of searchng wndows has to be lmted. Because of ths reason, our method requres partal overlap and consstent scale & orentaton between source and target mages. For mages that do not satsfy these requrements we recommend that the ntal rgd-body transformaton be estmated by hand or that some automatc rgd-body regstraton technque should be used. When the nsuffcent overlap s caused by large free-form deformaton (as n Fgure 9h), the algorthm may get stuck n some napproprate pose. In ths case, the user can j provde addtonal hnts by draggng a problematc part towards a desred poston or use bdrectonal regstraton,.e. to alternate pushng and regularzaton steps on both source and target mage. As compared to sngle mage deformaton where the target mage s statc, ths approach provdes better flexblty and so can overcome challengng confguratons. 4 Results and Applcatons We mplemented our algorthm and tested t on varous hand-drawn cartoon characters and human postures undergong both small and large free-form deformatons and changes n appearance. Selected results are presented n Fgures 1 and 9. All examples are n PAL resoluton. The wdth of squares on the embeddng lattce s the same as the wdth of neghborhood N n equaton (1),.e. 16 pxels (blue squares n Fgure 9) and the wdth of the search area M s 48 pxels (red squares). The number of nner teratons n the shape regularzaton phase s lnearly decreased from 256 to 32 durng the frst 50 push-regularze steps. To have an unfed overvew of the algorthm convergence we measured the average sum of absolute dfferences (blue curve) and the average dstance to the startng pose (red curve) durng the frst 100 teratons for all examples n Fgure 9. The actual number of teratons needed to reach stable confguraton vares wth the complexty of deformaton. In smple cases t does not exceed 30, however, for large deformatons such as human postures n Fgures 9f and 9g t ncreases to 80. A typcal processng speed s 20 teratons per second on a 3 GHz machne whle the most demandng part s the block matchng phase. However, t can be easly parallelzed and thus much better processng speeds could be reached on some parallel archtectures. As stated n Secton 3.4 the accuracy of our algorthm depends manly on the resoluton of the embeddng lattce. Such precson s typcally suffcent for applcatons where exact dense correspondences are not requred such as auto-pantng or moton capture. However, for nbetweenng and example-based shape deformaton, where smooth transtons are requred, subsequent refnement s necessary to obtan sub-pxel accurate dense mappng. When a local appearance does not change consderably t s possble to take the result of our method as an ntal guess for an energy-based approach (we use [Glocker et al. 2008]) and obtan refned sub-pxel accurate mappng. In Fgure 9 we show both the regstratons produced by our algorthm and also the correspondng refned results. In the rest of ths secton we dscuss several applcatons. Snce our work s manly motvated by the needs of the professonal cartoon anmaton producton ppelne we focus on ths feld. However, we beleve that our algorthm s versatle enough to be appled

6 LazyBrush regstraton scrbble transfer LazyBrush Fgure 7: Auto-pantng by unsupervsed scrbble transfer color scrbbles can be transferred from already panted to yet unpanted anmaton frames usng our deformable mage regstraton algorthm. The LazyBrush [Sýkora et al. 2009] algorthm s then utlzed to compute the fnal pantng. n other contexts such as pedestran regstraton, deformable object snappng or mprovng nteractve shape deformaton by provdng dynamc feedback that looks lke physcal smulaton. Unsupervsed nbetweenng. The knowledge of dense correspondences between several consecutve frames allows us to create smooth ntermedate transtons. One possblty for achevng ths s to lnearly nterpolate postons of correspondng pxels. However, ths s applcable only for small motons snce the local rgdty s not preserved. A better soluton s to dvde the transton to coarse and fne level. The coarse level conssts of the same square lattce as used for regstraton and the fne level s represented by two dense dsplacement maps (source-target and target-source) computed by the energy-based method [Glocker et al. 2008]. To generate the ntermedate frame we frst lnearly nterpolate the coarse lattces of the source and target frame and perform several shape regularzaton teratons to enforce rgdty. On the pxel level we scale transformed source-target and target-source dsplacements and resample source and target mages accordngly. Fnally we blend co-located pxels to obtan C 0 contnuty. Example-based shape deformaton. User-drven shape deformaton based on ntutve postonal constrants has recently become popular partcularly due to the work of Igarash et al. [Igarash et al. 2005]. Although many researchers attempt to mprove ths technque [Schaefer et al. 2006; Weng et al. 2006; Wang et al. 2008] they stll offer only sngle mage deformaton. Thanks to our deformable mage regstraton algorthm, we can easly extend ths technque to multple mages (see Fgure 6). By regsterng several anmaton phases we obtan smooth transtons as we do for nbetweenng, however, a key dfference here s that we allow the user to drag a specfc pont and move t to a dfferent locaton. We project ths new locaton on ts nbetweenng trajectory and generate a closest transton frame that s subsequently deformed to match the user-specfed poston. Ths enables nteractve shape deformaton whch respects the orgnal anmaton but s more flexble than smple nbetweenng. Moreover, the ease of manpulaton s mproved consderably snce n contrast to classcal approaches we do not need to place other postonal constrants to fx the global pose. Instead we apply a lower number of shape regularzaton teratons as descrbed n Secton 3.2 to suppress the dffuson of rgdty so that parts of the shape havng long geodesc dstances from the selected pont reman untouched. Auto-pantng and edtng. The process of addng colors to handdrawn mages s one of the most challengng tasks n the classcal cartoon anmaton ppelne. In the last decade researchers have developed varous auto-pantng approaches allowng sgnfcant reducton of manual effort [Madera et al. 1996; Chang and Lee 1997; Seah and Feng 2000; Sýkora et al. 2005; Qu et al. 2008]. As these technques explot smlarty of regons they requre drawng styles that can be easly converted to a set of homogenous regons. Recently, Sykora et al. [2009] ntroduced a more general approach based on color scrbbles that s applcable to a broad class of dfferent drawng styles. By regsterng panted and yet unpanted frames, we can transfer scrbbles between anmaton frames and consderably speed up the process (see Fgure 7). Besdes pantng, smlar workflow can be utlzed to perform varous edtng operatons, e.g. retouchng, nserton, and deleton. Moton capture and retargetng. In ths applcaton poneered by [Bregler et al. 2002] the am s to transfer specfc moton captured n a sequence of mages to a novel anmaton havng a dfferent vsual appearance. In our case ths can be done by supermposng a skeleton on a reference pose and then usng deformable mage regstraton to fnd correspondng postons of jonts and bones n subsequent anmaton frames (see Fgure 8). Moreover, the supermposed skeleton can be utlzed to form a set of rgd clusters and perform skeletal-lke deformaton va rgd square matchng as n [Wang et al. 2008]. 5 Conclusons and Future work We presented a new approach to deformable mage regstraton based on an approach analogous to the dynamc smulaton of deformable objects. In contrast to prevous technques t handles large free-form deformatons and notable changes n appearance. Although the algorthm prevals n challengng stuatons t s surprsngly easy to mplement. We beleve that, due to ts smplcty and robustness, t wll fnd numerous applcatons n tasks where the knowledge of correspondences between mages plays an mportant role. As an example of such usage we presented several use cases n the context of the cartoon anmaton producton ppelne. As future work we plan to extend our approach to handle occlusons and also to develop effcent mult-resoluton schemes to avod dependance on energy-based technques for applcatons where a pxel or sub-pxel precson s requred. Acknowledgements We are grateful to Ladslav Kavan for numerous frutful dscussons. Thanks must also go to anonymous revewers for ther nsghtful comments and to Tomáš Jarkovský, Vojtěch Votýpka, and Tomáš Rychecký from AnFlm studo for beng ntators of ths work. Cartoon mages used n ths paper are courtesy of Pavel Koutský / AnFlm, and studos Unversal Producton Partners & Dgtal Meda Producton. Ths work has been supported by the Mare Cure acton IEF, No. PIEF-GA

7 skeleton transfer Fgure 8: Moton capture by skeleton transfer the mage of the rest pose was manually annotated by a skeleton (left). Its correspondng postons on several new postures were obtaned wthout user nterventon usng our deformable mage regstraton algorthm (rght). References ALEXA, M., COHEN-OR, D., AND LEVIN, D As-rgd-as-possble shape nterpolaton. In ACM SIGGRAPH Conference Proceedngs, BELONGIE, S., MALIK, J., AND PUZICHA, J Shape matchng and object recognton usng shape contexts. IEEE Transactons on Pattern Analyss and Machne Intellgence 24, 24, BERGEN, J. R., ANANDAN, P., HANNA, K. J., AND HINGORANI, R Herarchcal model-based moton estmaton. In Proceedngs of European Conference on Computer Vson, BOTSCH, M., PAULY, M., WICKE, M., AND GROSS, M. H Adaptve space deformatons based on rgd cells. Computer Graphcs Forum 26, 3, BREGLER, C., LOEB, L., CHUANG, E., AND DESHPANDE, H Turnng to the masters: Moton capturng cartoons. ACM Transactons on Graphcs 21, 3, CHANG, C. W., AND LEE, S. Y Automatc cel pantng n computer-asssted cartoon producton usng smlarty recognton. The Journal of Vsualzaton and Computer Anmaton 8, 3, GHOLIPOUR, A., KEHTARNAVAZ, N., BRIGGS, R. W., DEVOUS, M., AND GOPINATH, K. S Bran functonal localzaton: A survey of mage regstraton technques. IEEE Transactons on Medcal Imagng 26, 4, GLOCKER, B., KOMODAKIS, N., TZIRITAS, G., NAVAB, N., AND PARAGIOS, N Dense mage regstraton through MRFs and effcent lnear programmng. Medcal Image Analyss 12, 6, IGARASHI, T., MOSCOVICH, T., AND HUGHES, J. F As-rgd-as-possble shape manpulaton. ACM Transactons on Graphcs 24, 3, DE JUAN, C. N., AND BODENHEIMER, B Re-usng tradtonal anmaton: methods for sem-automatc segmentaton and nbetweenng. In Proceedngs of the ACM SIGGRAPH/Eurographcs Symposum on Computer Anmaton, KLEIN, S., STARING, M., AND PLUIM, J. P. W Evaluaton of optmzaton methods for nonrgd medcal mage regstraton usng mutual nformaton and B- splnes. IEEE Transactons on Image Processng 16, 12, KORT, A Computer aded nbetweenng. In Proceedngs of Internatonal Symposum on Non-photorealstc Anmaton and Renderng, LI, W., AND SALARI, E Successve elmnaton algorthm for moton estmaton. IEEE Transactons on Image Processng 4, 1, LOWE, D. G Dstnctve mage features from scale-nvarant keyponts. Internatonal Journal of Computer Vson 60, 2, LUCAS, B. D., AND KANADE, T An teratve mage regstraton technque wth an applcaton to stereo vson. In Proceedngs of Internatonal Jont Conference on Artfcal Intellgence, MADEIRA, J. S., STORK, A., AND GROSS, M. H An approach to computersupported cartoonng. The Vsual Computer 12, 1, MAINTZ, J. B. A., AND VIERGEVER, M. A A survey of medcal mage regstraton. Medcal Image Analyss 2, 1, MODERSITZKI, J Numercal Methods for Image Regstraton. Oxford Unversty Press, UK. MÜLLER, M., HEIDELBERGER, B., TESCHNER, M., AND GROSS, M Meshless deformatons based on shape matchng. ACM Transactons on Graphcs 24, 3, PAPENBERG, N., SCHUMACHER, H., HELDMANN, S., WIRTZ, S., BOMMERSHEIM, S., ENS, K., MODERSITZKI, J., AND FISCHER, B A fast and flexble mage regstraton toolbox. In Bldverarbetung für de Medzn, QIU, J., SEAH, H. S., TIAN, F., CHEN, Q., AND WU, Z Enhanced auto colorng wth herarchcal regon matchng. Computer Anmaton and Vrtual Worlds 16, 3 4, QIU, J., SEAH, H. S., TIAN, F., CHEN, Q., WU, Z., AND MELIKHOV, K Auto colorng wth enhanced character regstraton. Internatonal Journal of Computer Games Technology, 1, 2. RIVERS, A. R., AND JAMES, D. L FastLSM: Fast lattce shape matchng for robust real-tme deformaton. ACM Transactons on Graphcs 26, 3, 82. SCHAEFER, S., MCPHAIL, T., AND WARREN, J Image deformaton usng movng least squares. ACM Transactons on Graphcs 25, 3, SEAH, H. S., AND FENG, T Computer-asssted colorng by matchng lne drawngs. The Vsual Computer 16, 5, SHEKHOVTSOV, A., KOVTUN, I., AND HLAVÁČ, V Effcent MRF deformaton model for non-rgd mage matchng. Computer Vson and Image Understandng 112, 1, SORKINE, O., AND ALEXA, M As-rgd-as-possble surface modelng. In Proceedngs of Eurographcs/ACM SIGGRAPH Symposum on Geometry Processng, SUMNER, R. W., SCHMID, J., AND PAULY, M Embedded deformaton for shape manpulaton. ACM Transactons on Graphcs 26, 3, 80. SÝKORA, D., BURIÁNEK, J., AND ŽÁRA, J Colorzaton of black-and-whte cartoons. Image and Vson Computng 23, 9, SÝKORA, D., DINGLIANA, J., AND COLLINS, S LazyBrush: Flexble pantng tool for hand-drawn cartoons. Computer Graphcs Forum 28, 2, SZELISKI, R. S., ZABIH, R., SCHARSTEIN, D., VEKSLER, O., KOLMOGOROV, V., AGARWALA, A., TAPPEN, M., AND ROTHER, C A comparatve study of energy mnmzaton methods for markov random felds wth smoothness-based prors. IEEE Transactons on Pattern Analyss and Machne Intellgence 30, 6, WANG, Y., XU, K., XIONG, Y., AND CHENG, Z.-Q D shape deformaton based on rgd square matchng. Computer Anmaton and Vrtual Worlds 19, 3 4, WENG, Y., XU, W., WU, Y., ZHOU, K., AND GUO, B D shape deformaton usng nonlnear least squares optmzaton. The Vsual Computer 22, 9, WIRTZ, S., FISCHER, G., MODERSITZKI, J., AND SCHMITT, O Superfast elastc regstraton of hstologc mages of a whole rat bran for 3d reconstructon. In Proceedngs of the SPIE, vol. 5370, XIE, M Feature matchng and affne transformaton for 2D cell anmaton. The Vsual Computer 11, 8, XU, X., WAN, L., LIU, X., WONG, T.-T., WANG, L., AND LEUNG, C.-S Anmatng anmal moton from stll. ACM Transactons on Graphcs 27, 5, 117.

As-Rigid-As-Possible Shape Manipulation

As-Rigid-As-Possible Shape Manipulation As-Rgd-As-Possble Shape Manpulaton akeo Igarash 1, 3 omer Moscovch John F. Hughes 1 he Unversty of okyo Brown Unversty 3 PRESO, JS Abstract We present an nteractve system that lets a user move and deform

More information

MANY of the problems that arise in early vision can be

MANY of the problems that arise in early vision can be IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. 26, NO. 2, FEBRUARY 2004 147 What Energy Functons Can Be Mnmzed va Graph Cuts? Vladmr Kolmogorov, Member, IEEE, and Ramn Zabh, Member,

More information

Who are you with and Where are you going?

Who are you with and Where are you going? Who are you wth and Where are you gong? Kota Yamaguch Alexander C. Berg Lus E. Ortz Tamara L. Berg Stony Brook Unversty Stony Brook Unversty, NY 11794, USA {kyamagu, aberg, leortz, tlberg}@cs.stonybrook.edu

More information

Boosting as a Regularized Path to a Maximum Margin Classifier

Boosting as a Regularized Path to a Maximum Margin Classifier Journal of Machne Learnng Research 5 (2004) 941 973 Submtted 5/03; Revsed 10/03; Publshed 8/04 Boostng as a Regularzed Path to a Maxmum Margn Classfer Saharon Rosset Data Analytcs Research Group IBM T.J.

More information

Turbulence Models and Their Application to Complex Flows R. H. Nichols University of Alabama at Birmingham

Turbulence Models and Their Application to Complex Flows R. H. Nichols University of Alabama at Birmingham Turbulence Models and Ther Applcaton to Complex Flows R. H. Nchols Unversty of Alabama at Brmngham Revson 4.01 CONTENTS Page 1.0 Introducton 1.1 An Introducton to Turbulent Flow 1-1 1. Transton to Turbulent

More information

Complete Fairness in Secure Two-Party Computation

Complete Fairness in Secure Two-Party Computation Complete Farness n Secure Two-Party Computaton S. Dov Gordon Carmt Hazay Jonathan Katz Yehuda Lndell Abstract In the settng of secure two-party computaton, two mutually dstrustng partes wsh to compute

More information

(Almost) No Label No Cry

(Almost) No Label No Cry (Almost) No Label No Cry Gorgo Patrn,, Rchard Nock,, Paul Rvera,, Tbero Caetano,3,4 Australan Natonal Unversty, NICTA, Unversty of New South Wales 3, Ambata 4 Sydney, NSW, Australa {namesurname}@anueduau

More information

Do Firms Maximize? Evidence from Professional Football

Do Firms Maximize? Evidence from Professional Football Do Frms Maxmze? Evdence from Professonal Football Davd Romer Unversty of Calforna, Berkeley and Natonal Bureau of Economc Research Ths paper examnes a sngle, narrow decson the choce on fourth down n the

More information

Ensembling Neural Networks: Many Could Be Better Than All

Ensembling Neural Networks: Many Could Be Better Than All Artfcal Intellgence, 22, vol.37, no.-2, pp.239-263. @Elsever Ensemblng eural etworks: Many Could Be Better Than All Zh-Hua Zhou*, Janxn Wu, We Tang atonal Laboratory for ovel Software Technology, anng

More information

From Computing with Numbers to Computing with Words From Manipulation of Measurements to Manipulation of Perceptions

From Computing with Numbers to Computing with Words From Manipulation of Measurements to Manipulation of Perceptions IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS I: FUNDAMENTAL THEORY AND APPLICATIONS, VOL. 45, NO. 1, JANUARY 1999 105 From Computng wth Numbers to Computng wth Words From Manpulaton of Measurements to Manpulaton

More information

TrueSkill Through Time: Revisiting the History of Chess

TrueSkill Through Time: Revisiting the History of Chess TrueSkll Through Tme: Revstng the Hstory of Chess Perre Dangauther INRIA Rhone Alpes Grenoble, France perre.dangauther@mag.fr Ralf Herbrch Mcrosoft Research Ltd. Cambrdge, UK rherb@mcrosoft.com Tom Mnka

More information

can basic entrepreneurship transform the economic lives of the poor?

can basic entrepreneurship transform the economic lives of the poor? can basc entrepreneurshp transform the economc lves of the poor? Orana Bandera, Robn Burgess, Narayan Das, Selm Gulesc, Imran Rasul, Munsh Sulaman Aprl 2013 Abstract The world s poorest people lack captal

More information

DISCUSSION PAPER. Is There a Rationale for Output-Based Rebating of Environmental Levies? Alain L. Bernard, Carolyn Fischer, and Alan Fox

DISCUSSION PAPER. Is There a Rationale for Output-Based Rebating of Environmental Levies? Alain L. Bernard, Carolyn Fischer, and Alan Fox DISCUSSION PAPER October 00; revsed October 006 RFF DP 0-3 REV Is There a Ratonale for Output-Based Rebatng of Envronmental Leves? Alan L. Bernard, Carolyn Fscher, and Alan Fox 66 P St. NW Washngton, DC

More information

The Relationship between Exchange Rates and Stock Prices: Studied in a Multivariate Model Desislava Dimitrova, The College of Wooster

The Relationship between Exchange Rates and Stock Prices: Studied in a Multivariate Model Desislava Dimitrova, The College of Wooster Issues n Poltcal Economy, Vol. 4, August 005 The Relatonshp between Exchange Rates and Stock Prces: Studed n a Multvarate Model Desslava Dmtrova, The College of Wooster In the perod November 00 to February

More information

Alpha if Deleted and Loss in Criterion Validity 1. Appeared in British Journal of Mathematical and Statistical Psychology, 2008, 61, 275-285

Alpha if Deleted and Loss in Criterion Validity 1. Appeared in British Journal of Mathematical and Statistical Psychology, 2008, 61, 275-285 Alpha f Deleted and Loss n Crteron Valdty Appeared n Brtsh Journal of Mathematcal and Statstcal Psychology, 2008, 6, 275-285 Alpha f Item Deleted: A Note on Crteron Valdty Loss n Scale Revson f Maxmsng

More information

EVERY GOOD REGULATOR OF A SYSTEM MUST BE A MODEL OF THAT SYSTEM 1

EVERY GOOD REGULATOR OF A SYSTEM MUST BE A MODEL OF THAT SYSTEM 1 Int. J. Systems Sc., 1970, vol. 1, No. 2, 89-97 EVERY GOOD REGULATOR OF A SYSTEM MUST BE A MODEL OF THAT SYSTEM 1 Roger C. Conant Department of Informaton Engneerng, Unversty of Illnos, Box 4348, Chcago,

More information

The Global Macroeconomic Costs of Raising Bank Capital Adequacy Requirements

The Global Macroeconomic Costs of Raising Bank Capital Adequacy Requirements W/1/44 The Global Macroeconomc Costs of Rasng Bank Captal Adequacy Requrements Scott Roger and Francs Vtek 01 Internatonal Monetary Fund W/1/44 IMF Workng aper IMF Offces n Europe Monetary and Captal Markets

More information

DISCUSSION PAPER. Should Urban Transit Subsidies Be Reduced? Ian W.H. Parry and Kenneth A. Small

DISCUSSION PAPER. Should Urban Transit Subsidies Be Reduced? Ian W.H. Parry and Kenneth A. Small DISCUSSION PAPER JULY 2007 RFF DP 07-38 Should Urban Transt Subsdes Be Reduced? Ian W.H. Parry and Kenneth A. Small 1616 P St. NW Washngton, DC 20036 202-328-5000 www.rff.org Should Urban Transt Subsdes

More information

4.3.3 Some Studies in Machine Learning Using the Game of Checkers

4.3.3 Some Studies in Machine Learning Using the Game of Checkers 4.3.3 Some Studes n Machne Learnng Usng the Game of Checkers 535 Some Studes n Machne Learnng Usng the Game of Checkers Arthur L. Samuel Abstract: Two machne-learnng procedures have been nvestgated n some

More information

Assessing health efficiency across countries with a two-step and bootstrap analysis *

Assessing health efficiency across countries with a two-step and bootstrap analysis * Assessng health effcency across countres wth a two-step and bootstrap analyss * Antóno Afonso # $ and Mguel St. Aubyn # February 2007 Abstract We estmate a sem-parametrc model of health producton process

More information

The Developing World Is Poorer Than We Thought, But No Less Successful in the Fight against Poverty

The Developing World Is Poorer Than We Thought, But No Less Successful in the Fight against Poverty Publc Dsclosure Authorzed Pol c y Re s e a rc h Wo r k n g Pa p e r 4703 WPS4703 Publc Dsclosure Authorzed Publc Dsclosure Authorzed The Developng World Is Poorer Than We Thought, But No Less Successful

More information

UPGRADE YOUR PHYSICS

UPGRADE YOUR PHYSICS Correctons March 7 UPGRADE YOUR PHYSICS NOTES FOR BRITISH SIXTH FORM STUDENTS WHO ARE PREPARING FOR THE INTERNATIONAL PHYSICS OLYMPIAD, OR WISH TO TAKE THEIR KNOWLEDGE OF PHYSICS BEYOND THE A-LEVEL SYLLABI.

More information

Finance and Economics Discussion Series Divisions of Research & Statistics and Monetary Affairs Federal Reserve Board, Washington, D.C.

Finance and Economics Discussion Series Divisions of Research & Statistics and Monetary Affairs Federal Reserve Board, Washington, D.C. Fnance and Economcs Dscusson Seres Dvsons of Research & Statstcs and Monetary Affars Federal Reserve Board, Washngton, D.C. Banks as Patent Fxed Income Investors Samuel G. Hanson, Andre Shlefer, Jeremy

More information

Ciphers with Arbitrary Finite Domains

Ciphers with Arbitrary Finite Domains Cphers wth Arbtrary Fnte Domans John Black 1 and Phllp Rogaway 2 1 Dept. of Computer Scence, Unversty of Nevada, Reno NV 89557, USA, jrb@cs.unr.edu, WWW home page: http://www.cs.unr.edu/~jrb 2 Dept. of

More information

What to Maximize if You Must

What to Maximize if You Must What to Maxmze f You Must Avad Hefetz Chrs Shannon Yoss Spegel Ths verson: July 2004 Abstract The assumpton that decson makers choose actons to maxmze ther preferences s a central tenet n economcs. Ths

More information

Should marginal abatement costs differ across sectors? The effect of low-carbon capital accumulation

Should marginal abatement costs differ across sectors? The effect of low-carbon capital accumulation Should margnal abatement costs dffer across sectors? The effect of low-carbon captal accumulaton Adren Vogt-Schlb 1,, Guy Meuner 2, Stéphane Hallegatte 3 1 CIRED, Nogent-sur-Marne, France. 2 INRA UR133

More information

Why Don t We See Poverty Convergence?

Why Don t We See Poverty Convergence? Why Don t We See Poverty Convergence? Martn Ravallon 1 Development Research Group, World Bank 1818 H Street NW, Washngton DC, 20433, USA Abstract: We see sgns of convergence n average lvng standards amongst

More information

MULTIPLE VALUED FUNCTIONS AND INTEGRAL CURRENTS

MULTIPLE VALUED FUNCTIONS AND INTEGRAL CURRENTS ULTIPLE VALUED FUNCTIONS AND INTEGRAL CURRENTS CAILLO DE LELLIS AND EANUELE SPADARO Abstract. We prove several results on Almgren s multple valued functons and ther lnks to ntegral currents. In partcular,

More information

Income per natural: Measuring development as if people mattered more than places

Income per natural: Measuring development as if people mattered more than places Income per natural: Measurng development as f people mattered more than places Mchael A. Clemens Center for Global Development Lant Prtchett Kennedy School of Government Harvard Unversty, and Center for

More information

WHICH SECTORS MAKE THE POOR COUNTRIES SO UNPRODUCTIVE?

WHICH SECTORS MAKE THE POOR COUNTRIES SO UNPRODUCTIVE? MŰHELYTANULMÁNYOK DISCUSSION PAPERS MT DP. 2005/19 WHICH SECTORS MAKE THE POOR COUNTRIES SO UNPRODUCTIVE? BERTHOLD HERRENDORF ÁKOS VALENTINYI Magyar Tudományos Akadéma Közgazdaságtudomány Intézet Budapest

More information