Captur and Modling of Non-Linar Htrognous Soft Tissu Brnd Bickl1 1 Moritz Ba chr Migul A. Otaduy3 ETH Zurich Harvard Univrsity Wojcich Matusik4 3 URJC Madrid Hansptr Pfistr Markus Gross1 4 Adob Systms, Inc. Figur 1: From lft to right: Forc-and-dformation captur of a non-linar htrognous pillow; synthsizd dformation with fittd matrial paramtrs; and intractiv dformation synthsizd with our soft tissu modling tchniqu. CR Catgoris: I.3.5 [Computr Graphics]: Computational Gomtry and Objct Modling Physically basd modling; I.3.7 [Computr Graphics]: Thr-Dimnsional Graphics and Ralism Animation Kywords: physically basd animation and modling, modl acquisition, dformations, data-drivn graphics 1 Introduction Rcnt yars hav witnssd significant progrss and popularity of physically-basd dformation modls. Numrous rsarchrs hav combind Nwtonian mchanics, continuum mchanics, numrical computation and computr graphics, providing a powrful forward This papr introducs a data-drivn rprsntation and modling tchniqu for simulating non-linar htrognous soft tissu. It simplifis th construction of convincing dformabl modls by avoiding complx slction and tuning of physical matrial paramtrs, yt rtaining th richnss of non-linar htrognous bhavior. W acquir a st of xampl dformations of a ral objct, and rprsnt ach of thm as a spatially varying strss-strain rlationship in a finit-lmnt modl. W thn modl th matrial by non-linar intrpolation of ths strss-strain rlationships in strain-spac. Our mthod rlis on a simpl-to-build captur systm and an fficint run-tim simulation algorithm basd on incrmntal loading, making it suitabl for intractiv computr graphics applications. W prsnt th rsults of our approach for svral nonlinar matrials and biological soft tissu, with accurat agrmnt of our modl to th masurd data. invrs Abstract Figur : Acquiring and modling non-linar quasi-static soft tissu bhavior. From lft to right: An objct is probd with a forc snsor to acquir svral xampl dformations, th applid forc dirction, and th forc magnitud. For vry masurmnt w stimat its strss-strain rlationship and rprsnt it as a sampl in strain spac. During runtim, w intrpolat ths sampls in strain spac using radial basis functions (RBFs) to synthsiz dformations for novl forc inputs. toolkit for physically-basd dformations and stunning simulations, with application in fatur films, vido gams, and virtual surgry, among othrs. Howvr, achiving ralistic soft-tissu dformations rquirs carful choics for matrial modls and thir paramtrs. Many ralworld objcts consist of htrognous matrials, rquiring spatially varying matrial paramtrs such as Young s modulus and Poisson s ratio. Stting thm is a difficult and tim-consuming procss. Evn mor challnging is th problm of matrial nonlinaritis. Most matrials, for xampl rubbr or biological soft tissu, show non-linar constitutiv bhavior, i.., a non-linar rlationship btwn strss and strain. Dspit th wid varity of nonlinar constitutiv modls in th litratur, such as th popular hyprlastic No-Hookan and Moony-Rivlin modls [Ogdn 1997], this is still an activ rsarch ara in matrial scinc. Nonthlss, non-linar physics quations ar oftn simplifid approximations to ral matrial bhavior, and choosing th appropriat modl as wll as tuning its paramtrs ar xtrmly complx tasks. W prsnt a novl data-drivn rprsntation and modling tchniqu for simulating non-linar htrognous soft tissu that simplifis th construction of convincing dformabl modls (Fig. ). Our tchniqu mploys finit lmnt mthods and xploits a st of masurd xampl dformations of ral-world objcts, thrby avoiding complx slction of matrial paramtrs. W transfr vry masurd xampl dformation into a local lmnt-wis strain spac, and rprsnt this xampl dformation as a locally linar
sampl of th matrial s strss-strain rlation. W thn modl th full non-linar bhavior by intrpolating th matrial sampls in strain spac using radial basis functions (RBFs). Finally, a simpl lastostatic finit-lmnt simulation of th non-linarly intrpolatd matrial sampls basd on incrmntal loading allows for fficint computation of rich non-linar soft-tissu simulations. Othr arlir work in computr graphics and robotics also proposd masurmnt-basd modl fitting as a mans for obtaining dformabl objct rprsntations [Pai t al. 001; Lang t al. 00; Schonr t al. 004], but was limitd to linar matrial modls with global support. In contrast, our work is th first to rprsnt complx non-linar htrognous matrials through spatially varying non-linar intrpolation of local matrial proprtis. Our complt soft tissu captur and modling piplin is also distinct for its simplicity. W prsnt a simpl-to-build captur systm consisting of forc probs and markr-basd trinocular stro, as wll as an fficint and robust algorithm for fitting th local strain-spac matrial sampls. W dmonstrat th ffctivnss of our soft-tissu captur and modling mthod for svral non-linar matrials and biological soft tissu. Th combination of simplicity and fficincy, both in acquisition and computation, and th high-xprssivnss of th rsults mak our tchniqu applicabl for intractiv applications in computr graphics and othr filds. Rlatd Work Rsarchrs in many filds, ranging from mchanical nginring to biology, hav long studid th problm of modling complx lasticity proprtis. For a rcnt survy of dformation modls in computr graphics, plas rfr to [Naln t al. 006]. Bio-Mchanical Modls For soft tissu modling, a common approach is to dvis a constitutiv modl [Ogdn 1997] that capturs in a sufficintly accurat mannr th various bhavior rgims of th matrial, and thn tun th modl paramtrs until thy bst fit mpirical data. This approach is, howvr, tdious and uttrly complx, as it rlis on accurat modling of tissu gomtry (.g., th bons, fat, and muscls in facial tissu), rich xcitation of matrial rgims, and accurat masurmnt of forcs and dformations (vn in typically inaccssibl rgions). Dspit th complxity of th approach, it has sn larg application in computr graphics sinc th pionring work by Trzopoulos t al. [1987], as it can lad to stunning rsults with th appropriat amount of ffort. Som xampls of complx bio-mchanical modls in computr graphics includ th nck [L and Trzopoulos 006], th torso [Zordan t al. 004; Tran t al. 005; DiLornzo t al. 008], th fac [Koch t al. 1996; Magnnat-Thalmann t al. 00; Trzopoulus and Watrs 1993; Sifakis t al. 005], and th hand [Suda t al. 008]. Masurmnt-Basd Modl Fitting To circumvnt th complxity of paramtr tuning, svral authors hav proposd masurmntbasd modl fitting approachs. Th sminal work of Pai t al. [001] prsnts a captur and modling systm for a dformabl objct s shap, lasticity, and surfac roughnss. Thir dformabl modl was basd on a Grn s functions matrix rprsntation [Jams and Pai 1999], and was latr xtndd to incras fitting robustnss [Lang t al. 00], and to handl viscolasticity [Schonr t al. 004]. Our approach shars thir stratgy for masuring surfac displacmnts as th rsult of applid surfac forcs, but, unlik thirs, is not limitd to linar matrial bhavior and dos not rly on global rspons functions. Sifakis t al. [005] giv a diffrnt spin to masurmnt-basd modling approachs, as thy larn th rlationship btwn facial muscl activation and skin positions. Othrs, particularly in biomchanics, hav xplord masurmnt-basd fitting of th paramtrs of various constitutiv modls, such as Young modulus stimation basd on a non-linar last squars problm [Schnur and Zabaras 199], Young modulus and Poisson ratio stimation through linar last squars [Bckr and Tschnr 007], stimation of non-linar viscolastic matrials [Kaur t al. 00], or vn plasticity stimation [Kajbrg and Lindkvist 004]. Our work borrows from ths approachs for th stimation of ach individual sampl of th strss-strain rlationship. Howvr, this alon is not sufficint for capturing th rich non-linar bhavior of soft tissu. In contrast to prvious work, th ralism of our matrial modl is gratly nhancd with spatially varying non-linar intrpolation in strain spac. Data-drivn Mthods Purly data-drivn tchniqus hav gaind larg popularity in computr graphics, as thy may produc highly ralistic rsults for phnomna that ar othrwis xtrmly complx to modl. Th intrpolation of lightfild sampls [Buhlr t al. 001] allows simulating th illumination of complx scns, whil rcnt data-drivn rflction modls [Matusik t al. 003] rprsnt ach BRDF through a dns st of masurmnts. Datadrivn mthods hav also bn applid to svral othr aspcts of dformation modling in computr graphics, such as facial wrinkl formation from local skin dformations [Ma t al. 008; Bickl t al. 008], grasping of objcts [Kry and Pai 006], sklton-drivn cloth wrinkls [Kim and Vndrovsky 008], body-skin dformation [Park and Hodgins 006], or larning of sklton-drivn skin dynamics [Park and Hodgins 008]. Our mthod is a mixtur of modl fitting tchniqus (i.., stimating strss-strain paramtrs from local masurmnts) and data-drivn mthods (i.., using tabulatd strss-strain paramtrs and non-linar intrpolation during runtim). Shap Modling Anothr common approach in computr graphics to modl dformations is shap modling [Botsch and Sorkin 008]. Som of th xisting approachs rly on prdfind xampls [Sloan t al. 001; Alln t al. 00; Sumnr t al. 005], or vn xploit intrpolation [Brgron and Lachapll 1985; Lwis t al. 000; Blanz t al. 003], but ths tchniqus cannot modl dformations as a raction to contact in th way our tchniqu dos. Som rcnt approachs connct shap modling with physicallybasd ractiv modls, by rigging using tmplats of forcs [Capll t al. 005] or by skltal intrpolation of lastic forcs [Galoppo t al. 009], but ths approachs cannot modl gnral non-linar soft tissu. Dformation Captur Our work capturs dformation xampls by combining a stro-vision acquisition systm and forc snsors, similar to arlir approachs [Pai t al. 001]. Othr tchniqus in matrial scincs also dirctly masur th paramtrs of constitutiv modls, such as th tnsil tst [Hart 1967], or apparatus for in-vivo masurmnt through tissu aspiration [Nava t al. 003] or indntation [Ottnsmyr and Salisbury Jr. 004]. 3 Modling of Non-Linar Matrials In this sction, w dscrib our rprsntation of non-linar htrognous lastic matrials, and how this rprsntation is usd for modling soft tissu dformations. W first giv an ovrviw of th rprsntation, and thn dscrib how w paramtriz th matrials and how this paramtrization xtnds from th continuum stting to a finit lmnt discrtization. W also xplain how w support matrial non-linaritis through intrpolation of local linar modls, and finally w dscrib our algorithm for computing non-linar lastostatic dformations basd on incrmntal loading. 3.1 Ovrviw of our Approach In matrials scinc, (on-dimnsional) lasticity proprtis hav long bn dscribd through strss-strain curvs. Inspird by this popular rprsntation, w opt for modling thr-dimnsional lastic proprtis by sampling th strss-strain function at various oprating rgims and intrpolating ths sampls in strain-spac (S Fig. ). Mor spcifically, w charactriz ach sampl of th strss-strain
function using a (local) linar constitutiv modl. Thn, in ordr to captur matrial non-linarity, w dfin th paramtr valus of th constitutiv modl at an arbitrary oprating point through scattrddata intrpolation in strain-spac. Morovr, in ordr to captur matrial htrognity, w comput both th strss-strain sampls and th scattrd-data intrpolation in a spatially varying mannr. Fig. 3 shows xampl dformations with color-codd Young s modulus, which varis both as a function of th location and th local strain. It is worth noting that our modl can captur lasticity proprtis, but not plasticity or viscosity, among othrs. Our modl builds on FEM and linar lasticity thory, and w rfr th intrstd radr to books on th topic [Bath 1995; Hughs 000]. 3. Discrtization and Paramtrization W us linar co-rotational FEM to locally rprsnt a dformabl objct s lastic proprtis. In othr words, givn an objct s dformd configuration, w modl th strss-strain rlationship with linar FEM. W captur non-linarity by varying th paramtrs of th strss-strain rlationship as a function of th strain itslf. Givn a displacmnt fild u, th linar co-rotational ( FEM ) mploys Cauchy s linar strain tnsor ε(u) = 1 u + ( u) T. Invarianc of th strain undr rotations is obtaind by xtracting th ro- tational part of th dformation gradint through polar dcomposition, and thn warping th stiffnss matrix [Müllr and Gross 004]. Thanks to symmtry of th strain and strss tnsors, w can rprsnt both as 6-vctors. Givn th strain tnsor, w construct th 6-vctor as ε = (ε xx ε yy ε zz ε xy ε xz ε yz) T, and similarly for th strss. Th local linar matrial yilds thn a rlationship σ(u) = Eε(u) (1) btwn strain and strss. For ach lmnt (in our cas, a ttrahdron), assuming locally linar isotropic matrial, th 6 6 strssstrain rlationship matrix E can b rprsntd by Young s modulus E and Possion ratio ν E = with th two constant matrics and H = E (G + νh), () (1 + ν)(1 ν) G = diag (1, 1, 1, 0.5, 0.5, 0.5) (3) 1 1 1 0 0 0 1 1 1 0 0 0 1 1 1 0 0 0 0 0 0 1 0 0 0 0 0 0 1 0 0 0 0 0 0 1. (4) This paramtrization is intuitiv, whr th Poisson ratio ν is unitlss and dscribs matrial comprssibility, whil Young s modulus E dfins matrial lasticity. Howvr, w mploy an altrnativ paramtrization (λ, α) that allows us to dscrib th strssstrain rlationship as a linar function of th paramtrs [Bckr and Tschnr 007]: with λ = E = λg + αh, (5) E (1 + ν)(1 ν) and α = λν. (6) Th paramtr α is also known as Lamé s first paramtr in lasticity thory, whras λ is not dirctly rlatd to any lasticity constant. With th (λ, α) paramtrization, th stiffnss matrix and Figur 3: Two xampls of a dformd pillow with color-codd Young s modulus ( blu is low, rd is high), which varis both as a function of location and th local strain. Prob prssur was highr on th right. th lastic forcs bcom linar in th paramtrs. W xploit this proprty in our paramtr fitting algorithm in Sction 4.1. Th pr-lmnt stiffnss matrix can b writtn as K = λ V B T GB + α V B T HB, (7) whr V is th volum of th lmnt (i.., ttrahdron), and B is a matrix dpndnt on th initial position of th lmnt s nods. Th complt stiffnss matrix is obtaind by assmbling th warpd pr-lmnt stiffnss matrics R K R T, whr R is th lmnt s rotation. By grouping all matrial paramtrs {λ, α } in on vctor p, th stiffnss matrix is paramtrizd as K(p). 3.3 Strain-Spac Intrpolation As introducd arlir, w dscrib th non-linar matrial proprtis through scattrd-data intrpolation of known local linar paramtrs in an lmnt-wis mannr. W obtain ths known local paramtrs from a st of xampl dformations, largly simplifying an artist s job of tuning matrial paramtrs for complx non-linar constitutiv modls. Lt us assum a st of M known xampl masurmnts, ach with a corrsponding lmnt-wis strain vctor ε i IR 6 and a paramtr vctor p i = (λ i, α i) T. Rcall that w us a rotationallyinvariant strain by xtracting th rotation of th dformation gradint through polar dcomposition [Müllr and Gross 004]. Our non-linar strain-dpndnt matrial p(ε) is formd by intrpolating linar matrial sampls p i(ε i). At a givn dformd configuration, th non-linar matrial is rprsntd by th corrsponding linar matrial that achivs th sam forc-displacmnt rlationship. Not that w do not xploit linarization in th mor traditional way of capturing th local slop of a non-linar function. For ach lmnt, w dfin th strss-strain rlationship through scattrd-data intrpolation in th strain-spac IR 6 using radial basis functions (RBFs). Th lmnt-wis function dscribing th matrial, p(ε) : IR 6 IR, has th form p(ε) = M w i ϕ( ε ε i ), (8) i=1 whr ϕ is a scalar basis function, and w i IR and ε i ar th wight and fatur vctor for th i th masurmnt, rspctivly.
Figur 4: Two lft-most columns: Comparison of capturd and synthsizd dformations for a foam block. Two right-most columns: Exampls of intractiv dformations producd by sliding a cylindr on top of th modl. W mploy th biharmonic RBF krnl ϕ(r) = r. This globally supportd krnl allows for smoothr intrpolation of sparsly scattrd xampl poss than locally supportd krnls, and avoids difficult tuning of th support radius [Carr t al. 001]. As a prprocss, w comput th RBF wights wi. This rducs to solving T linar M M systms for a dformabl objct with T lmnts du to th fact that th strss-strain rlationship is an lmnt-wis dscription of th matrial. This also lads to scattrd-data intrpolation of th matrial paramtrs in a rathr low-dimnsional IR6 domain. In contrast, intrpolation of matrial proprtis is much mor complicatd in arlir approachs basd on linar modls with global support [Pai t al. 001] du to th xtrmly high dimnsionality of th paramtrization. 3.4 Elastostatic FEM Simulation W comput novl dformations using an lastostatic FEM formulation Ku = F, whr th forc F includs, among othrs, th load producd by a contact prob. To corrctly captur th matrial s non-linarity during th dformation, w apply th load of th prob gradually, and solv th lastostatic FE problm for ach load incrmnt. In othr words, at ach loading stp w masur th currnt strain ε, w comput th matrial paramtrs p(ε) by mans of th intrpolation dscribd abov, w formulat th lastostatic problm, and w solv it for th nw dformations. Th incrmntal loading procdur nsurs that th non-linarity of th matrial is corrctly capturd during th complt dformation procss, with th matrial paramtrs dpnding on th strain at all tims. For contact handling, w comput a distanc fild for th rigid prob objct that producs th dformations. W tst for collisions btwn points on th dformabl objct and th distanc fild and, upon collision, w comput th pntration dpth and dirction. W thn dfin a linar forc fild at ach colliding point and solv th FEM simulation through itrativ quasi-static simulation. At ach itration of th quasi-static FEM simulation, w first comput th matrial paramtrs for th currnt configuration basd on th intrpolation algorithm dscribd abov. Thn, givn th stiffnss matrix and th linar collision forc fild, w dfin a quasi-static problm and solv for th nw displacmnts. W comput svral itrations until an quilibrium is rachd. 4 Fitting th Matrial Paramtrs W now dscrib how w comput th actual matrial paramtrs for a givn objct. This consists of two parts: First, stimating paramtr valus for ach dformation xampl, and scond, slcting a suitabl basis from all th dformation xampls. 4.1 Paramtr Estimation Algorithm In ordr to stimat a sampl of th strss-strain rlationship, w apply a known input forc to th objct undr study. For ach capturd dformation w can distinguish thr diffrnt rgions on th objct s surfac: (i) th probing rgion, with masurd non-zro forcs and masurd displacmnts, (ii) th attachd rgion, with unknown forcs and zro displacmnts, and (iii) th fr rgion, with zro forcs and masurd displacmnts. W us x and F to dnot th vctors of known displacmnts and forcs, rspctivly, at th points corrsponding to msh nods in th modl. Givn masurd displacmnts and forcs, w comput spatially varying matrial paramtrs p as: ) ( n X p = arg min xi (p, F )) x i + γ Lp, (9) p i=1 whr xi (p, F ) dnots th position of a msh nod as a function of matrial paramtrs and th masurd forcs. Th spars Laplacian matrix L nforcs spatial smoothnss of paramtrs. W mploy P th umbrlla oprator [Zhang 004] (Lp)i = j wi,j (pi pj ), whr i and j rfr to ttrahdron labls, and wi,j = 1 iff two ttrahdra shar a vrtx. This rgularization is rquird to prvnt ovrfitting du to nois in th acquird data. This is also mathmatically rquird to obtain a wll-posd problm bcaus th numbr of paramtrs is always twic th numbr of ttrahdra, p = T, whras th numbr of masurd positions x = n may b smallr, which would rsult in an undrconstraind problm. W also considrd scattrd data intrpolation of matrial paramtrs in objct spac as an altrnativ for addrssing th undrconstraind problm, but it would b difficult to dcid whr to plac th sampls for highly htrognous objcts. W minimiz th non-linar rsidual Eq. (9) itrativly using th Lvnbrg-Marquardt algorithm [Lvnbrg 1944]. W driv th Jacobian matrix in th Appndix. Instad of dfining th rsidual in
(a) (b) Figur 6: (a) Contact prob with intgratd forc snsor. (b) From lft to right: USB Intrfac Kit, Forc Snsing Rsistor (rd circl), Phidgt Voltag Dividr, and connction cabl. (a) (b) Figur 5: Our trinocular stro vision systm consists of thr high-rsolution camras (indicatd in rd) and two to thr light sourcs (indicatd in grn). Th camras ar arrangd in a triangular stup, which hlps maximiz visibility during captur of a contact intraction. Th light sourcs nsur uniform illumination during th acquisition. trms of masurd positions, th rror functional could also b dscribd in trms of masurd forcs [Bckr and Tschnr 007], yilding a linar optimization problm. Howvr, our obsrvations hav shown that this approach is unstabl whn th forcdisplacmnt rlationship is not clos to linar matrial bhavior. 4. Strain-Spac Basis Slction A matrial captur sssion consists of capturing N xampl dformations, from which w obtain th training datast of N paramtr vctors for ach lmnt in th msh. Howvr, this datast may b rathr larg, and w ar intrstd in slcting a compact st of M basis paramtr vctors for ach lmnt. Not that M nd not b th sam for all lmnts. W slct th basis in th sam grdy mannr as proposd by [Carr t al. 001]. W start by stting a paramtr vctor at zro strain with th avrag paramtrs computd for vry small-strain dformations. W thn add th paramtr vctor with largst rror, until a givn rror tolranc is achivd. Aftr ach paramtr vctor is addd to th basis, w nd to comput th RBF wights that bst fit th paramtr vctors for all N xampl dformations in a last-squars mannr, as dscribd in Eq. (8). This figur shows th volution of th fitting r6 Validation rror (mm) ror for th foam block in 4 Fig. 4. This rror plot accumulats th rror for all capturd dformations, not only thos addd to 1 th basis. Th rror drops 0.6 0 5 10 15 0 5 30 quickly aftr adding th Basis siz scond paramtr vctor to th basis bcaus th first vctor may not rprsnt th avrag matrial bhavior wll. S Sction 6 for mor dtails on th validation of our mthod. thr Canon 40D camras that captur imags at a rsolution of 3888 59. Ths camras ar placd in a triangular configuration to minimiz occlusions causd by th contact probs during data acquisition. W built an xtrnal triggr dvic to synchroniz th thr camras, and us additional light sourcs to nsur uniform illumination during th acquisition procss. Th surfac displacmnt during static dformations is masurd using a st of markrs that w paint on th objct s visibl surfac. Our systm is capabl of masuring viwpoint-rgistrd markr positions to an accuracy of < 1 mm. W built contact probs with arbitrary shaps and circular disks of diffrnt diamtrs attachd to th tip of a long scrwdrivr (s Fig. 6). W stimat th position and orintation of th contact prob using two makrs on th whit shaft of th scrwdrivr. To masur th magnitud of th contact forcs w us a 0. inch Forc Snsing Rsistor (FSR) (Itm S-0-1000-FS) connctd to a Phidgt Voltag Dividr (Itm S-50-P111) and USB Intrfac Kit 8/8/8 (Itm C-00-P1018) by Trossn Robotics. Th forc snsor s rad opration is synchronizd with th xtrnal camra triggr signal. 6 Rsults Modl Evaluation. W hav valuatd th quality of our soft-tissu captur and modling tchniqu on svral ral-world objcts, including two foam blocks, a htrognous soft pillow, and a human fac. Th data is publicly availabl at th authors wb sits. Fig. 4 shows a foam block with homognous matrial. W acquird 48 dformation xampls, wll distributd ovr th foam to induc dformations in all 1, 805 ttrahdra of our modl. W thn constructd th non-linar matrial rprsntation, with bass of 8 sampls pr ttrahdron on avrag, using th procdur in Sction 4.1. Evn though th objct is homognous, it should b notd that th matrial paramtrs that wr stimatd for ach input xampl ar non-homognous du to non-linaritis in th strss-strain rlationship. Th avrag fitting rror for th capturd dformations is lss than 1 mm (s figur in Sction 4.). Fig. 4 shows synthsizd dformations producd with our tchniqu using a prob with a largr, diffrnt contact ara than th prob usd for data acquisition. W dvlopd a simpl data acquisition systm consisting of forc probs and a markr-basd trinocular stro systm. Dformations ar inducd by physical intraction with th objct. W dcidd to us a markr-basd systm du to its simplicity, robustnss, and indpndnc of th objct s surfac proprtis. To compar our modl to a uniform linar co-rotational modl w us th homognous foam shown in Fig. 7 and Fig. 8. W capturd 1 dformation xampls with th prob nar th cntr of th block and modld th objct with 3, 40 ttrahdra. W computd an avrag-fit linar co-rotational modl that bst approximats all th input dformations. As shown in Fig. 8, our modl (blu) accuratly capturs th hyprlastic bhavior of th foam, whil th avrag-fit linar co-rotational modl (grn) undrstimats th dformation at small forc valus and ovrstimats it at larg ons. In addition, th linar co-rotational modl suffrs from lmnt invrsion for larg forcs. Figur 5 shows our trinocular stro vision systm, consisting of Our modl is of cours not confind to th contact shaps that wr 5 Data Acquisition
Figur 7: Comparing ral (top) and modld (bottom) dformations with a diffrnt contact prob than th on usd in th data acquisition phas. masurd ours linar co-rotational Figur 8: Comparison of dformations using our mthod vs. an avrag-fit linar co-rotational modl. usd during data acquisition. Fig. 7 shows a sid-by-sid comparison of our modl (bottom) to ral dformations (top) using a diffrnt contact prob than th circular on w usd for data acquisition. W capturd th applid forc with th nw contact prob, and thn distribut it uniformly in th simulatd stting. Th figur shows high corrspondnc btwn th ral and simulatd scnarios. W rfr th radr to th accompanying vido for an animatd sidby-sid comparison. To valuat th snsitivity of our captur and modling approach to masurmnt nois w cratd xampl dformations of a virtual block with thr layrs of usr-dfind non-linar matrials. W thn valuatd th accuracy in matching ths dformations with our modl undr diffrnt lvls of nois in th input data. Spcifically, w applid Gaussian nois with a varianc of 10%, 0% and 30% to th input displacmnts and thn masurd th L rror for all dformations and rror lvls. On avrag, w obtain an rror of 0.3% of th maximum displacmnt for th cas without rror, and.1%, 3.1% and 4.4% for th cass with 10%, 0% and 30% input nois, rspctivly. Fig. 9 shows a pillow objct with htrognous bhavior vn in its rst stat. Th scrnshots compar th capturd dformations with th dformations of th 1, 691 ttrahdra modl synthsizd with our algorithm. Th figur also shows scrnshots of dformations at intractiv fram rats of about 10 Hz on a standard PC. Facial Dformation. W hav also applid our soft-tissu captur and modling tchniqu to th challnging task of facial dformations, as shown in Fig. 10. W hav modld th facial tissu with a singl layr of 8, 61 ttrahdra that ar attachd to a low-rsolution skull modl. To modl th sliding contacts btwn th tissu and th skull w us th sam contact handling as for th prob objct (s Sction 3.4). Givn th dformation of th ttrahdral msh, w comput th dformation of a high-rsolution triangl msh using a smooth mbdding basd on moving last squars intrpolation lik Kaufmann t al. [008]. Not that our fac modl dos not corrctly captur all typs of dformations bcaus w us a modl with closd lips, and all th dformation xampls in th training datast wr capturd with rlaxd muscls and closd jaw. Nvrthlss, th modl is abl to produc complling dformations vn without anatomically corrct modling of th musculoskltal structur of th fac. 7 Discussion W hav prsntd a novl data-drivn mthod for modling nonlinar htrognous soft tissu. Th major practical contribution of our work is th ability to modl rich non-linar dformations in a vry simpl mannr, without th complx task of carfully choosing matrial modls and paramtrs. Instad, our data-drivn mthod rlis on a simpl-to-build acquisition systm, a novl rprsntation of th matrial through spatially varying intrpolation of fittd linar modls, and a simpl dformation synthsis mthod. Our work suggsts a highly innovativ approach to non-linar matrial modling, but it also suffrs from limitations. Du to its formulation, our tchniqu is currntly limitd to capturing lastic proprtis. A fully dynamic simulation of soft tissu would rquir capturing othr proprtis such as viscosity and plasticity. On intrsting conclusion of our work is that it is oftn possibl to obtain complling surfac dformations with a volumtric mshing unawar of an objct s actual volumtric structur. This is of cours not valid for all situations. For xampl, our fac modl could b gratly nhancd with accurat lip contact and jaw motion modls. Thr ar svral aspcts of our modl that dsrv furthr xploration. On of thm is its ability for capturing anisotropic bhavior. Th undrlying linar co-rotational matrial modl that w us for rprsnting dformation sampls can only captur isotropic bhavior, but dformation sampls with th sam total strain but in diffrnt dirctions will lad to anisotropic bhavior. In othr words, w locally modl th matrial isotropic in strain spac, yt strain-spac intrpolation of matrial paramtrs provids global anisotropic bhavior. It is worth xploring to what xtnt our approach capturs anisotropy.
Figur 9: Two lft-most columns: Comparisons of capturd and synthsizd dformations for a htrognous non-linar pillow. Right column: Intractiv dformations of th modl producd by pushing (top) and pulling (bottom). Figur 10: Lft: Captur of facial dformations; Middl: Synthsizd dformations for th capturd xampls; Right: Frams of an animation with a cylindrical prob prssing on th chk.
Anothr aspct that dsrvs furthr analysis is th formulation of th quasi-static dformation problm. Givn a crtain strain, w mploy a local linar co-rotational modl to formulat a quasi-static dformation problm. Howvr, our modl is not strictly a local linarization, which mans that th stiffnss matrix of th quasi-static dformation problm dos not mploy corrct forc drivativs. At th sam tim, our linar modl is mor robust than a modl obtaind by local diffrntiation and avoids non-passiv rgims. Similar to othr approachs, our paramtr fitting algorithm is formulatd as a minimization problm and may nd up in a local minimum. In fact, w hav idntifid fitting rror as th major sourc of potntial inaccuracis in th dformation synthsis. Somtims, fitting rror also appars bcaus w limit Poisson s ratio to physically valid valus during th minimization. Robust paramtr idntification is still an opn rsarch problm in matrials scinc, and som rcnt approachs xplor altrnativ solutions including particl filtrs [Burion t al. 008]. Multi-rsolution fitting may b anothr way of incrasing robustnss. Currntly, w only masur th forc in dirction of th prob s shaft without masuring tangntial forcs and friction bhavior. Mor accurat forc snsors could captur such ffcts. Finally, on could build a fully automatd captur systm using robotics. Using a mor fficint paramtr stimation algorithm for matrial fitting, on could valuat th nd for furthr sampls of th strss-strain rlationship onlin, and dtrmin th optimal probing pattrns on th fly. Acknowldgmnts W would lik to thank th anonymous rviwrs for thir hlpful commnts, and thank Manul Lang, Ptr Kaufmann, Sbastian Martin, Giuspp Barbarino, Guillrmo Diz-Caas, Kalyan Sunkavalli, and th mmbrs of th CG Lab in Zurich. This rsarch was supportd by th NCCR Co-M grant of th Swiss National Scinc Foundation. Moritz Bächr was partially supportd by th Initiativ in Innovativ Computing (IIC) and th School of Enginring and Applid Scincs (SEAS) at Harvard. Rfrncs ALLEN, B., CURLESS, B., AND POPOVIĆ, Z. 00. Articulatd body dformation from rang scan data. ACM Trans. Graph. 1, 3, 61 619. BATHE, K.-J. 1995. Finit Elmnt Procdurs. Prntic-Hall. BECKER, M., AND TESCHNER, M. 007. Robust and fficint stimation of lasticity paramtrs using th linar finit lmnt mthod. In SimVis, 15 8. BERGERON, P., AND LACHAPELLE, P., 1985. Controlling facial xprssion and body movmnts in th computr gnratd short Tony d Pltri. Siggraph Cours Nots. BICKEL, B., LANG, M., BOTSCH, M., OTADUY, M. A., AND GROSS, M. 008. Pos-spac animation and transfr of facial dtails. In Proc. of th ACM SIGGRAPH / Eurographics Symposium on Computr Animation, 57 66. BLANZ, V., BASSO, C., POGGIO, T., AND VETTER, T. 003. Ranimating facs in imags and vido. Computr Graphics Forum, 3 (Spt.), 641 650. BOTSCH, M., AND SORKINE, O. 008. On linar variational surfac dformation mthods. IEEE Transactions on Visualization and Computr Graphics (TVCG) 14, 1, 13 30. BUEHLER, C., BOSSE, M., MCMILLAN, L., GORTLER, S., AND COHEN, M. 001. Unstructurd lumigraph rndring. In Proc. of ACM SIGGRAPH, ACM, 45 43. BURION, S., CONTI, F., PETROVSKAYA, A., BAUR, C., AND KHATIB, O. 008. Idntifying physical proprtis of dformabl objcts by using particl filtrs. In Proc. of th Intrnational Confrnc on Robotics and Automation, 111 1117. CAPELL, S., BURKHART, M., CURLESS, B., DUCHAMP, T., AND POPOVIĆ, Z. 005. Physically basd rigging for dformabl charactrs. In ACM SIGGRAPH/Eurographics Symposium on Computr Animation, 301 310. CARR, J. C., BEATSON, R. K., CHERRIE, J. B., MITCHELL, T. J., FRIGHT, W. R., MCCALLUM, B. C., AND EVANS, T. R. 001. Rconstruction and rprsntation of 3D objcts with radial basis functions. In Proc. of ACM SIGGRAPH, 67 76. DILORENZO, P., ZORDAN, V., AND SANDERS, B. 008. Laughing Out Loud: Control for modling anatomically inspird laughtr using audio. ACM Trans. Graph. (Proc. of ACM SIG- GRAPH Asia) 7, 5. GALOPPO, N., OTADUY, M. A., MOSS, W., SEWALL, J., CUR- TIS, S., AND LIN, M. C. 009. Controlling dformabl matrial with dynamic morph targts. In ACM SIGGRAPH Symposium on Intractiv 3D Graphics and Gams. HART, E. W. 1967. Thory of th tnsil tst. Acta Mtallurgica 15, 351 355. HUGHES, T. J. R. 000. Th Finit Elmnt Mthod. Linar Static and Dynamic Finit Elmnt Analysis. Dovr Publications. JAMES, D. L., AND PAI, D. K. 1999. ArtDfo: Accurat ral tim dformabl objcts. In Proc. of ACM SIGGRAPH, ACM Prss/Addison-Wsly Publishing Co., 65 7. KAJBERG, J., AND LINDKVIST, G. 004. Charactrisation of matrials subjctd to larg strains by invrs modlling basd on in-plan displacmnt filds. Intrnational Journal of Solids and Structurs 41, 13, 3439 3459. KAUER, M., VUSKOVIC, V., DUAL, J., SZEKELY, G., AND BA- JKA, M. 00. Invrs finit lmnt charactrization of soft tissus. Mdical Imag Analysis 6, 3, 57 87. KAUFMANN, P., MARTIN, S., BOTSCH, M., AND GROSS, M. 008. Flxibl simulation of dformabl modls using discontinuous galrkin fm. Procdings of th ACM SIGGRAPH / Eurographics Symposium on Computr Animation, 105 115. KIM, T.-Y., AND VENDROVSKY, E. 008. Drivnshap - a datadrivn approach to shap dformation. In Proc. of th ACM SIG- GRAPH / Eurographics Symposium on Computr Animation. KOCH, R. M., GROSS, M. H., CARLS, F. R., VON BÜREN, D. F., FANKHAUSER, G., AND PARISH, Y. 1996. Simulating facial surgry using finit lmnt mthods. In Proc. of ACM SIG- GRAPH, 41 48. KRY, P. G., AND PAI, D. K. 006. Intraction captur and synthsis. ACM Trans. Graph. (Proc. of ACM SIGGRAPH) 5, 3, 87 880. LANG, J., PAI, D. K., AND WOODHAM, R. J. 00. Acquisition of lastic modls for intractiv simulation. Intrnational Journal of Robotics Rsarch 1, 8, 713 733. LEE, S.-H., AND TERZOPOULOS, D. 006. Hads up!: Biomchanical modling and nuromuscular control of th nck. ACM Trans. Graph. (Proc. of ACM SIGGRAPH) 5, 3. LEVENBERG, K. 1944. A mthod for th solution of crtain nonlinar problms in last squars. Th Quartrly of Applid Mathmatics,, 164 168. LEWIS, J. P., CORDNER, M., AND FONG, N. 000. Pos spac dformation: A unifid approach to shap intrpolation and sklton-drivn dformation. In Proc. of ACM SIGGRAPH, 165 17.
MA, W.-C., JONES, A., CHIANG, J.-Y., HAWKINS, T., FRED- ERIKSEN, S., PEERS, P., VUKOVIC, M., OUHYOUNG, M., AND DEBEVEC, P. 008. Facial prformanc synthsis using dformation-drivn polynomial displacmnt maps. ACM Trans. Graph. (Proc. of ACM SIGGRAPH Asia) 7, 5. MAGNENAT-THALMANN, N., KALRA, P., LÉVÊQUE, J. L., BAZIN, R., BATISSE, D., AND QUELEUX, B. 00. A computational skin modl: fold and wrinkl formation. IEEE Trans. on Information Tchnology in Biomdicin 6, 4, 317 33. MATUSIK, W., PFISTER, H., BRAND, M., AND MCMILLAN, L. 003. A data-drivn rflctanc modl. ACM Transactions on Graphics (Proc. of ACM SIGGRAPH), 3, 759 770. MÜLLER, M., AND GROSS, M. 004. Intractiv virtual matrials. In GI 04: Procdings of Graphics Intrfac 004, Canadian Human-Computr Communications Socity, School of Computr Scinc, Univrsity of Watrloo, Watrloo, Ontario, Canada, 39 46. NAVA, A., MAZZA, E., KLEINERMANN, F., AVIS, N. J., AND MCCLURE, J. 003. Dtrmination of th mchanical proprtis of soft human tissus through aspiration xprimnts. In Proc. of MICCAI, 9. NEALEN, A., MLLER, M., KEISER, R., BOXERMAN, E., AND CARLSON, M. 006. Physically basd dformabl modls in computr graphics. Computr Graphics Forum 5, 4 (Dc.), 809 836. OGDEN, R. W. 1997. Non-Linar Elastic Dformations. Courir Dovr Publications. OTTENSMEYER, M. P., AND SALISBURY JR., J. K. 004. In-vivo data acquisition instrumnt for solid organ mchanical proprty masurmnt. In Proc. of MICCAI, 975 98. PAI, D. K., VAN DEN DOEL, K., JAMES, D. L., LANG, J., LLOYD, J. E., RICHMOND, J. L., AND YAU, S. H. 001. Scanning physical intraction bhavior of 3d objcts. In Procdings of ACM SIGGRAPH, 87 96. PARK, S. I., AND HODGINS, J. K. 006. Capturing and animating skin dformation in human motion. ACM Transactions on Graphics (Proc. of ACM SIGGRAPH) 5, 3. PARK, S. I., AND HODGINS, J. K. 008. Data-drivn modling of skin and muscl dformation. ACM Transactions on Graphics (Proc. of ACM SIGGRAPH) 7, 3. SCHNUR, D. S., AND ZABARAS, N. 199. An invrs mthod for dtrmining lastic matrial proprtis and a matrial intrfac. Intrnational Journal for Numrical Mthods in Enginring 33, 10, 039 057. SCHONER, J. L., LANG, J., AND SEIDEL, H.-P. 004. Masurmnt-basd intractiv simulation of viscolastic solids. Computr Graphics Forum (Proc. Eurographics) 3, 3, 547 556. SIFAKIS, E., NEVEROV, I., AND FEDKIW, R. 005. Automatic dtrmination of facial muscl activations from spars motion captur markr data. ACM Transactions on Graphics (Proc. of ACM SIGGRAPH) 4, 3, 417 45. SLOAN, P.-P. J., ROSE, III, C. F., AND COHEN, M. F. 001. Shap by xampl. In I3D 01: Procdings of th 001 symposium on Intractiv 3D graphics, ACM, Nw York, NY, USA, 135 143. SUEDA, S., KAUFMAN, A., AND PAI, D. K. 008. Musculotndon simulation for hand animation. ACM Trans. Graph. (Proc. SIGGRAPH) 7, 3. SUMNER, R. W., ZWICKER, M., GOTSMAN, C., AND POPOVIĆ, J. 005. Msh-basd invrs kinmatics. In ACM Trans. on Graphics (Proc. of ACM SIGGRAPH), vol. 4, 488 495. TERAN, J., SIFAKIS, E., BLEMKER, S., NG THOW HING, V., LAU, C., AND FEDKIW, R. 005. Crating and simulating skltal muscl from th visibl human data st. IEEE TVCG 11, 317 38. TERZOPOULOS, D., PLATT, J., BARR, A., AND FLEISCHER, K. 1987. Elastically dformabl modls. In Proc. of ACM SIG- GRAPH 87, 05 14. TERZOPOULUS, D., AND WATERS, K. 1993. Analysis and synthsis of facial imag squncs using physical and anatomical modls. IEEE Trans. PAMI 14 (Jun), 569 579. TOLEDO, S., CHEN, D., AND ROTKIN, V. 003. Taucs: A library for spars linar solvrs. ZHANG, H. 004. Discrt combinatorial laplacian oprators for digital gomtry procssing. In Proc. of SIAM Confrnc on Gomtric Dsign and Computing, Nashboro Prss, 575 59. ZORDAN, V., CELLY, B., CHIU, B., AND DILORENZO, P. C. 004. Brath asy: Modl and control of human rspiration for computr animation. In Proc. of th ACM SIGGRAPH / Eurographics Symposium on Computr Animation, 9 38. Appndix: Jacobian for Paramtr Fitting During fitting of matrial paramtrs p = {λ, α } through minimization of Eq. (9), w nd to comput th Jacobian of th dformd vrtx positions w.r.t. th paramtrs, i.., J = x, in ach p itration of th Lvnbrg-Marquardt algorithm. Givn xtrnal forcs F and initial positions x 0, th dformd positions undr th linar co-rotational lastostatic problm [Müllr and Gross 004] ar x = K 1 ( F + K ) x 0, (10) with K = [ ] R K R T and K = [R K ]. Hr [...] dnots th assmbly of th submatrix of th -th lmnt into th complt stiffnss matrix. Th Jacobian w.r.t. ach paramtr p i {λ, α } can thn b computd as J i = K 1 p i ( F + K x 0 ) + K 1 K p i x 0, (11) with K 1 p i = K 1 K p i K 1. Not that w do not comput th invrs of K. Instad, w comput a spars Cholsky factorization [Toldo t al. 003], and thn us this factorization many tims for solving th linar systms abov. Rcall th xprssion for th (unwarpd) pr-lmnt stiffnss matrix in Eq. (7). Th rmaining trms ar dfind as: K = λ K = α [ ] V R B T GB R T [ V R B T HB R T ],, K ] = [V R B T GB, λ K ] = [V R B T HB. α (1) In th cas whn som nods ar constraind not to dform (.g., whn th bottom of th capturd objcts is fixd), thir known positions mov to th right-hand sid in Eq. (10), and th Jacobians must b slightly modifid.