The Virtual Movable Human Upper Body for Palpatory Diagnostic Training

Transcription

1 Paper Number Sesson DHM5 The Vrtual Movable Human Upper Body for Palpatory Dagnostc Tranng Meng-Yun Chen, Robert L. Wllams II, Robert R. Conatser Jr. and John N. Howell Interdscplnary Insttute for Neuromusculoskeletal Research, Oho Unversty Copyrght 26 SAE Internatonal ABSTRACT Ths paper presents our vrtual model for the dynamc human upper body, ncludng the spne, shoulders, and arms skeletal structure. We enable realstc human moton wth anatomcally-accurate jont moton lmts and a 71 degrees-of-freedom branchng seral chan model. Our nterest s to provde realstc motons when the student doctor moves the vrtual patent for palpatory dagnoss tranng. To date, ths work nclude only skeletal moton wth skn stretched on the model, but n the future we wll augment our statc Vrtual Haptc Back model wth ths moton. In ths way, student doctors can feel changes n human tssue due to motons, a common dagnostc technque. KEYWORDS Vrtual Haptc Back, vrtual movable human upper body, palpatory dagnoss, haptcs, skeletal moton, bomechancs INTRODUCTION Most vrtual human research focuses on workspace and obstacle avodance analyss [1, 2, 3]. Human lmbs are consdered as the end-effectors. The majorty of vrtual human research studes the results of the end-effectors movement. The Vrtual Haptc Back (VHB focuses on ndvdual vertebra movement rather than the work space of the lmbs. However, the ndvdual bone movement s as a result of end-effectors movement. The ntermedate jont angles of the vrtual haptc back model were calculated n real tme and usng robotcs methods rather than the nterpolaton method of [4]. The Vrtual Haptc Back (VHB s under development at Oho Unversty as a teachng and learnng tool for medcal palpatory dagnoss (fndng dsease and somatc dysfuncton va touch. Two PHANToM 3. haptc nterfaces permt palpaton of a lfe-szed vrtual human back. A graphcs mage of the back s dsplayed and local moton of back components (skn or underlyng vertebrae by exerton of palpatory force by the user s reflected graphcally. Mechancal propertes of the back, e.g., sprng constants of the surface, are chosen n part based on measurements and n part based on feedback from physcans experenced n palpatory dagnoss. To date our VHB model s statc (other than the local moton descrbed above; that s, there s no gross moton of the smulated human under dagnoss. A common technque used by palpatng physcans s to mpart gross moton of the human upper body to create spnal movement n order to fnd more easly and accurately abnormal vertebrae and surroundng tssues. Ths paper summarzes our efforts to create a usermovable vrtual human upper body. Our model ncludes 71 degrees-of-freedom, wth the human spne, shoulders, and arms, havng realstc graphcs and anatomcally-accurate jont moton lmts. We use robotcs methods and equatons to produce a surprsngly-realstc vrtual human moton. We have also covered ths model wth dynamc skn, allowng us n the near future to use haptcs technology to enable medcal students to feel realstc dfferences n touch whle usng the gross moton technque. Ths paper consst of fve sectons: modelng the human spnal column and upper body, dynamcs, deformable skn, haptcs and software development. MODELING THE HUMAN SPINAL COLUMN AND UPPER BODY The human skeleton can be consdered as seres of rotatonal lnks connected together. The anatomy of a human spne s shown n Fgure 1. Ths research focuses on the thoracc vertebrae, lumbar vertebrae, shoulders and two arms. A 3D human shoulder model s shown n Fgure 2. To descrbe the translatonal and rotatonal relatonshps between adjacent lnks of the open knematc chans, Denavt and Hartenberg (DH notaton [5] has been used because of ts strength n handlng large numbers of degrees of freedom and because of ts ablty to systematcally enable knematc and dynamc analyses.

2 Fgure 3. Jont reference frames for D-H representaton (taken from [1] The poston vector of a pont of nterest on the endeffector (any pont or ponts of nterest, where the student doctor can hold and move, such as the elbow of a human artculated model can be wrtten n terms of jont coordnates as X=X(θ Fgure 1. Anatomy of the human spne where θ = [ θ1... θ ] T n n R s the vector of n generalzed jont coordnates defnng the moton of a lnk wth respect to ts prevous neghbor n the seral chan. The global poston vector X(θ can be obtaned from the multplcaton of the 4x4 homogeneous transformaton matrces T, defned by the D-H representaton method [1] as follows: cos( θ 1 = sn( θ * cos( α T sn( θ *sn( α sn( θ cos( θ * cos( α cos( θ *sn( α sn( α cos( α a 1 sn( α * d cos( α * d 1 Fgure 2. 3D human shoulder model Consder Fgure 3 where two consecutve jonts are shown. The four DH parameters depcted n Fgure 3 are: a dstance from Z to Z +1 measured along X α angle between Z and Z +1 measured about X d dstance from X -1 to X measured along Z θ angle between X -1 to X measured about Z The 4x4 transformaton matrx T used to represent th jont coordnate system wth respect to the global base coordnate system {} s: T( θ = T( θ T( θ... T( θ We use the augmented 4x1 vectors r and r to express the Cartesan coordnates of a pont fxed n the th local frame n terms of the global coordnate system, respectvely: X ( θ X r = r = 1 1 where X s the fxed pont of nterest on lnk, expressed wth respect to the th coordnate system. Usng these relatonshps, r can be wrtten as:

3 r = T( θ r Gven the set of jont angle θ, we can calculate the end-effector coordnates X. Ths s also called forward pose (poston and orentaton knematcs, whch s a straght-forward computaton: X = f( θ The Vrtual Haptcs Human Upper Body s focused on movng the end-effector n order to fnd all the jont angles. Ths procedure requres the soluton of the nverse pose knematcs problem: θ = f 1 ( X Our Vrtual Haptcs Human Upper Body model conssts of 71 degrees-of-freedom (DOF. The endeffector has sx DOF (x,y,z,rx,ry,rz. The system jont DOF s much greater than the end-effector Cartesan DOF. Our model s a hyper-redundant system. A redundant system has an nfnte number of solutons for jont angles n the nverse pose knematcs problem. In our case, we used Jacoban-based Inverse Knematcs to obtan the jont angles at every smulaton step. Ths s an effectve method to handle our hyperredundancy wth 71-6 = 65 redundant jont freedoms. The results yeld surprsngly human-lke motons. The followng bullets descrbe a Jacoban matrx: A Jacoban s a vector dervatve wth respect to another vector, so t s a mult-dmensonal form of the dervatve. If we have a vector-valued functon of a vector of varables f(θ, the Jacoban s a matrx of partal dervatves, wth one partal dervatve for each combnaton of components of the vectors (the rows represent the functons f and the columns ndcate the varables θ. The Jacoban matrx thus contans all of the nformaton necessary to relate a change n any component of f to a change n any component of θ. The Jacoban s usually wrtten as J(f,θ, and t s generally represented as the matrx [ f θ ]: s a Jacoban matrx, where n our case m = 6 Cartesan DOF and n = 71 jont DOF. The Jacoban-based Inverse Knematcs can be expressed as: d = J dx dx s the change of the end-effector dθ s the change of jont angles n the system θ However, J s not always obtanable. For a hyperredundant system, J s not a square matrx and cannot be nverted. If we have a non-square matrx arsng from a hyper-redundant system, we can use the underconstraned pseudonverse [6, 7, 8]: J + = ( J T J Ths s a method for fndng a matrx that effectvely nverts a non-square matrx. Our Vrtual Haptcs Human Upper Body program uses Jacoban Inverse Knematcs to acqure the changes of jont angles n every smulaton + step accordng to dθ = J dx. DYNAMICS The dynamc analyss of the Vrtual Haptcs Human Upper Body s to fnd the force and torque actng on each jont by the external load actng on the endeffector. We choose teratve Newton-Euler dynamc formulaton [1] to obtan the forces and torques. We solve the nverse dynamcs problem drectly wth the standard general method n [5] to solve for the jont forces and torques gven the desred human upper body moton. Ths work s stll n progress. DEFORMABLE SKIN The Vrtual Haptcs Human Upper Body wll be mplemented wth deformable skn for a realstc deformaton accordng to the bone movement underneath the skn. Ths s crtcal to our future haptcs feel, enablng the student to feel realstc tssue texture changes due to moton of the vrtual patent. J T J ( f, θ f1 f1 f1... θ1 θ2 θ n f f f = = θ1 θ2 θ fm fm θ1 θ n

4 Renderng wth rgd components works just fne for robots, mechancal characters, and vehcles, but t s clearly not approprate for organc characters wth contnuous skn. Fgure 4. Vrtual Haptcs Human Upper Body wth skn attached Robots and smple characters made up from a collecton of rgd components can be rendered through classcal herarchcal renderng approaches [9, 1]. Each component mesh s smply transformed nto world space by the approprate jont world matrx. Ths results n every vertex n the fnal rendered character beng transformed by exactly one matrx. Wth the smple sknnng approach [11], the character s skn s modeled as a sngle contnuous mesh. Every vertex n the mesh s attached to exactly one jont n the skeleton, and when the skeleton s posed, the vertces are transformed by ther jont s world space matrx. As wth the rgd component method, every vertex s transformed by exactly one matrx usng an dentcal equaton: v = v W. Ths mples that smple sknnng should run about the same speed as renderng a character as rgd parts, and n practce, the two technques often perform smlarly wth equal-szed meshes. The smple sknnng technque s adequate for lowdetal models, but s not suffcent for hgher-qualty characters. In practce, the smple sknnng algorthm can be made to work for characters wth perhaps 5 or even as many as 1 trangles, as long as care s taken n vertex placement and bone attachment. Smple sknnng may be suffcent for lower-detal characters, but for hgher qualty, t s too lmted and a better soluton s requred. Expressed mathematcally, we can say that for every vertex, we compute the world space poston v by transformng the local space poston v by the approprate jont world matrx W: v = v W W1 W2 W1 W2 where W s a 4x4 matrx and v s a 1x4 homogeneous poston vector: v = [ v v v 1] x Every vertex n each mesh s transformed from the jont local space (where t s defned nto the world space, where t can be used for further graphcal processng such as lghtng and renderng. y v x y z W d b v a An unbent knee wth skn attached to jonts 1 and 2 Every vertex s attached to exactly one jont, so as the knee bends, we get some dstorton Smooth skn [12, 13] extends the concepts used n smple skn. Wth smooth sknnng, each vertex n the mesh can be attached to more than one jont, each attachment affectng the vertex wth a dfferent strength or weght. The fnal transformed vertex poston s a weghted average of the ntal poston transformed by each of the attached jonts. For example, the vertces n a character s knee could be partally weghted to both the hp jont (controllng the upper thgh and knee jont (controllng the calf. Many vertces wll only need to be attach to one or two jonts and rarely s t necessary to attach a vertex to more than four. y Vertex v defned n jont s local space x Vertex v transformed to world space usng matrx W

5 W1 W2 M 1 [ ] = B [ ] W[ ] The sknnng equaton that must be computed for each vertex then smplfes to v = w v M[] Usng smooth skn, a vertex can be attached to more than one jont wth and receve a weghted average of the transformatons Let us say that a partcular vertex s attached to N dfferent jonts. Each attachment s assgned a weght w whch represents how much nfluence the jont wll have on t. To ensure that no undesred scalng wll occur, we enforce the constrant that all of the weghts for a vertex must add up to 1: Wth the smooth skn algorthm, the attached skn can react realstcally and automatcally accordng to the underlyng bone movement. HAPTICS The Vrtual Haptcs Human Upper Body mplementaton wll combne two haptcs devces. The Immerson Cyber Grasp Haptc Glove (Fgure 5 wll be used for graspng the vrtual human upper body and movng the vrtual human upper body (whle feelng the shape of the human body where grasped. w = 1 = w + w1 + + wn To compute the world space poston v of the vertex, we transform t by each jont that t s attached to, and compute a weghted sum of the results: v = w v B[ ] W[ ] where v s the untransformed vertex n skn local space, the space n whch the skn mesh was orgnally modeled. The matrx W [] s the world matrx of the jont for attachment I, resultng from forward pose knematcs computatons. We use the ndexng notaton [] to ndcate that we don t want the matrx of the th jont n the skeleton (whch would be wrtten W, but nstead we want the world matrx of attachment s jont. For example, f a partcular vertex s weghted 6% to jont #37, 3% to jont #6, and 1% to jont #14, then we have: N=3 W [ ] = W37 W, [ 1] = W6 W, and [ 2] = W14 w =.6 w 3 w 1 1 =. 2 =. The matrx B [] s called the bndng matrx for jont []. Ths matrx s a transformaton from jont local space to skn local space, and so the nverse of ths matrx, B [] -1, represents the opposte transformaton from skn local space to jont local space. The combned transformaton B [] -1 W [] therefore frst transforms v from skn local to jont local, then from jont local to world space. As the number of jonts s lkely to be small compared to the total number of vertces that need to be sknned, t s more effcent to compute B [] -1 W [] for each jont before loopng through all of the vertces. We wll call ths transform M [], defned by: Fgure 5. Cyber Grasp haptcs glove The second haptcs devce s the PHANToM 3. (Fgure 6 whch wll be used to feel the spnal processes and soft tssue. The feel wll change automatcally n a realstc manner (as determned by palpatory expert D.O.s as the vrtual patent moves. Fgure 6. PHANToM 3.

6 SOFTWARE DEVELOPMENT The program for the Vrtual Haptcs Human Upper Body s wrtten usng Vsual C++ and the OpenGL graphc lbrary for 3D graphcs. Later, the haptc effects wll be added by usng the Vrtual Hand SDK and the PHANToM Ghost SDK. The program wll run n a Mcrosoft Wndows envronment. Fgure 7 shows a screen capture of the program. Fgure 8 s another vew of the program. The user can navgate (rotate, tlt, zoom the graphcs n a 3D envronment. nterests along the spne, shoulder and two arms to move. Then our nverse pose knematcs automatcally calculates the moton of the vrtual upper body. Fgure 1 shows there are two drop-down lsts n the program settngs by whch the user can choose any left and/or rght ponts of nterest. Another unque feature of the Vrtual Haptc Human Upper Body program s user defnable paths for the end-effector (pont of nterest. Fgure 11 shows a user-created path (3D nputs by mouse clcks to draw control ponts for the path and the created path s shown n Fgure 12. Fgure 12 also shows the end-effector (pont of nterest followng the user-created path, wth realstc human moton accordng to the nverse pose knematcs soluton at each moton step. The Haptc (force feedback effects wll be programmed nto the Vrtual Haptc Human Upper Body by usng Vrtual Hand SDK and Phantom Ghost SDK n the near future. Fgure 7. A screen capture of the program Fgure 9. A screen capture of the program settngs Fgure 8. Another vew of the program Fgure 9 s the screen capture of the program settngs that ncludes user defnable jont lmts (set by default to standard anatomcal lmts, pont of nterest (one or two, also called end-effector selecton and enable or dsable path followng. The data of the default jont lmts s gathered from Kapandj [14, 15] and Whte and Panjab [16]. The user can select pont(s of Fgure 1. Choose pont(s of nterest.

7 be noted and the vrtual model wll be mproved n order to reduce errors. Now, there are some problems wth ths valdaton and verfcaton plan. Currently, t s not nearly as natural for the doctor to move the vrtual patent n the same way she moves the real-world patent. Thus, some component of the error wll be due to nablty to exactly replcate the real-world motons n the vrtual world. Also, the moton capture laboratory can only record moton of marked ponts on the surface of the human subject, whereas we would lke to verfy human moton down to the vertebral level, whch s not possble n the proposed lab work. Nonetheless, we beleve that ths proposed valdaton and verfcaton wll help mprove the realsm of our Vrtual Haptcs Human Upper Body and lead to deas for future work. Fgure 11. User-created path for the end-effector (elbow CONCLUSION The Vrtual Haptcs Human Upper Body s currently under development. The purpose s to generate automatc and realstc human moton for our Vrtual Haptc Back project, n whch student doctors are traned n ther sense of touch for palpatory dagnoses. Ths work s a major addton to our work snce t represents the frst tme our vrtual human patent can move n a global sense, whch changes the palpatory feel of vertebrae and soft tssues. We ncorporate forward and nverse pose knematcs solutons and user-selectable end-effector(s nto our program. Another feature s userdefnable paths. Ths feature can be used for valdatng our model wth bomechancal human data n the future. Soft skn, dynamcs and haptc feedback wll be mplemented n the near future. ACKNOWLEDGMENTS The authors gratefully acknowledge fundng for ths work from the Osteopathc Hertage Foundaton. Fgure 12. Elbow followng the user-created path VALIDATION AND VERIFICATION Currently we have no valdaton and verfcaton data. Ths secton explans our valdaton and verfcaton plans, snce realsm s crucal to our work and phlosophy. We collaborate wth the Oho Unversty Physcal Therapy Department, who mantans a human body moton capture laboratory. We plan to use ths lab wth an expert palpator movng human subjects through standard palpatory dagnoss motons (such as sdebendng and flexon-extenson. The moton capture data wll be compared wth the same expert palpator movng the vrtual patent n the vrtual world. Errors wll REFERENCES 1. M.P. Reed, K. Satchell, and A. Nchols, 25, "Applcaton of Dgtal Human Modelng to the Desgn of a Postal Delvery Vehcle", Dgtal Human Modelng for Desgn and Engneerng Symposum. 2. W. Maurel and D. Thalmann, "Human Shoulder Modelng ncludng Scapulo-Thoracc Constrant and Jont Snus Cones", C&G 2-Computer and Graphcs Vol. 24, No. 2, pp J.K. Hodgns, W.L. Wooten, D.C. Brogan, and J.F. O'Bren, "Anmatng Human Athletcs", Proceedngs of SIGGRAPH 95, August, 1995, pp

8 4. K. Abdel-Malek, J. Yang, Z. M, V.C. Patel, K. Nebel, 24, "Human Upper Body Moton Predcton", Appled Smulaton and Modelng (ASM, June J.J. Crag, 25, Introducton to Robotcs: Mechancs and Control, 3 rd Edton, Pearson Prentce Hall, Upper Saddle Rver, NJ. 6. R.L. Wllams II, 1994, "Local Performance Optmzaton for a Class of Redundant Eght- Degree-of-Freedom Manpulators", NASA Techncal Paper S.R. Buss and J.S. Km, 24, "Selectvely Damped Least Squares for Inverse Knematcs", typeset manuscrpt, Aprl. 8. D. Tolan, A. Goswam, and N.I. Badler, 2, "Real- Tme Inverse Knematcs Technques for Anthropomorphc Lmbs", Graphcal Models 62: M. Teschner, S. Kmmerle, B. Hedelberger, G. Zachmann, L. Raghupath, A. Fuhrmann, M.-P. Can, F. Faure, N. Magnenat-Thalmann, W. Strasser, and P. Volno, 25, "Collson Detecton for Deformable Objects", The Eurographcs Assocaton and Blackwell Publshng. 1. C. DeCoro and S. Rusnkewczy, 25, "Pose- Independent Smplfcaton of Artculated Meshes", ACM Symposum on Interactve 3D Graphcs and Games. 11. J. Lander, 1998, "Skn Them Bones: Game Programmng for the Web Generaton", Game Developer Magazne, May: D.L. James and C.D. Twgg, 25, "Sknnng Mesh Anmatons", ACM Transactons on Graphcs (SIGGRAPH 25: 24(3, August. 13. J. P. Lews, M. Cordner, and N. Fong, 2, "Pose Space Deformaton: A Unfed Approach to Shape Interpolaton and Skeleton-Drven Deformaton", Internatonal Conference on Computer Graphcs and Interactve Technques, A.I. Kapandj, 1974, The Physology of The Jonts volume 2, Churchll Lvngstone. 15. A.I. Kapandj, 1974, The Physology of The Jonts volume 3, Churchll Lvngstone. 16. A.A. Whte and M.M. Panjab, 199, Clncal Bomechancs of the Spne, Lppncott Wllams & Wlkns. CONTACT Robert L. Wllams II, Ph.D., s the correspondng author and professor of mechancal engneerng n the Oho Unversty Russ College of Engneerng & Technology; he s also a member of the Interdscplnary Insttute for Neuromusculoskeletal Research at Oho Unversty. wllar4@oho.edu