Kinematic Analysis of Motor Performance in Robot-Assisted Surgery: A Preliminary Study Ilana NISKY a,, Sangram PATIL a Michael H. HSIEH b,c and Allison M. OKAMURA a a Department of Mechanical Engineering, Stanford University, Stanford, CA b Department of Urology, Stanford University, Stanford, CA c Pediatric Urology Service, Lucile Salter Packard Children s Hospital, Stanford, CA Abstract. The inherent dynamics of the master manipulator of a teleoperated robotassisted surgery (RAS) system can affect the movements of a human operator, in comparison with free-space movements. To measure the effects of these dynamics on operators with differing levels of surgical expertise, a da Vinci Si system was instrumented with a custom surgeon grip fixture and magnetic pose trackers. We compared users performance of canonical motor control movements during teleoperation with the manipulator and freehand cursor control, and found significant differences in several aspects of motion, including target acquisition error, movement speed, and acceleration. In addition, there was preliminary evidence for differences between experts and novices. These findings could impact robot design, control, and training methods for RAS. Keywords. Robot-Assisted Surgery, Teleoperation, Human Motor Control Introduction Robot-assisted surgical (RAS) systems, in particular the da Vinci from Intuitive Surgical, Inc., are gaining popularity in a wide variety of medical procedures, due to improvements in visualization and instrument dexterity, as well as an intuitive mapping between the movements of the operator s hands and the instruments inside the patient. From a human motor control perspective, the introduction of robotics in minimally invasive surgery (MIS) is a significant improvement over manual laparoscopy. However, patient outcomes in comparison with manual MIS have fallen short of expectations, given the robot s many potential advantages. In addition, there are many procedures for which robotics seems promising, but have not been broadly adopted. We propose that a quantitative understanding of how surgeons control the movements of a teleoperated robot and adapt to it can be used to improve robot design and surgeon training, ultimately leading to better patient outcomes. Surgery is an acquired skill, and includes both cognitive (procedural) as well as motor (technical) aspects. We focus here on the motor aspects, where efficient training pro- Corresponding Author: Ilana Nisky, Mechanical Engineering Department, Stanford University, 424 Panama mall, Stanford, CA, 9433; E-mail:nisky@stanford.edu
tocols and assessment metrics can be designed based on a quantitative understanding how the robot contributes to or detracts from motor skill development. Most current curricula invoke only task completion time and number of errors as performance metrics [], even though it has been suggested that they are not sufficient for skill assessment [2]. Because RAS facilitates efficient collection of data about the trajectories of the surgeon s hands and instruments, there is an unexploited potential for utilizing human motor control and learning theories [3] for understanding and improving skill acquisition in RAS. By comparing human movements with and without the robot, we can identify the effects of using a teleoperator, including its dynamics, motion scaling, and tremor filtering. Long-term, combining this information with state-of-the-art computational motor control theories could facilitate the development of more efficient robot design, control, as well as training approaches and skill assessment methods for RAS. Prior approaches to the analysis of surgeon movements have included breaking down trajectories into gestures surgemes that were suggested by senior surgeons, and creating a high-level language of surgery [4]. Other approaches have used Hidden Markov Models applied to position and force information for modeling surgery and skill evaluation [5, 6]. Our approach is unique in that it aims to model surgeon motion using the framework of human motor control and sensorimotor learning. The current study is a first step toward understanding the effects of teleoperated RAS on surgeon movements. We studied two simple and well-understood movements reach and reversal. Reach is a movement between two points that is characterized by a straight path and bell-shaped velocity trajectory [7] that can be explained using various models [8]. Reversal is a movement from a start point to an end point and back, without pausing at the end point, and can be modeled as a concatenation of two reaches in opposite directions with overlap [9]. While these movements are very simple and of limited clinical relevance when studied in isolation, they allow us to utilize the theoretical framework of human motor control, provide insight into the fundamental impact of teleoperator dynamics, and represent building blocks for more complicated surgical motions to be studied in future work.. Methods Participants: Five volunteers participated in the experiment, approved by the Stanford Institutional Review Board, after giving informed consent. Three of the participants had no surgical experience, but had some experience in interacting with robotic manipulators. One of the participants was a surgical fellow, and one was an experienced surgeon with a high volume of RAS cases. Apparatus and experimental procedure: We instrumented a da Vinci Si system at Lucile Packard Children s Hospital with a custom surgeon grip fixture attached to a master manipulator through a force sensor (Nano-7, ATI Industrial Automation), and magnetic pose trackers (TrakStar, Ascension Technologies), as shown in Fig.. The participants were asked to make consecutive center-out planar reach and reversal movements towards one of 6 possible targets, although their movements were not restricted to a plane. Three of the participants first performed the experiment freehand, holding the fixture (shown in Fig. B) but detached from the master manipulator, and then via teleoperation, and two of the participants performed it in the reverse order. To
D MRT fs B C Feedback long reach cursor tool tip marker target start 3 E C m m 6mm B A short reversal target Figure. Experimental setup. (A) Master manipulator with magnetic transmitter (MRT) and externally mounted fixture (B), with force sensor (fs) and magnetic tracker (). (C) A monitor is placed on the surgical table, and, through the endoscopic camera, presents the experimental view (D) with targets, cursor and feedback (circles and annotations are not visible during experiment). (E) Surgeon with 3 additional magnetic trackers attached to the shoulder, elbow, and wrist. provide consistent visual feedback in all trials, the position of the master manipulator was shown as a cursor on a monitor that was placed on the surgical table and presented to the user via the endoscopic camera (i.e., the actual patient-side manipulator was not visible), as shown in Fig. C-D. However, in the teleoperated setting, the master manipulator did control the movement of the non-visible patient-side manipulator, and therefore the dynamics were identical to standard clinical teleoperation. All targets were centered on one of two circles with radii 3 mm (short, S) or 6 mm (long, L), and were located in one of eight directions on these circles: -35, -9, -45,, 45, 9, and 35 degrees (Fig. D). The desired movement type was communicated to the participant by the color of the target. The presentation sequence of different movement types, distances, and directions was pseudorandom and identical for all participants and for both sessions of each participant. The participants were instructed to complete a reach within sec, and a reversal within.5 sec. After completion of each movement, participants were provided with feedback about their movement time. A movement was considered complete when the cursor stayed within 5 mm from the target center for.5 sec for reach, and the cursor returned to within 5 mm from start point for reversal. Data analysis: We sampled the position and force information at 2 Hz, and filtered the data offline with a 4th order low-pass Butterworth filter with cutoff at 6 Hz. We calculated velocity and acceleration by successive backward differentiation and additional filtering after each differentiation. We discarded reaches that were longer than 2.5 sec, and reversals that were longer than 3 sec. For comparisons between different movement directions, we calculated the position, velocity, and acceleration projections on desired movement directions, and also the speed and speed derivatives, which are the magnitudes of velocity and acceleration tangential to the path, respectively. Examples of these trajectories are depicted in Fig. 2. For each movement, we used the method described in [] to identify movement onset. This also provided us with estimation of initial jerk of each movement. Then, for each movement we identified points of peak speed, peak acceleration, peak deceleration, and endpoint, as depicted in Fig. 2. Importantly, we defined reach endpoint based on the main reach motion in each trial, without subsequent corrections (Fig. 2B), unless the corrective movement was completely fused within the original reach (Fig. 2C). Reversal endpoint was identified as the maximal point of position trajectory. Next, we defined the following metrics for comparison between teleoperated and freehand movements:
position [mm] pos [mm] velocity vel [mm/sec] acceleration [mm/sec acc 2 ] speed der [mm/sec 2 ] speed [mm/sec] speed [mm/sec] speed der. [mm/sec 2 ] 5-5 2-2 -.2.2.4.6.8 -.2.2.4.6.8 peak deceleration peak acceleration - -.2.2.4.6.8 2 5 A B C end of movement peak speed -.2.2.4.6.8.5.5.5 corrective movement.5 -.2.2.4.6.8 -.2.2.4.6.8-5 -.2.2.4.6.8 -.2.2.4.6.8 -.2.2.4.6.8.5 -.2.2.4.6.8 time [sec] time [sec] time [sec] pos [mm] vel [mm/sec] acc [mm/sec 2 ] 5-5 2 fused corrective movement speed der [mm/sec 2 ] speed [mm/sec] 2-2 5 4 2 D pos [mm].5.5 vel [mm/sec].5.5 acc [mm/sec 2 ] - 5-5 2-2.5.5 speed der [mm/sec 2 ] speed [mm/sec].5.5.5.5 time [sec] 4 2 2 E.5.5.5.5.5.5.5.5.5.5 time [sec] Figure 2. Examples of movement trajectories. (A) Simple reach. (B) Reach followed by a distinct corrective movement. (C) Reach with fused corrective movement. (D) Reversal without stopping at the end point is characterized by a single large deceleration peak. (E) Reversal with stop at end point characterized by two deceleration peaks that are similar in size to the acceleration peak. Endpoint error the norm of the vector between target and endpoint. Movement time the temporal difference between end and onset of movement. EE*MT endpoint error multiplied by movement time. Incorporates the combined effect of accuracy and timing. Peak speed largest magnitude of velocity. Peak acceleration largest magnitude of positive acceleration. Peak deceleration largest magnitude of negative acceleration. Initial jerk acceleration derivative at movement onset. Ratio peak acceleration to peak deceleration peak acceleration divided by peak deceleration. In reaches, this metric quantifies the symmetry of the velocity trajectory, and value > indicates the existence of a fused corrective movement. In reversals, it distinguishes between a real reversal (.5) and two reaches ( ), e.g. Fig. 2D-E. Ratio deceleration time to acceleration time temporal difference between maximum speed and movement onset divided by temporal difference between end of movement and maximum speed. This is an additional symmetry metric. We performed a 4-way ANOVA with first-order interactions for each of the metrics above with the following main effects: teleoperation or freehand (T), movement type (M), movement distance (Ds), and movement direction (Dr). Statistical significance was defined as α <.5. 2. Results Teleoperated long reach paths in the x-y plane of one of the novices and of the experienced surgeon are depicted in Fig. 3. It is evident that there is directional structure to the endpoint errors and movement path variability of the novice. The movements are curved only in some of the directions, which may be related to the nonlinear dynamics of the
A B 5 5 y [mm] y [mm] 5 5 5 5 x [mm] 5 5 x [mm] Figure 3. Examples of teleoperated long reach paths of one of the novices (A) and the experienced surgeon (B). Thin color traces are single movement paths; thick solid traces with shaded regions around them are the mean and standard deviation of all movements to a specific target. Black squares are the identified endpoints of each movement, and filled black circles with black ellipses around them are estimated endpoint mean and 39% error ellipse. master manipulator. The movements of the expert do not exhibit such a clear pattern of errors and curvature, likely because he has already adapted to these dynamics. A detailed quantitative analysis of such structure is beyond the scope of the current manuscript. The results of the ANOVA are given in Table.. Since we are focusing on the effect of teleoperation, it is important to examine the main effect of factor T, and its interac- Table. Results of ANOVA with 4 main factors (T teleoperation, M movement, Ds distance, and Dr direction) and their first-order interactions. All F-test d.o.f are (, 39) except for Dr and its interactions, which are (7, 39). < stands for <., and shaded cells indicate statistically significant effects. Endpoint Error Mov. Time EE*MT Peak Speed Peak Acc. Peak Dec. Initial Jerk R. Peak Acc R. Acc. Times T M Ds Dr T*M T*Ds T*Dr M*Ds M*Dr Ds*Dr F 26.54 2 68.44.2.25.8 2.73.7.2 p < < <.8.65.62.6..3.35 F 272 286 63 5.4 5.44 5.83.5.45.37 p < < < <.3.5 <.7.8.2 F.32.3 23 4.8.99.53.73.36.72 p.57.58 < <.77.3.5.9.22.65 F 383 72 3664 22.2 9.69.2.6.22 3.52 p < < < <.89.2 <.43.28 < F 472 6 96 24.22.6 5.39.55.85 3.3 p < < < <.63 < <.45.54.2 F 45 84 83 3 8.9 4.56 9.79 2.7.95 p < < < <.3.3 <.9.46.4 F 264. 5 4.27.2 6.42 4.2.2.87 p <.74 < <.73. <.72.98.7 F 49 679 27.4.4 2.5 2.7 7.82 3.85.95 p < < < <...4.5 <.6 F 3.52 659 35 24 5.5.99 2.98 2.29.4 p.6 < < <.2.98.53.8.2.2
Teleoperated Freehand Endpoint Error Movement Time A 7 B.8 C absolute endpoint error [mm] peak speed [mm/sec] 6 5 4 3 2 Movement Peak Speed type 25 2 5 5 25 2 5 5 Movement type movement time [sec] peak acceleration [mm/sec 2 ].6.4.2 Peak Movement Acceleration type 2 Teleoperated Freehand D E F 5 5 peak deceleration [mm/sec 2 ] Peak Deceleration 2 5 5 Initial Jerk Ratio Peak Acc. to Peak Dec. Ratio Dec. Time to Acc. Time G 3 H.4 I.5 initial jerk [mm/sec 3 ] accelerations ratio.2.8.6.4.2 Movement type error*time [mm*s] time ratio EE * MT 4 3 2.5 Movement type Figure 4. Comparison between teleoperated and freehand movements, averaged across all participants. S and L stand for short and long movements, bars are estimations of mean and error-bars are their standard errors. tions with the other factors. Fig. 4 presents estimations of means of all metrics together with their standard errors. For clarity of presentation, and because the detailed analysis with respect to movement directions is beyond the scope of this paper, we included all movement directions together in the estimation of means in Fig. 4. On average, participants had smaller endpoint error (2%), moved slower (3%) and with smaller peak speed (5.5%), absolute acceleration (22.5%), deceleration (27%), and initial jerk (4%) when teleoperating compared to freehand. All these effects were statistically significant. The smaller endpoint error might imply better performance in the teleoperated scenario; however, it is most likely related to slower movements and the speed-accuracy tradeoff. Instead, the EE*MT metric suggests that teleoperated and freehand performances are similar in terms of overall performance. Interestingly, we noticed that the expert was better in long teleoperated reaches and short teleoperated reversals than freehand, but such difference in performance was absent in the other movement types (data not shown). This is consistent with the expert s fewer corrective movements during teleoperated reaches, and revealed by symmetry metrics. 3. Conclusions & Discussion We present the results of a preliminary study of the effects of teleoperation on surgeon sensorimotor performance in RAS. Using metrics developed in the study of human mo-
tor control, we show that using the master manipulator alters the kinematics of users movements in RAS. The dynamics of the manipulator (in particular, inertia) likely play an important role in this effect. However, modeling the inertial effects with respect to different movement directions using acquired force sensor data is left for future work. The anecdotal observation regarding the effects of teleoperation on different movements (reach and reversal) by an expert suggests that expert motor performance, as well as differences between experts and novices, may depend on the relevance of the particular movement to surgery. For example, it is likely that long reaches and short reversals are more relevant to surgery than the other movement types. We plan to study more experts in order to test this hypothesis. Such findings could have direct implications for assessment of the motor (technical) aspects of surgical skill, independent of the cognitive (procedural) aspects. In this study, we focused on kinematic analysis of user motion during simple planar movements. In future work, we plan to extend the analysis to dynamics and explore how interaction forces between the user and the manipulator change with different movement types, directions, extents, and user expertise. In addition, we will analyze how different users utilize the redundancy in their arm movement, and include movements with more clinical relevance. Acknowledgments This work was supported by Stanford University, and by the Marie Curie IOF and Weizmann Institute of Science National Postdoctoral Award Program for Advancing Women in Science (to IN). The authors thank Taru Roy for hand fixture design, and Anthony Jarc for valuable discussions. References [] Ugarte, D.A., et al., Robotic surgery and resident training, Surgical Endoscopy, 7(6):96-963, 23. [2] Narazaki, K., D. Oleynikov, and N. Stergiou, Robotic surgery training and performance, Surgical Endoscopy, 2():96-3, 26. [3] Shadmehr, R. and S.P. Wise, The Computational Neurobiology of Reaching and Pointing: A Foundation for Motor Learning, MIT Press, 25. [4] Lin, H., et al., Towards automatic skill evaluation: Detection and segmentation of robot-assisted surgical motions, Computer Aided Surgery, (5):22-23, 26. [5] Rosen, J., et al., Generalized approach for modeling minimally invasive surgery as a stochastic process using a discrete Markov model, IEEE Transactions on Biomedical Engineering, 53(3):399-43, 26. [6] Megali, G., et al., Modelling and Evaluation of Surgical Performance Using Hidden Markov Models, IEEE Transactions on Biomedical Engineering, 53():9-99, 26. [7] Morasso, P., Spatial control of arm movements, Experimental Brain Research, 42(2):223-227, 98. [8] Flash, T. and N. Hogan, The coordination of arm movements: An experimentally confirmed mathematical model, Journal of Neuroscience, 5(7):688-73, 985. [9] Scheidt, R.A. and C. Ghez, Separate Adaptive Mechanisms for Controlling Trajectory and Final Position in Reaching, Journal of Neurophysiology, 98(6):36-363, 27. [] Botzer, L. and A. Karniel, A simple and accurate onset detection method for a measured bell-shaped speed profile, Frontiers in Neuroscience, 3, 29. DOI:.3389/neuro.2.2.29.