Au ton o mous Sur gi cal Ro bot ics Us ing 3-D Ul tra sound Guid ance: Fea si bil ity Study

Transcription

1 ULTRASONIC IMAGING 29, (2007) Au ton o mous Sur gi cal Ro bot ics Us ing 3-D Ul tra sound Guid ance: Fea si bil ity Study JOHN WHITMAN, MATTHEW P. FRONHEISER, NIKOLAS M. IVANCEVICH AND STEPHEN W. SMITH Department of Biomedical Engineering Duke University Dur ham, NC ssmith@duke.edu The goal of this study was to test the fea si bil ity of us ing a real-time 3D (RT3D) ul tra sound scan ner with a transthoracic ma trix ar ray trans ducer probe to guide an au ton o mous sur gi cal ro bot. Em ploy ing a fiducial align ment mark on the trans ducer to ori ent the ro bot s frame of ref er ence and us ing sim ple thresholding al go rithms to seg ment the 3D im ages, we tested the ac cu racy of us ing the scan ner to au to - mat i cally di rect a ro bot arm that touched two nee dle tips to gether within a wa ter tank. RMS mea sure - ment er ror was 3.8% or 1.58 mm for an av er age path length of 41 mm. Us ing these same tech niques, the au ton o mous ro bot also per formed sim u lated nee dle bi op sies of a cyst-like le sion in a tis sue phan tom. This fea si bil ity study shows the po ten tial for 3D ul tra sound guid ance of an au ton o mous sur gi cal ro bot for sim ple interventional tasks, in clud ing le sion bi opsy and for eign body re moval. Key words: Autonomous; biopsy; robotics; segmentation; 3D ultrasound. IN TRO DUC TION We have pre vi ously in ves ti gated the fea si bil ity of us ing real time 3D lap aro scopic ul tra - sound to guide a sur gi cal ro bot. 1 In that study, a 1-cm di am e ter probe for RT3D 2 was used laparoscopically for in vivo im ag ing of a ca nine. The probe was op er ated at 5 MHz and used to im age the spleen, liver and gall blad der as well as to guide hand-held sur gi cal in stru ments. Fur ther more, the three-di men sional (3-D) mea sure ment sys tem of the vol u met ric scan ner, used with this probe was tested as an au to matic guid ance mech a nism for a ro botic lin ear mo - tion sys tem in or der to sim u late RT3D/ro botic sur gery in te gra tion. Us ing the scan ner mea - sure ment soft ware with the B-mode and C-mode im ages dis played from the real time 3D scan, the re mote op er a tor mea sured spa tial co or di nates that were used to di rect a roboticallycon trolled nee dle to ward de sired in vi tro tar gets as well as tar gets in a post mor tem ca nine. The root mean squared (RMS) er ror for these mea sure ments was 1.34 mm us ing a fiducial align ment mark on the trans ducer hous ing and 0.76 mm us ing only the mea sure ment soft - ware of the 3D scan ner to com pute a path be tween two tar gets in the vol u met ric scan. In sim - i lar fash ion, 2D ul tra sound scan ners have been used for im age guided sur gi cal ro bot ics. 3, 4 More re cently, we dem on strated anal o gous re sults 5 us ing 3D cath e ter trans duc ers with the 3D mea sure ment soft ware of the real- time scan ner to di rect a ro bot arm that touched two nee dle tips to gether within a wa ter tank and in side a vas cu lar graft. RMS mea sure ment er ror ranged from mm for two cath e ter de signs /07/08 $18.00 Copy right 2007/08 by Dynamedia, Inc. All rights of re pro duc tion in any form re served.

2 214 WHITMAN ET AL The goal of this pa per is to ex tend our pre vi ous re sults of 3D ul tra sound guid ance to guide an au ton o mous ro bot in sim u lated interventional pro ce dures by us ing real-time 3D ul tra - sound im ag ing and a com mer cial transthoracic ma trix ar ray com bined with sim ple im age seg men ta tion anal y sis. METH ODS Real-time, 3-D ul tra sound was de vel oped at Duke Uni ver sity by Smith et al 6 and von Ramm et al 7 for cardiac imaging and commercialized by Volumetrics Medical Imaging. The Duke/VMI scan ner (Volu met rics Med i cal Im ag ing, Dur ham, NC) pro duces a py - ram i dal scan at rates up to 30 vol umes per sec ond. The scan ner si mul ta neously dis plays two B-scans and up to three par al lel C-scans at any angle and depth as well as real time 3D vol - ume ren der ing, 3-D color and 3-D pulsed Dopp ler. Each 3D ul tra sound scan is com prised of 64 x 64 = 4096 B-mode lines, in clud ing 512 sam ples per im age line yield ing 2 MB of data per 3D scan. Echo am pli tude is eight bits rang ing from 0 to 255. For this study, we used the com mer cial VMI transthoracic ma trix ar ray, which in cludes 440 trans mit chan nels and 256 re ceive chan nels op er at ing at 2.5 MHz as pre vi ously de scribed. 8 The ro bot used for these fea si bil ity ex per i ments was a Gan try III Car te sian Ro bot Lin ear Mo tion Sys tem man u fac tured by Techno-Isel (Techno, Inc., New Hyde Park,NY) us ing a Model H26T55-MAC200SD au to mated con trol ler that ac cepts in put com mands and 3-D co or di nates from a con nected com puter. The XY stage (model HL32SBM ) is a step per mo tor de sign pro vid ing 340 mm 290 mm of travel on a 600 mm 500 mm stage. The Z-axis slide (model HL31SBM ) pro vides 125 mm of ver ti cal clear ance and al lows 80 mm of travel in the z-di men sion. Fig ure 1 shows a sim ple sche matic of the 3-axis ro bot hold ing a probe nee dle and the 3D scan ner im ag ing a ver ti cal tar get nee dle in a wa ter tank. Etched into the side of the 3-D ma - trix ar ray trans ducer was a fiducial crosshair whose co or di nates were in put to the ro bot con - trol ler. Thus the ro bot s frame of ref er ence was ze roed on the trans ducer s fiducial spot. In the cur rent study, at a user-con trolled trig ger, the scan ner halted and the lat est vol ume of 3D ul tra sound data was au to mat i cally trans ferred over a Cat e gory 5 Ethernet con nec tion via Uni ver sal Data Pro to col to a PC con trol ler which was run ning our Matlab (Mathworks, Natick, MA) im age anal y sis soft ware. No fur ther user in put was re quired. Data trans mis - sion and load ing of the 2 MB file into the Matlab pro gram re quired ap prox i mately 15 sec - onds. Note that the com plete 3D scan data was trans ferred with no user se lec tion of the tar get or choice of im age slices. Also note that the im age data was not scan con verted but was transmitted in r, co or di nates in clud ing depth, r, az i muth an gle,, and el e va tion an gle,, as pre vi ously de scribed. 6 Fig ure 2a il lus trates an ex am ple vol ume ren dered view of a 3D scan of a tar get nee dle (1 mm in di am e ter and 30 mm long) in a wa ter tank whose echo data was trans mit ted to the ro - bot con trol ler for the task of nee dle tip touch ing. The 3D ren dered im age shows the tip of the tar get nee dle (solid ar row) with a tail of re ver ber a tion ech oes within the nee dle. Also shown in the 3D im age is an echo clus ter ar ti fact re sult ing from other re ver ber a tions within the wa - ter tank (dot ted ar row). To seg ment the im age and iso late the tar get nee dle tip, our Matlab scrip au to mat i cally re jected all ech oes be low a pre set thresh old and cal cu lated the 3D lo ca - tion of all re main ing ech oes. Figure 2b shows three or thogo nal views of the seg mented im - age of the nee dle tip in figure 2a when the thresh old set ting is above a level of 200. Fig ure 2c shows the lo ca tion of the nee dle tip (black dot) su per im posed on the py ram i dal scan icon by the Matlab pro gram. These im ages were au to mat i cally dis played to the pas sive ob server

3 AUTONOMOUS SUR GI CAL RO BOT ICS US ING 3-D UL TRA SOUND GUID ANCE 215 FIG. 1 Sche matic of the au ton o mous ro bot ex per i ment. The hor i zon tal probe nee dle was at tached to the ro botic arm while the ver ti cal tar get nee dle was fixed within a piece of rub ber. FIG. 2 Vol ume ren dered 3D im age of nee dle (a), si mul ta neous cross sec tions af ter im age thresholding (b)and needle tip location in pyramidal icon (c). dur ing the im age seg men ta tion pro cess. Fig ure 2c also shows a cir cu lar re gion of ex clu sion at the apex of the pyr a mid, which the probe nee dle could not en ter, to pre vent col li sions be - tween the probe nee dle and the trans ducer. The com plete im age seg men ta tion pro cess re - quired ap prox i mately one sec ond. The lo ca tion of the tar get nee dle tip in the co or di nate sys tem of the scan ner was then con - verted into the co or di nate sys tem of the ro bot as pre vi ously de scribed, 1 and the dis tance be - tween the tar get tip and probe tip was cal cu lated by sim ply sub tract ing the co or di nates of the probe nee dle tip rest ing at the fiducial mark from those of the tar get nee dle tip. The ro bot then moved its arm hold ing the probe nee dle the cor re spond ing dis tances in each rect an gu lar co or di nate. Since the ro bot was ca pa ble of only 3-axis mo tion it was im por tant to align all me chan i cal com po nents with ver ti cal and hor i zon tal axes. The ro bot move ments re quired ap prox i mately five sec onds. Af ter the ro bot arm had com pleted move ment, the er ror in each co or di nate was mea sured by re cord ing the dis tance needed to man u ally jog the ro botic arm un til the probe nee dle tip

4 216 WHITMAN ET AL (a) (b) (c) FIG. 3 Vol ume ren dered 3D im age of cyst in tis sue mim ick ing phan tom (a), si mul ta neous cross sec tion (b), dis - tri bu tion of 3D im age voxels af ter me dian fil ter ing and thresholding (c). touched the tip of the tar get nee dle based on vi sual in spec tion. RMS er ror was then cal cu - lated. Nee dle tips were orig i nally sep a rated by sev eral cm, and the pro ce dure was re peated 5 times. A sec ond ex per i men tal study of 3D ul tra sound guid ance of an au ton o mous ro bot was per - formed to sim u late the task of a cyst bi opsy. We sus pended a roughly spher i cal anechoic le - sion ap prox i mately 2 cm in di am e ter (a wa ter filled bal loon) in a tis sue-mim ick ing phan tom de vel oped by Madsen et al 9 con sist ing of a slurry of agar spheres con tain ing graph ite scat ter - ers. As in the nee dle tip ex per i ment above, the Volu met rics sys tem ac quired a 3D scan of the ob ject and trans mit ted the com plete 3D vol ume of echo data to the ro bot con trol ler with no slice se lec tion or tar get se lec tion by a hu man ob server. Fig ure 3a shows a real-time vol ume

5 AUTONOMOUS SUR GI CAL RO BOT ICS US ING 3-D UL TRA SOUND GUID ANCE 217 FIG. 4 Pho to graph of nee dle touch ex per i ment (a), si mul ta neous B-scan (b) and C-scan (c) of nee dle touch ex - periment. ren dered im age of the le sion im mersed in the tis sue-mim ick ing slurry. Note the over ly ing tis sue which ob scures the le sion. Fig ure 3b shows a sin gle B-mode slice of the le sion for il - lus tra tion pur poses only. Note the pres ence of bright inhomogeneities in the tis sue mim ick - ing material. For this task, we mod i fied the im age seg men ta tion soft ware. The Matlab scrip first ap - plied a 3 x 3 x 3 me dian fil ter to the pix els of the 3D scan to smooth the im age speckle. Next, all voxels were re jected whose am pli tude fell above a selectable thresh old value of one quar - ter of the mean echo am pli tude of the vol ume. The re sult ing map of the re main ing voxels is shown in fig ure 3c wherein the cyst voxels (black) stand out as the larg est group. Other re - main ing voxels cor re spond to min ima in the speckle back ground. Fi nally, the soft ware lo - cated the me dian voxel in three space of the larg est group of con tig u ous voxels that ap prox i mated the cen ter of the cyst. These soft ware op er a tions re quired ap prox i mately five sec onds. As de scribed above, these co or di nates were then passed to the ro bot that di rected the probe nee dle to the cen ter of the le sion. No vi sual er ror mea sure ments were made in this ex per i ment due to the op ti cal opac ity of the phan tom. The ex per i ment was re peated five times. RESULTS Figure 4 il lus trates the re sults of one typ i cal nee dle touch ex per i ment in clud ing a pho to - graph (Fig. 4a) show ing the face of the ma trix ar ray trans ducer and the hor i zon tal probe nee - dle tip in close prox im ity to the ver ti cal tar get nee dle tip. In like man ner, figure 4 shows a B-mode im age plane (Fig. 4b) and si mul ta neous C-mode im age plane (Fig. 4c) from the 3D scan ner that was used to mon i tor the au ton o mous ro bot op er a tion. The fig ures show si mul - ta neous per pen dic u lar long axis views of the probe nee dle (solid ar row) as its tip is about to make con tact with the tar get nee dle tip. For a set of five in de pend ent ex per i ments wherein the tar get nee dle was re po si tioned re quir ing an av er age ro bot mo tion of 41.2 mm, a maximum error of 2.00 mm and an average RMS error 1.58 mm or 3.82 %. Fig ure 5 shows anal o gous data for one typ i cal trial of the task of the sim u lated cyst bi opsy per formed by the au ton o mous ro bot while mon i tored by the 3D scan ner as the nee dle tip con tacts the cyst sur face. Fig ure 5 in cludes a long axis view of the probe nee dle in a B-mode mode plane (Fig. 5a). Note that the nee dle tip is ac cu rately aligned in depth with the cen ter of the le sion. Note also the re ver ber a tion tail from the probe nee dle. Fig ure 5b shows the si - mul ta neous or thogo nal long axis view in a C-scan of the probe nee dle tip at the sur face of the cyst. Note that this view of the nee dle tip shows slight lat eral mis align ment with the cen ter of the cyst. In like man ner, fig ure 5c shows a si mul ta neous B-mode short axis view of the nee - dle tip ac cu rately aligned in depth but slightly off-cen ter lat er ally.

6 218 WHITMAN ET AL (a) (b) (c) Fig. 5 Si mul ta neous or thogo nal views from 3D ul tra sound scan of sim u lated nee dle bi opsy of cyst in clud ing: (a) long axis B-scan, (b) long axis C-scan and (c) short axis B-scan. DISCUSSION We have dem on strated fea si bil ity of au ton o mous ro bot op er a tion to ac com plish sim u lated sur gi cal tasks un der the guid ance of 3D ul tra sound im ag ing com bined with im age seg men ta - tion. The RMS er ror for the task of nee dle tip touch ing was 1.58 mm, which com pared fa vor - ably to our pre vi ously-pub lished re sults of 1.34 mm for a hu man ob server us ing the 3D scan ner s mea sure ment soft ware to ac com plish a sim i lar task. The small mea sured er ror is en cour ag ing given the lim i ta tions of the ex per i ment in clud ing: (1) the re stric tions of only 3 axis mo tion by the ro bot, (2) the lim ited spa tial res o lu tion of the real time 3D scan ner and (3) the sim ple im age thresholding soft ware used to seg ment the 3D ul tra sound im ages. For the more dif fi cult task of a cyst bi opsy, the er ror ap peared larger due to the in creased com plex ity of the tar get and the sim ple seg men ta tion al go rithm. How ever, in each trial, the nee dle squarely im pacted the sur face of the cyst to en able a bi opsy. An other is sue for fur ther in ves ti ga tion is the po ten tial er ror in a ro botic sur gi cal in ter ven - tion due to ve loc ity inhomogeneities in tis sue. For ex am ple, fat and mus cle sound ve loc ity can dif fer by as much as 10%, which would re duce the ac cu racy of lo ca tion for any tar get such as a le sion or for eign body em bed ded in tis sue. This topic has been long been of in ter est in our lab o ra tory and oth ers. 10 We note, how ever, that this prob lem is not ex clu sive to a sur - gi cal ro bot but arises for all interventional ul tra sound bi op sies that are per formed many times daily by a hu man op er a tor.

7 AUTONOMOUS SUR GI CAL RO BOT ICS US ING 3-D UL TRA SOUND GUID ANCE 219 The most sig nif i cant lim i ta tion of this fea si bil ity study was the lengthy time re quired for 3D data trans fer and load ing into the Matlab im age anal y sis pro gram. How ever, more mod - ern data trans mis sion and faster soft ware would re duce that time by at least an or der of mag - ni tude. In clin i cal use it would also be nec es sary to re duce sig nif i cantly the five sec onds re quired for the ro bot to move through its task. This prob lem could be solved through a stateof-the-art sur gi cal ro bot, such as the DaVinci sys tem (In tu itive Sur gi cal, Sunnyvale, CA) with mul ti ple arms that could ex ploit the ad van tages of real time 3D ul tra sound and co or di - nate the mo tion of the 3D trans ducer with the con trol of the sur gi cal tools. In ad di tion, the well-known tech nol o gies of res pi ra tion gating and car diac gating would ame lio rate the prob lems as so ci ated with tis sue mo tion dur ing the sur gi cal pro ce dure. We also note that there are much more so phis ti cated im age anal y sis tech nol o gies 11 that could be ap plied to more com plex tasks for an au ton o mous sur gi cal ro bot. Fi nally, the real time 3D color Dopp - ler fea tures of the vol u met ric scan ner of fer ad di tional ca pa bil i ties for au ton o mous ro botic sur gery in tar get ing or avoid ing blood ves sels. CON CLU SION The goal of this study was to test the fea si bil ity of us ing a real-time 3D (RT3D) ul tra sound scan ner with a ma trix ar ray trans ducer probe to guide an au ton o mous sur gi cal ro bot. Em - ploy ing a fiducial align ment mark on the trans ducer to ori ent the ro bot s frame of ref er ence and us ing sim ple thresholding al go rithms to seg ment the 3D im ages, we tested the ac cu racy of us ing the scan ner to au to mat i cally di rect a ro bot arm in sim ple sim u lated sur gi cal tasks in - clud ing nee dle tip touch ing and a sim u lated cyst bi op sies. This fea si bil ity study shows the po ten tial for 3D ul tra sound guid ance of an au ton o mous sur gi cal ro bot for sim ple inter ven - tional tasks in clud ing le sion bi opsy and for eign body re moval. REF ER ENCES 1. Pua EC, Fronheiser MP, No ble J, et al. 3-D ul tra sound guid ance of sur gi cal ro bot ics: a fea si bil ity study. IEEE Trans Ultrason Ferroelec Freq Contr 53, (2006). 2. Pua EC, Idriss F, Wolf PD, Smith SW. Real-time 3D transesophageal echocardiography. Ultrason Imaging 26, (2004). 3. Traumapod 4. Leven JBD, Kumar R, Zhang et al. DaVinci can vas: a telerobotic sur gi cal sys tem with in te grated, ro bot-as - sisted, lap aro scopic ul tra sound ca pa bil ity, pre sented at Med Im age Comput Comput As sist Interv Int Conf., Whit man J, Fronheiser MP, Ivancevich NM, Smith SW. 3-D ul tra sound guid ance of sur gi cal ro bot ics us ing cath e ter trans duc ers: fea si bil ity study. IEEE Trans Ultrason Ferroelec Freq Contr (accepted) (2007). 6. Smith SW, Pavy HE, von Ramm OT. High speed ul tra sound vol u met ric im ag ing sys tem part I: trans ducer de - sign and beam steer ing. IEEE Trans Ultrason Ferroelec Freq Contr 38, (1991). 7. von Ramm OT, Smith SW, Pavy HE. High speed ul tra sound vol u met ric im ag ing sys tem part II: Par al lel pro - cess ing and dis play. IEEE Trans Ultrason Ferroelec Freq Contr 38, (1991). 8. Light ED, Davidsen RE, Fiering JO, et al. Prog ress in two-di men sional ar rays for real-time vol u met ric im ag - ing. Ultrasonic Imaging 20, 1-15 (1998). 9. Madsen EL, Zagzebski JA, Insana MF, et al. Ul tra son i cally tis sue-mim ick ing liver in clud ing the fre - quency-dependence of backscatter. Med Phys 9, (1982). 10. An der son ME, McKeag MSTrahey GE. The im pact of sound speed er rors on med i cal ul tra sound im ag ing. J Acoust Soc Amer 107, (2000). 11. Sonka M, Hlavac, V., Boyle, R. Image Processing, Analysis, and Machine Vision (PWS Pub lish ing, Pa cific Grove, CA, 1999).