Interactive Level-Set Segmentation on the GPU
Problem Statement Goal Interactive system for deformable surface manipulation Level-sets Challenges Deformation is slow Deformation is hard to control Solution Accelerate level-set computation with GPU Visualize computation in real-time 2
Collaborators University of Utah Joe Kniss Joshua Cates Charles Hansen Ross Whitaker 3
Introduction Introduction Deformable Surfaces Applications of Level-Sets Fluid simulation Surface reconstruction for 3D scanning Surface processing Image / Volume segmentation 4
Introduction Level-Set Method Implicit surface Distance transform denotes inside/outside Surface motion F = Signed speed in direction of normal 5
Introduction CPU Level-Set Acceleration Narrow-Band/Sparse-Grid Compute PDE only near the surface Adalsteinson et al. 1995 Whitaker et al. 1998 Peng et al. 1999 Houston et al., 2004 Museth et al., 2004-2006 Time-dependent, sparse-grid solver Initialize Domain Compute Update Domain 6
GPU Level-Set History Introduction Strzodka et al. 2001 2D level-set solver on NVIDIA GeForce 2 No narrow-band optimization Lefohn et al. 2002 Brute force 3D implementation on ATI Radeon 8500 No faster than CPU, but ~10x more computations No narrow-band optimization Lefohn et al. 2003 / 2004 Narrow band GPU 3D level set solver Crane et al. 2007 3D level set solver as part of fluid simulation in NVIDIA G80 launch demo Mask unused grid cells Kolb et al. 2007 GPU particle level sets 7
GPU Level-Set History Introduction Strzodka et al. 2001 2D level-set solver on NVIDIA GeForce 2 No narrow-band optimization Lefohn et al. 2002 Brute force 3D implementation on ATI Radeon 8500 No faster than CPU, but ~10x more computations No narrow-band optimization Lefohn et al. 2003 / 2004 Narrow band GPU 3D level set solver Crane et al. 2007 3D level set solver as part of fluid simulation in NVIDIA G80 launch demo Mask unused grid cells Kolb et al. 2007 GPU particle level sets 8
Algorithm GPU Narrow-Band Solver Sparse Volume Computation CPU algorithm: Traverse list of active voxels GPU algorithm: Compute all active voxels in parallel Initialize Domain Compute Update Domain Data structures change after each PDE time step 9
Algorithm A Dynamic, Sparse GPU Solver GPU: Computes PDE CPU: Manages GPU memory Physical Addresses for Active Memory Pages CPU GPU PDE Computation 15-250 passes Memory Requests 10
Level-Set Segmentation Surface velocity attracts level set to desired feature % Smoothing Data-Based Speed Curvature Speed Segmentation Parameters 1) Intensity value of interest (center) 2) Width of intensity interval (variance) 3) Percentage of data vs. smoothing 11
Data speed term Attract level set to range of voxel intensities Width (Variance) Center (Mean) D(I) D(I)= 0 I (Intensity) 12
Curvature speed term Enforce surface smoothness Prevent segmentation leaks Smooth noisy solution Seed Surface No Curvature With Curvature 13
Movie 14
Interactive 3D Level Set Visualization Use GPU to perform interactive volume rendering of the level set solution while it evolves Render with original data Directly render level set data without reformatting data 3D user interface to guide evolving level set surface 15
A Dynamic, Sparse GPU Data Structure Algorithm Multi-Dimensional Virtual Memory 3D virtual memory 2D physical memory 16 x 16 pixel pages 16
Direct Volume Rendering of Level Set Reconstruct 2D Slice of Virtual Memory Space On-the-fly decompression on GPU Use 2D geometry and texture coordinates Visualization 17
Direct Volume Rendering of Level Set Deferred Filtering: Volume Rendering Compressed Data 2D slice-based rendering: No data duplication Tri-linear interpolation Full transfer function and lighting capabilities Visualization 18
Application Level-Set Segmentation Application Idea: Segment Surface from 3D Image Begin with seed surface Deform surface into target segmentation 19
Results Demo Segmentation of MRI volumes 128 3 scalar volume Hardware Details ATI Radeon 9800 Pro 1.7 GHz Intel Pentium 4 1 GB of RAM 20
Movie 21
Region-of-Interest Volume Rendering Limit extent of volume rendering Use level-set segmentation to specify region Add level-set value to transfer function 22
Evaluation User Study Goal Can a user quickly find parameter settings to create an accurate, precise 3D segmentation? Evaluation Relative to hand contouring 23
User Study Methodology Evaluation Six users and nine data sets Harvard Brigham and Women s Hospital Brain Tumor Database 256 x 256 x 124 MRI No pre-processing of data & no hidden parameters Ground truth Expert hand contouring STAPLE method (Warfield et al. MICCAI 2002) 24
Evaluation User Study Results Efficiency 6 ± 3 minutes per segmentation (vs multiple hours) Solver idle 90% - 95% of time Precision Intersubject similarity significantly better 94.04% ± 0.04% vs. 82.65% ± 0.07% Accuracy Within error bounds of expert hand segmentations Compares well with other semi-automatic techniques Kaus et al., Radiology, 2001 25
Summary Conclusions Interactive Level-Set System 10x 15x speedup over optimized CPU implementation Intuitive parameter tuning User study evaluation But 26
That was three+ years ago GPUs are 6-7x faster! New GPU capabilities make building dynamic data structures easier and more efficient GPU data structures better understood (Glift, etc.) New, faster CPU level-set methods (RLE, etc.) Tremendous opportunity for new research 27
Conclusions Future Directions Other Level-Set Applications Surface processing, surface reconstruction, physical simulation Better User Interface for Level Sets Add more user control of evolving level set solver More powerful editing of level set solution Interactive Visulation User-controllable PDE solvers Combine automatic and by-hand methods New visualization and computation challenges 28
Acknowledgements Joe Kniss Volume rendering Josh Cates Tumor user study Gordon Kindlmann Teem raster-data toolkit Milan Ikits GLEW OpenGL extension wrangler Ross Whitaker, Charles Hansen, Steven Parker and John Owens ATI: Evan Hart, Mark Segal, Jeff Royle, and Jason Mitchell Brigham and Women s Hospital National Science Foundation Graduate Fellowship Office of Naval Research grant #N000140110033 National Science Foundation grant #ACI008915 and #CCR0092065 29
Questions? For More Information Google Lefohn level set http://graphics.cs.ucdavis.edu/~lefohn/ Journal Papers Based on this Work Lefohn, Kniss, Hansen, Whitaker, A Streaming Narrow Band Algorithm: Interactive Computation and Visualization of Level Sets, IEEE Transactions on Visualization and Computer Graphics, 10 (40), Jul / Aug, pp. 422-433, 2004 Cates, Lefohn, Whitaker, GIST: An Interactive, GPU-Based Level-Set Segmentation Tool for 3D Medical Images, Medical Image Analysis, to appear 2004 30