REGISTRATION OF 3D ULTRASOUND IMAGES TO SURFACE MODELS OF THE HEART

Transcription

1 To appear in: Proceedings of the Interface to Real & Virtual Worlds (Montpellier, France, May 28-30, 1997) REGISTRATION OF 3D ULTRASOUND IMAGES TO SURFACE MODELS OF THE HEART Stefan Pieper, Michael Weidenbach*, Thomas Berlage 1 Introduction GMD - German National Research Center for Information Technology D Sankt Augustin, Germany stefan.pieper@gmd.de, thomas.berlage@gmd.de *Department of Pediatric Cardiology, University Bonn Adenauerallee 119, D Bonn, Germany Ultrasound images of the heart are obtained by a probe placed on the patient's chest. The resultant image is a triangular plane with its base in the middle of the body and the tip close to the probe on the chest. Thus, image planes with a different orientation (e.g. parallel to the surface of the body) are impossible to obtain. The aim of current 3D ultrasound of the heart is to provide the physician with diagnostically important views that are impossible to gain with standard 2D ultrasound devices. These imaging planes are calculated from 3D data sets generated by rotating a standard ultrasound probe around its center axis. Imaging planes obtained from a 3D data set are independent of the probe position. The disadvantage is that the physician evaluating such an image off-line has to orient himself in this volume without knowing the exact position of the plane in respect to the diagnostic relevant views. He needs a geometrical reference scenario as guidance. In the CardiAssist project (Berlage & Grunst, 1995), we have supplemented the ultrasound image by a surface model of the heart (see Fig. 1). This scenario not only serves as an orientation during diagnosis (Berlage et al., 1996), but also for training purposes and teleconsultation (Berlage, 1996). Figure 1: Augmenting a slice view of a 3D data set with contour outlines and a 3D visualization of the plane location within the heart. The ultrasound data sets have to be registered with the heart model so that every action undertaken in the data set can be controlled by visual feedback from the model. We have implemented a registration mechanism based on the identification of unambiguous and easily recognizable structures, so called landmarks. Based on the positions of these landmarks, an algorithm adjusts the model to the data set.

2 Because of geometric differences between real hearts and our ideal model, the match will never be perfect, but our first experiences suggest that this does not diminish its value as an orientation. 2 Related Work Robb (1995) gives a broad overview of different registration techniques. Automatic or semi-automatic procedures are covered as well as manual post-processing of data sets in order to acquire registration information. The focal point of the reflections is shape-based registration since it is believed to carry the biggest potential in fast and robust automatic registration. Problems inherent in the different approaches are addressed, too, like e.g. attraction to local minima in shape-based registration or inconvenient use of physical markers for landmark points. Other authors describe approaches that try to overcome the well-known problems. Malandain et al. (1995) discuss a "physically based" method for shape-based registration that allows the transformed volume to "jump over" local minima, thus preventing the algorithm from stopping the search at the wrong point. Van den Elsen and Viergever (1991) use a marker attached to the patient during image acquisition that is visible in CT as well as MRI scans. Multi-modal imaging becomes possible through constant identification of the markers in both data sets. However, all these registration procedures address the problem of registering different images of the same patient. Although the quality of the fit may also be limited by different non-linear properties of different imaging modalities, in general it is possible to identify an optimal mapping. In our case, the model nearly always has a different geometrical structure than the patient image that it is registered to, therefore different criteria apply to our procedure. Here it is more important to quickly identify a match that is "good enough" for subsequent exploration of the data set. 3 Review of Different Registration Procedures The approaches to registration of different 3D imaging modalities can be classified into two major groups: automatic registration and registration through manually acquired information. The first category includes the following techniques: 1. shape-based registration, where surfaces are computed from volume data, 2. registration based on internal landmarks, identified automatically within each of the volumes, and 3. registration based on physical landmarks (markers attached to the patient). Registration by manually acquired information includes the following techniques: 1. manual transformation of a 3D object in order to match another 3D object, where both are simultaneously visualized in a common 3D scene, where their spatial relation can be verified from arbritary view points, and 2. registration based on manually identified landmarks. Automatic registration techniques are very difficult to apply in echocardiography. One reason is that ultrasound images are of low quality compared to other imaging modalities, which makes automatic structure detection difficult. And second, the heart is a complex moving structure and it is not possible to attach physical markers. From the manual mechanisms, we have chosen the landmark method, because it uses existing medical knowledge and is easier to apply. On the other hand, depending on the landmarks chosen, it may lead to a worse overall fit between data set and model. A critical evaluation is therefore necessary. 2

3 4 Landmark-Based Registration for 3D Echocardiography The 3D ultrasound data set is registered with the surface model of the heart in two steps: first the cardiologist interactively identifies three particular landmarks in the data set, second an algorithm computes a transformation that yields a close match between structures in the data set and the model. The CardiAssist system offers different sets of landmarks for different situations. The heart model already includes the positions of these landmarks. Depending on what structures can be identified in the data the cardiologist selects a suitable set. The selection of landmarks needs special attention since it influences the quality of registration. Best results will be achieved if the ultrasound volume is acquired with regard to distribution of easily recognizable structures. Navigation through the ultrasound data is managed by a user interface that mainly consists of three simultaneous views on the volume as can be seen in Figure 2. Each view is a slice of the data set and each slice is orthogonal to the other two. The intersection of the three slice planes defines the current position in the data volume. Figure 2: The graphical user interface. The main part of the interface displays ultrasound slices. A cone and a plane visualize the slice plane's position and orientation in the ultrasound volume. In the lower right portion of the window the list of selected landmarks and the motion control elements are situated. In each of the views a cross-hair cursor represents the current position within that plane. The horizontal and vertical lines of the cross-hair cursor mark the position of the two other planes. The cursor can be moved within a plane by mouse interaction. As a result the other two planes move along their normals (Fig. 3) and the views are updated continuously to provide maximum feedback. 3

4 Figure 3: Cursor movement. The two images on the left show the cursor placed in the middle of the volume. On the right the cursor has been moved within the upper view as shown by the shift of the vertical cursor line. The lower view has been updated accordingly as can be seen by the new position of its slice plane representation in the lower right corner. Assisted by this mechanism the user is able to control his positioning process from different points of view. Navigation starts with the standard view on the heart that was used during volume data acquisition and is therefore best suited as a starting point in landmark identification. The navigation mechanism could be extended in two ways: First, planes could be rotated so that they are not necessarily perpendicular to each other. This would allow the cardiologist to choose views that resemble standard views in 2D ultrasound (e.g. four chamber view). Second, the complete set of planes could be rotated with respect to the data volume preserving their relation to each other. Both cases increase the navigational complexity and thus make orientation more difficult. The three landmarks are represented as points in volume coordinate space and form a triangle. The corresponding coordinates in the heart model also define a triangle. In order to find a transformation from the volume triangle to the model triangle we use the unique center axis from which all three points have equal distance. The algorithm performs four steps: 1. scaling the volume triangle so that the radius is identical, 2. rotating the volume triangle such that center axes are parallel, 3. translating the volume triangle so that the center points match, and 4. rotating the volume triangle around its center axis to minimize distance. 4

5 5 Technical Implementation The system was developed on Silicon Graphics workstations under C++ using RapidApp, a high-level development tool for graphical applications, and OpenInventor, a high-level object oriented graphics library based on OpenGL. OpenInventor objects are used for simple 3D graphical elements as well as for storage and computation of geometric transformations. Because of its modular structure and the availability of OpenInventor on other platforms the system can easily be ported, e.g. to Windows NT. Computation of slices of an ultrasound volume is implemented as a C++ class which was designed to provide convenient and efficient handling of 4D data (3D space plus time). Since the class offers slicing in arbitrary planes, more complex navigational mechanisms can be implemented without changes. The system's demand for resources, i.e. memory and cpu time, is relatively high due to the fact that ultrasound volumes are large quantities of data (approximately 30 to 70 Megabytes) and slicing has to be fast in order to provide optimal interactivity. 6 First Experiences We have evaluated our registration mechanism with several data sets captured with a TomTec device. The registration module provides a number of different landmarks identified in the model. The muscular, curvilinear chamber walls of the heart (except the apex) were not suitable for the registration procedure since it was impossible to unambiguously identify concordant spots in the original data set and the model respectively. The landmarks we have identified are primarily specific parts of the heart valves (like the commissures and the centre of the valves) and the apex of the heart. A sufficient alignment is possible in most data sets we tried, allowing an intuitive exploration of the data set. Although outlines, e.g. of the chambers or valves, mostly do not overlap exactly between the model and the data set, it is easy to identify their relation visually. In the contour view it is even an advantage if the contours do not obscure the underlying data, so that the cardiologist can easily distinguish between real and virtual data. The shape of the heart significantly varies with age and hemodynamic conditions, sometimes making the geometric assumptions of our heart model invalid. The alignment therefore diminishes the more the geometry of the real heart changes, for example in heart diseases. Nevertheless, also in cases the registration was not optimal it still gave a rough idea of the position in the data set, the structures within the specific plane and the direction of movement when the plane was adjusted. In some complex congenital heart diseases registration was not even attempted with the current heart model. As we are at the same time developing mechanisms to reflect pathologies in our heart model, this will be the topic of future work. 7 Future Work Extensions of the current system are possible in three respects: extension of the user interface, definition of new types of landmarks and landmark sets, and automatic optimization of registration. Extension of the user interface. More complex navigation mechanisms like those described in Section 4 can only be offered if the cardiologist is assisted by an additional 3D model representing the structures visible in the data set. One way to achieve this is to provide a heart model that is roughly registered with the ultrasound data during acquisition using a spatial tracking device. Another approach is to display 5

6 a surface model computed from 3D ultrasound data that must at least be precise enough to allow the cardiologist to recognize the key structures of the heart. In both cases the current slicing planes have to be visualized within the model to provide orientational cues. New types of landmarks and landmark sets. Further types of landmarks could be simple geometric objects other than points, e.g. lines, curves or ellipsis. They can be used for more precise registration and automatic adaption of the heart model to ultrasound data. This approach needs a more complex navigation mechanism to place the two-dimensional structures in an arbritary plane of the volume. New types of landmark sets are those that contain more than three landmarks, the minimal amount of points needed for registration. This overspecification can be used for automatic adaption of the heart model to ultrasound data as well as for a better overall match of volume data and heart model. The main problem is to identify a bigger number of different landmarks in most data sets. Automatic optimization. In order to optimize matching between ultrasound data and heart model the latter can be deformed to reflect morphological features of the actual heart. This adaption can be automated if the cardiologist can supply sufficient landmark information about the ultrasound volume. Methods to achieve this are described in the preceding paragraph. 8 Conclusion We have developed a registration procedure to map a virtual heart model onto 3D ultrasound data sets of the heart. Even though the match is not perfect, the resulting information can be used as a guidance in the diagnostic process. The first target of further development must be to make the procedure simple, fast and effective in order to be used in daily routine. We will enter a validation phase with this mechanism built into actual ultrasound scanners. This field trial is performed as a part of the CardiAssist project supported by the European Union under the Health Telematics program. 9 References Berlage, T., Grunst, G., CardiAssist: Developing a support platform for 3D ultrasound. In Proceedings of the 9th Microgravity Summer School Space- and Telemedicine (Attendorn, Germany, 4 7 July 1995), DLR, Köln, IB /04, pp Berlage, T., Fox, T., Grunst, G., Quast, K., Supporting Ultrasound Diagnosis Using An Animated 3D Model of the Heart. In Proceedings of IEEE Multimedia Systems 96, (Hiroshima, Japan, June 17 21, 1996), Berlage, T., Design of a Teleconsultation Service for Echocardiography. In Proceedings of the Interface to Real & Virtual Worlds, (Montpellier, France, May 21 24, 1996), pp Malandain, G., Fernandez-Vidal, S., Rocchisani, J.-M., Physically Based Rigid Registration of 3-D Free-Form Objects: Application to Medical Imaging. Rapport de recherche no Institut National de Recherche en Informatique et en Automatique, Sophia-Antipolis, Robb, Richard A., Three-Dimensional Biomedical Imaging. Principles and Practice. VCH Publishers, Inc., New York, Van den Elsen, P.A., Viergever, M.A., Marker Guided Registration of Electromagnetic Dipole Data with Tomographic Images. In Proceedings of the Information Processing in Medical Imaging (Wye, UK, July 1991), Springer, Berlin, 1991, pp