Real-Time Implementation of a New Contour Tracking Procedure in a Multi-Processor DSP System.

Transcription

1 Real-Time Implementation of a New Contour Tracking Procedure in a Multi-Processor DSP System. V. GEMIGNANI (1), M. DEMI (1)(2), M. PATERNI (1), A. BENASSI (1) (1) CNR - Institute of Clinical Physiology, v. Savi 8, Pisa (2) Esaote SpA, Florence ITALY Abstract: - In this paper the implementation of a contour tracking procedure is presented. If a rough contour of the desired structure is available on the first frame of a sequence, the contours on the subsequent frames are automatically outlined in real time. Furthermore, indices of clinical interest are calculated from the computed contour and they are graphically displayed in real time. The software runs in an integrated software/hardware environment based on the interaction of a PC and a multi-processor DSP (TMS320C80). The paper describes the solutions adopted in order to obtain an efficient software implementation. The performance of the system was investigated and a comparison with a PC implementation is reported. The system was tested by monitoring the area changes of the thoracic aorta in sequences of echographic images. Key-Words: real-time, contour tracking, DSP, medical imaging, multi-processor. 1 Introduction In recent years, there has been increasing interest in the design and implementation of systems for real time contour tracking [1][2]. In particular, in medical imaging, a real time quantitative analysis of sequences of images is of considerable clinical interest [3][4]. Our task is to compute the contour of the structure under examination automatically, to estimate the indices of interest and to display the results in real-time. In a previous paper [5], a new automatic contour tracking procedure was presented and tested over sequences of cardiovascular images. Moreover, in [6] the TMS320C80 multi-processor DSP [7][8] proved to be a suitable device for obtaining a real time execution of the new contour tracking algorithm. In this paper, a Windows NT based PC equipped with a TMS320C80 DSP board [9] is adopted to realize the real time image processing system. The PC provides a suitable environment for the development of a flexible and powerful user interface. The DSP board provides the processing capability, the image capture and the image display resources necessary to obtain real time performances. The aim of this paper is to illustrate the solutions we adopted in the implementation of the procedure. The most onerous part of the work was the implementation of the algorithms on the DSP. In fact, the new high performance DSPs adopt special architectures which make it difficult for a compiler to produce an efficient code. Therefore, these devices are typically programmed in assembly language and efficient programming is actually a difficult task. Several library routines are available to simplify the implementation work. These routines can be used as building blocks to assemble more complex applications in a quicker manner [10]. Furthermore, in [11] a multi-layered library was developed to decrease the overhead of the resultant application. However, in this paper the algorithm was completely implemented without the use of libraries. This solution allowed a higher optimization of the code. Sequences of images of the descending thoracic aorta recorded by transesophageal echocardiography were used to test the system. The cross-sectional area of the aorta was computed in real time and plotted against time on the user interface. 2 Algorithm Overview

2 A new contour tracking procedure based on the FOAM (first order absolute moment) mass center operator was presented in [5][6]. In this paragraph, an overview of the procedure is reported. Let f(n, be the gray level map of an image, then the generalized first order absolute moment is computed as follows: e( n, = f1( n, f ( n m g2( Θ2 where f1( n, = f ( n m g1( Θ1 where Z represents the integer numbers and g i ( are discrete Gaussian functions with standard deviation σ i defined as: if ( Θ i ; g i (=0 elsewhere. Let Γ be a discrete vector with components l; then the FOAM mass center vector is defined as follows: 1 b( n, = e( n, Θi = { Z : k + l 3σ } gi( = Θ Θ i 2 if e(n, 0; b(n,=0 elsewhere. 1 f ( n, f ( n m à g 2( ) 1 l Let us consider a step straight discontinuity and let us compute the vector b at a point p at a distance d from the discontinuity. If d<3σ 2 and if the right configuration (σ 1,σ 2 ) of the filter is chosen, the vector b provides a mass center p which is closer to the discontinuity than p independently of the distance d [5]. Therefore, if an approximate starting contour C S is traced manually by an operator on the first frame of an image sequence, the true contour C 1 can be estimated by iteratively computing the FOAM mass center at the points of C S. Three iterations were enough to locate the contour in the synthetic and clinical images used to test the procedure. Once C 1 is computed, it is used as the i ( 2 2 k + l ) 2 2σ i 2 2 ( ) e k + l 2 2 σ i e approximate starting contour to compute C 2 in the second frame and so on. Therefore, if a starting contour is available on the first image of the sequence, the procedure automatically outlines the contours on the subsequent images. The computational cost of the procedure is primarily due to the iterative FOAM mass center computation. However, the algorithm and the number of steps are the same for all the points of the contour and the FOAM is computed separately at every single point. Therefore, the same algorithm is repeated several times with different input data. This allows an efficient parallel implementation in a SPMD (single program multiple data) scheme. 3 Implementation The system is based on the interaction between a single-cpu PC and a multi-cpu DSP board. In all, 6 CPUs are present: I. One general purpose processor (Pentium MMX). II. One RISC processor, the Master Processor (MP). III. Four DSP processors, the Parallel Processors (PPs). Due to the different architectures of the above mentioned processors, it is necessary to use three different implementation levels and three different software development tools (figure 1). Personal Computer DSP Board MP Contour data PP0 PP1 PP2 PP3 Host level MP level PP level Figure 1: Six CPUs and three levels of software implementation are present in the system. 3.1 Host level The Pentium-based PC can be efficiently programmed by using a high-level language. In this paper, the Microsoft Visual C language was adopted. The main function of the software running on the PC is to act as a user interface. It allows the operator to set-up the parameters of the filters, and to start and stop both the processing and the display

3 activities. Furthermore, it allows the operator to trace the starting contour on the first frame of the sequence by using the mouse. Once the processing activity has started, the PC continuously receives the data of the contours which are calculated by the DSP board. A new contour is received from the DSP board every 40ms and it is displayed in realtime in a special window. The computation of indices such as area and perimeter was implemented at the host level. This solution was chosen because the computational cost of these algorithms is not high, so they can be easily executed in real time by using a high level system. Moreover, even if the contour tracking procedure is always the same, the computation of specific indices is necessary for every application. Therefore, the use of the same DSP software with different PC programs proves to be a more flexible solution. 3.2 MP level The MP coordinates the on board processing activities and the communication with the PC. The software of the RISC MP was written in C and it runs in a multitasking environment. Three tasks were implemented (figure 2): Initialization Wait Initialization Task Wait for a command from the PC Process the command T_REG T_TRANSF Host Communication Task Image transfer from input to output buffer Regularization Wait until a new frame is captured PP start processing command Wait for PP end-ofprocessing Video Processing Task Figure 2: Three tasks were implemented at the MP level implementation. Initialization task: The task only operates when the board is started up. The program initializes the on board resources and the constants which are used by the PPs. Host communication task: This task manages the communication between the DSP board and the PC. The MP is configured as a server so it continuously waits for a command from the host. Typical commands regard the configuration of the filter, the start and stop of data acquisition and processing, the management of the mouse pointer, etc. Video task: The video task manages the video processing activity which needs to be synchronized with the data flow. For this purpose, before starting a new processing iteration, the MP waits until a new frame is captured in memory. Afterwards, all the processing activities need to be completed in 40ms to obtain a real time execution. The contour tracking procedure requires two distinct algorithms. The first algorithm computes the FOAM mass center and it is implemented at the PP level. In this case, the role of the MP is limited to coordinating the PPs which work in parallel in a SPMD scheme. The second one is a regularization algorithm which operates on the points of the contour. The computational cost of this algorithm is not onerous because of the small amount of data. However, the implementation is demanding because of the complexity of the algorithm. Moreover, more than one algorithms must be implemented to provide a configurable regularization. Therefore, the best solution in terms of speed, simplicity and flexibility proved to be the implementation of the algorithms on the MP using a high level language. Once the new contour is computed, it is superimposed on the image data and the obtained image is transferred to the output buffer. To conclude, three operations are executed sequentially: the iterative computation of the FOAM mass center, the regularization process and the image data transfer. If we define the time necessary for the above mentioned operations T_FOAM, T_REG and T_TRANSF, respectively, the constraint in order to obtain real time processing at 25fps is: T_FOAM + T_REG + T_TRANSF < 40ms These values vary depending on the data size. However, the image size, the filter configuration and the number of points of the contours are constant and consequently, the contour tracking procedure requires a constant number of operations at every image of the sequence. Therefore, since the processing time is about the same for every image, the real time working capability can be easily verified by using the software development tools. 3.3 PP level Since the computation of the FOAM mass center is the most onerous part of the procedure, it was implemented at the PP level. This implementation

4 required a lot of work because its optimization greatly influences the performance of the whole procedure. The use of the assembly language is mandatory at the PP level to achieve reasonable performances. Moreover, if the data are in the offchip memory, the DMA coprocessor must be used. In fact, direct external memory access is very costly on the PPs (10~15 cycles per access in the best case) and the data caching is not performed. Therefore, before starting the processing activity, the data need to be downloaded in the internal memory where the access time is only 1 cycle. The DMA coprocessor permits automatic multidimensional data transfer called PT (Packed Transfer). The algorithm can be divided into two parts: I. The convolution to obtain the value f 1 (n,. II. The summation of the absolute values of the differences. The input data of the convolution algorithm are the constant matrix g 1 ( and a region of interest ROI_1 of the image. Since g 1 ( is a constant, it was permanently stored in the local memory of every PP where it is accessible at the fastest speed. Instead, the data of ROI_1 are located in the external memory. Therefore, a PT is necessary in order to transfer the data to the internal memory before the data processing can be started. The inner loop of the procedure is two instructions long. Nevertheless, at every loop iteration two MACs (Multiply and Accumulate) are executed. Therefore, one multiplication and one addition are carried out per cycle. As regards the summation algorithm, the input data are again a constant matrix g 2 ( and a ROI_2 of the image. g 2 ( is permanently stored in the local memory whereas the ROI_2 is transferred from the external memory to the internal memory by using the DMA coprocessor. The inner loop of the procedure, which is 13 instructions long, processes four pixels per iteration. If the FOAM mass center is computed at two near points, the two ROI_1 and the two ROI_2 partially overlap. Therefore, when the ROI_i of the second point have to be transferred, some data are already present in the internal memory. We can either reload the whole ROI_i or introduce a more suitable mechanism which is able to re-use the previous data. The first approach was preferred because it simplifies and speeds up the subsequent processing phase. Moreover, when processing a single point, ROI_1 is always included in ROI_2. Nevertheless, two distinct PT are used to transfer the two ROI_i. This solution was adopted because every PT rearranges the organization of the data to speed up the execution of the relative processing algorithm. In particular, the management of the pointers and the loop controllers can be simplified by means of an optimized data arrangement. The adopted solution increases the data transfer time but greatly optimizes the implementation of the algorithm. The two functions g i ( have non-zero values only in the domains θ i so the two ROI_i should also be circular. Nevertheless, managing circular regions is a difficult task. Therefore, the ROI_i are square regions which include the θ i. The size of the ROI_i is n i xn i where n i =int(6 σ i )+1. In table 1 the size of the data and the number of operations of the algorithms are summarized. (a) Convolutio n (b) Summation Convolutio n Summation 2x(n 2 xn 2 )+n Input data Output Constant Variable data g 1 : [n 1 xn 1 ] ROI_1 : [n 1 xn 1 ] f 1 : [1x1] g 2 : [n 2 xn 2 ] ROI_2 : [n 2 xn 2 ] b : [2x1] MAC Subtr. Abs. Divis. n 1 xn n 2 xn 2 n 2 xn 2 2 Table 1: (a) The data and their size are summarized for both parts of the algorithm. (b) The number of MACs, subtractions, absolute values and divisions are reported as a function of the data size. 3.4 Arithmetic The processing speed of the TMS320C80 is primarily due to the performances of the four PPs. Each PP is an advanced 32-bit fixed-point DSP with special features to improve processing capability. In fact, they offer the possibility of splitting the data unit to process one 32-bit operation, two 16-bit operations or four 8-bit operations. The multiplier can also execute one 16-bit operation or two 8-bit operations. Therefore, the data representation

5 heavily influences the performance of the procedure. If a TMS320C80 is adopted, a fixed-point arithmetic must be mainly used. In fact, although the multiprocessor we used offers the remarkable floating point performance of up to 80 MFLOPS (at 40MHz), the device is optimized for fixed-point arithmetic. Moreover, because of its complexity, the use of this DSP is justified only if its resources are efficiently exploited. In the computation of the FOAM mass center, the input data are 8-bit integer values (256 gray level images are used). The masks g 1 ( and g 2 ( were also implemented as 8-bit integer values. Therefore, in the convolution algorithm, the multiplication of 8-bit data and the accumulation of 16-bit data were performed. Subsequently, the value f 1 (n,, which is computed as a 16 bit value, is rounded to an 8-bit value. Consequently, the input data in the summation algorithm are 8-bit wide and the splitting capability of the data unit and of the multiplier can be used again efficiently. Moreover, the SPMD scheme where the four PPs work in parallel under the management of the MP is exploited. To conclude, the computation of the FOAM mass center in 8-bit gray level images proved to be a task which efficiently matched the architecture of the DSP we adopted. 4 Results and Conclusions The performances of the system were tested with several filter configurations. Let us consider the case in which three iterations of the FOAM mass center are computed at 40 points of a contour and the configuration of n 1 =17pixels, n 2 =25pixels is adopted: the time required to compute the three iterations at all the points is T_FOAM=4.6ms. The measurement was obtained in run-time from the start-of-processing command of the PPs to the endof-processing notification. Therefore, the time required to compute the FOAM mass center using one PP was: 4.6ms / (40knots 3iterations) 4PPs = 153µs Let us now consider the PP assembler procedure only, without taking into consideration the ROI_i data transfer time. The time required to compute the FOAM mass center was 77.5µs. Therefore the computational time is roughly doubled in comparison to the time required to process the data once they are in the on-chip memory. Three regularization algorithms were implemented: the uniform spacing of the points of the contour, a mean filtering, and a median filtering. When a contour with 40 points was used, T_REG was less then 1.5ms for all the procedures tested. The T_TRANSF parameter depends on the size of the image data. In this paper, 8-bits gray level images of 512x512 pixels were used and the time T_TRANS necessary to transfer the data to the output buffer resulted in 4.4ms. In order to evaluate the advantages obtained thanks to the adoption of a DSP architecture, a comparison with a PC implementation is reported in figure 3. The results refer to the computation of the FOAM mass center. A PC Pentium II 350 MHz was used and the algorithm was implemented in C. In all the configurations tested, a considerable increase in the computational speed was obtained. For example, if n 1 =25pixels and n 2 =37pixels, the computational time is 1330µs for the PC and 54µs for the DSP. Therefore, the remarkable advantage obtained justifies the adoption of the DSP architecture and the onerous implementation. [us] Figure 3: Computational time of the FOAM mass center algorithm for both the PC and the DSP implementation. The configuration of the filter is σ 1 =(2/3)σ 2. PC n2 [pixels] In figures 4 and 5, the result of the monitoring of the aorta area changes in an echographic examination is reported. The images were captured with a resolution of 512x512 pixels and a frame rate of 25fps. The new system proved to be a versatile tool for the real time estimation of indices of medical interest. Our current efforts are directed towards increasing the robustness of the procedure as this seems to be the weakness of the system. In this paper, constraints on the shape of the contour have

6 not been introduced and a large flexibility in the contour tracking capability was obtained. However, the robustness of the system is lower when compared to other papers which introduced such constraints [3][4]. Figure 4: Sequences of images of the descending thoracic aorta recorded by transesophageal echocardiography were used to test the system. Figure 5: User interface. At the top of the screen, the menus and the toolbar are located. In the central windows, the contour (on the left) and the cross-sectional area changes of the aorta (on the right) are depicted in real time. At the bottom, minimum and maximum values of the area are reported. Echocardiographic Sequence. Proc. ICCV98, Bombay, India, 1998, pp [4] McEachen, J.C.II and Ducan,J.S. Shapebased tracking of left ventricular wall motion. IEEE Trans. Medical Imaging, 1997,16 (3), pp [5] M. Paterni, M. Demi, C. Morizzo, M. Kozakova, C. Palombo The First Order Absolute Moment in Contour Tracking. Proc. Computers in Cardiology, Lund, Sweden, 1997, pp [6] V. Gemignani, S. Provvedi, M. Demi, M. Paterni, A. Benassi A DSP-Based Real Time Contour Tracking System. Proc. of ICIAP99, Venice, Italy, 1999, pp [7] Texas Instruments TMS320c8x System-Level Synopsis, SPRU113, [8] K. Guttag, R.J. Gove, J.R. Van Aken, A single-chip multiprocessor for multimedia: The MVP, IEEE Computer Graph. Appl., 1997, 12 (6), pp [9] Texas Instruments TMS320c80 Software Development Board, SPRU178, [10] F. Lohier, L. Lacassagne A Generic Methodology for the Software Managing of Caches in Multi-processors DSP architectures. Proc. of ICASSP99, Phoenix, Arizona, 1999, vol. 4, pp [11] J.Kim, Y.Kim UWICL: A Multi-Layered Parallel Image Computing Library for Single- Chip Multiprocessor-based Time-Critical Systems. Real Time Imaging, 1996, 2, pp References: [1] M. Hernández, J. Cabrera, M. Castrillón, A. Domínguez, C. Guerra, D. Hernández, J. Isern DESEO: An Active Vision System for Detection, Tracking and Recognition. Proc. ICVS99, Las Palmas, Spain, 1999, pp [2] Z.Li, H.Wang Real time 3D Motion Tracking with Known Geometric Models. Real-Time Imaging, 1999, 5, pp [3] Gary Jacob, J. Alison Noble, Andrew Blake Robust Contour Tracking in