58 A Survey of Image Processing Software and Image Databases

Reeves, S.J. A Survey of Image Processing Software and Image Databases Digital Signal Processing Handbook Ed. Vijay K. Madisetti and Douglas B. Williams Boca Raton: CRC Press LLC, 1999 c 1999byCRCPressLLC

58 A Survey of Image Processing Software and Image Databases Stanley J. Reeves Auburn University 58.1 Image Processing Software General Image Utilities Specialized Image Utilities Programming/Analysis Environments 58.2 Image Databases Images by Form Image processing has moved into the mainstream, not only of the engineering world, but of society in general. Personal computers are now capable of handling large graphics and images with ease, and fast networks and modems transfer images in a fraction of the time required just a few years ago. Image manipulation software is a common item on PCs, and CD-ROMs filled with images and multimedia databases are standard fare in the realm of electronic publishing. Furthermore, the development of areas such as data compression, neural networks and pattern recognition, computer vision, and multimedia systems have all contributed to the use of and interest in image processing. Likewise, the growth of image processing as an engineering discipline has fueled interest in these other areas. As a result of this symbiotic growth, image processing has increasingly become a standard tool in the repertoire of the engineer. Because of the popularity of image processing, a large array of tools has emerged for accomplishing various image processing tasks. In addition, a variety of image databases has been created to address the needs of various specialty areas. In this article, we will survey some of the tools available for accomplishing basic image processing tasks and indicate where they may be obtained. Furthermore, we will describe and provide pointers to some of the most generally useful images and image databases. The goal is to identify a basic collection of images and software that will be of use to the nonspecialist. It should also be of use to the specialist who needs a general tool in an area outside his or her specialty. 58.1 Image Processing Software Image processing has become such a broad area that it is sometimes difficult to distinguish what might be considered an image processing package from other software systems. The boundaries among the areas of computer graphics, data visualization, and image processing have become blurred. Furthermore, to discuss or even to list all the image processing software available would require many pages and would not be particularly useful to the nonspecialist. Therefore, we emphasize a representative set of image processing software packages that embody core capabilities in scientific image processing applications. Core capabilities, in our view, include the following:

Image utilities: These include display, manipulation, and file conversion. Images come in such a variety of formats that a package for converting images from one format to another is essential. Furthermore, basic display and manipulation (cropping, rotating, etc.) are essential for almost any image processing task. The ability to edit images using cut-and-paste, draw, and annotate operations is also useful in many cases. Image filtering and transformation: These are necessary capabilities for most scientific applications of image processing. Convolution, median filtering, FFTs, morphological operations, scaling, and other image functions form the core of many scientific image processing algorithms. Image compression: Anyone who works with images long enough will learn that they require a large amount of storage space. A number of standard image compression utilities are available for storing images in compressed form and for retrieving compressed images from image databases. Image analysis: Scientific image processing applications often have the goal of deriving information from an image. Simple image analysis tools such as edge detection and segmentation are powerful methods for gleaning important visual information. Programming and data analysis environment: While many image processing packages have a wide variety of functions, a whole new level of utility and flexibility arises when the image processing functions are built around a programming and/or data analysis environment. Programming environments allow for tailoring image processing techniques to the specific task, developing new algorithms, and interfacing image processing tasks with other scientific data analysis and numerical computational techniques. Other capabilities include higher-level object recognition and other computer vision tasks, visualization and rendering techniques, computed imaging such as medical image reconstruction, and morphing and other special effects of the digital darkroom and the film industry. These areas require highly specialized software and/or very specialized skills to apply the methods and are not likely to be part of the image processing world of the nonspecialist. The packages to be discussed here encompass as a group all of the core image processing capabilities mentioned above. Because these packages offer such a wide variety and mix of functions, they defy simple categorization. We have chosen to group the packages into three categories: general image utilities, specialized utilities, and programming/analysis environments. Keep in mind, however, that the distinctions among these groups is blurry at best. We have chosen to emphasize packages that are freely distributable and available on the Internet because these can be obtained and used with a minimum of expense and hassle. 58.1.1 General Image Utilities netpbm pbmplus is a set of tools that allows the user to convert to and from a large number of common image formats. The package has its own intermediate formats so that the conversion routines can be written to convert to or from one of these formats. The user can then convert to and from any combination of formats by going through one of the intermediate formats. Functions are also provided to convert from different color resolutions, such as from color to grayscale. Several other functions do basic image manipulation such as cropping, rotating, and smoothing. The source is available from ftp://ftp.wustl.edu/graphics/graphics/packages/netpbm/.

xv xv is an X11 utility that combines several important image handling functions. It can display images in a wide variety of display formats, including binary, 8-bit, and 24-bit. It allows the user to manipulate the colormap both in RGB and HSV space. It crops, resizes, smooths, rotates, detects edges, and produces other special effects. In addition, it reads and writes a large variety of image formats, so it can serve as a format conversion utility. Until recently, xv has been freely distributable. The latest version, however, is shareware and requires a small fee to become a registered user. The source is available from http://www.trilon.com/xv. NCSA Image NCSA Image is available in versions for the Mac, DOS, and Unix (X11). The Unix version is called ximage. ximage allows the user to display color images. It can also display the actual data in the form of a spreadsheet. A number of other display options are available. Like xv, it allows for manipulation of the colormap in a variety of ways. In addition, the user may display multiple images as an animated sequence, either from disk or server memory. The functionality of NCSA Image is augmented by other programs available from NCSA, including DataSlice for visualization tasks and Reformat for converting image formats. The source is available from NCSA by ftp at ftp://ftp.ncsa.uiuc.edu/visualization/image/. ImageMagick ImageMagick is an X11 package for display and interactive image manipulation. It reads and writes a large number of standard formats, does standard operations such as cropping and rotating as well as more specialized editing operations such as cutting, pasting, color filling, annotating, and drawing. Separate utilities are provided for grabbing images from a display, for converting, combining, resizing, blurring, adding borders, and doing many other operations. The source is available by ftp from ftp://ftp.x.org/contrib/applications/imagemagick/. NIH Image NIH Image is available only in a Macintosh version. However, the popularity of NIH Image among Mac users and the breadth of features justify inclusion of the package in this survey. It reads/writes a small number of image formats, acquires images using compatible frame grabbers, and displays. It allows image manipulation such as flipping, rotating, and resizing; and editing such as drawing and annotating. It has a number of built-in enhancement and filtering functions: contrast enhancement, smoothing, sharpening, median filtering, and convolution. It supports a number of analysis operations such as edge detection and measurement of area, mean, centroid, and perimeter of user-defined regions of interest. It also performs automated particle analysis. In addition, the user can animate a set of images. NIH Image has a Pascal-like macro capability and the ability to add precompiled plug-in modules. The source is available from NIH by ftp at ftp://zippy.nimh.nih.gov/pub/nih-image/. LaboImage LaboImage is an X11 package for mouse- and menu-driven interactive image processing. It reads/writes a special format as well as Sun raster format and displays grayscale and RGB and provides dithering. Basic filtering operations are possible, as well as enhancement tasks such as background subtraction and histogram equalization. It computes various measures such as histograms, image statistics, and image power. Region outlining and object counting can be done as well. Images can

be modified interactively at the pixel level, and an expert system is available for region segmentation. LaboImage has a macro capability for combining operations. LaboImage can be obtained from http://cuiwww.unige.ch/ftp/sgaico/research/geneve/vision/labo.html. Paint Shop Pro Paint Shop Pro is a Windows-based package for creating, displaying, and manipulating images. It has a large number of image editing features, including painting, photo retouching, and color enhancement. It reads and writes a large number of formats. It includes several standard image processing filters and geometrical transformations. It can be obtained from http://www.jasc.com/psp.html. It is shareware and costs $69. 58.1.2 Specialized Image Utilities Compression JPEG is a standard for image compression developed by the Joint Photographic Experts Group. Free, portable C code that implements JPEG compression and decompression has been developed by the Independent JPEG Group, a volunteer organization. It is available from ftp://ftp.uu.net/graphics/jpeg. The downloadable package contains source and documentation. The code converts between JPEG and several other common image formats. A lossless JPEG implementation can be obtained from ftp://ftp.cs.cornell.edu/pub/multimed/. A fractal image compression program is available from ftp://inls.ucsd.edu/pub/young-fractal/. The package contains source for both compression and decompression. A number of other fractal compression programs are also available and can be found in the sci.fractal FAQ at ftp://rtfm.mit.edu/pub/usenet/news.answers/sci/fractals-faq. JBIG is a standard for binary image compression developed by the Joint Binary Images Group. A JBIG coder/decoder can be obtained from ftp://nic.funet.fi/pub/graphics/misc/test-images/. MPEG is a standard for video/audio compression developed by the Moving Pictures Experts Group. A set of MPEG tools is available from ftp://mm-ftp.cs.berkeley.edu/pub/multimedia/mpeg/. These tools allow for encoding, decoding (playing), and analyzing the MPEG data. H.261 and H.263 are standards for video compression for videophone applications. An H.261 coder/decoder is available from ftp://havefun.stanford.edu/pub/p64/. An H.263 video coder/decoder is available from http://www.fou.telenor.no/brukere/dvc/h263 software/. Computer Vision Vista is an X11-based image processing environment specifically designed for computer vision applications. It allows a variety of display and manipulation options. It has a library that lets the user easily create applications with menus, mouse interaction, and display options. Vista defines a very flexible data format that represents a variety of images, collections of images, or other objects. It also has the ability to add new objects or new image attributes without changing existing software or data files. It does edge detection and linking, optical flow estimation and camera calibration, and viewing of images and edge vectors. Vista includes routines for common image processing operations such as convolution, FFTs, simple enhancement tasks, scaling, cropping, and rotating. Vista is available from http://www.cs.ubc.ca/nest/lci/vista/vista.html.

58.1.3 Programming/Analysis Environments Khoros Khoros is a comprehensive software development and data analysis environment. It allows the user to perform a large variety of image and signal processing and visualization tasks. A graphical programming environment called Cantata allows the user to construct programs visually using a data flowgraph approach. It has a user interface design tool with automatic code generation for writing customized applications. Software objects (programs) are accessible from the command line, from within Cantata, and in libraries. A large set of standard numerical and statistical algorithms are available within Khoros. Common image processing operations such as FFTs, convolution, median filtering, and morphological operators are available. In addition, a variety of image display and geometrical manipulation programs, animation, and colormap editing are included. Khoros has a very general data model that allows for images of up to five dimensions. Khoros is free-access software it is available for downloading free of charge but cannot be distributed without a license. It can be obtained from Khoral Research, Inc., at ftp://ftp.khoral.com/pub/. Note that the Khoros distribution is quite large and requires significant disk space. MATLAB MATLAB is a general numerical analysis and visualization environment. Matrices are the underlying data structure in MATLAB, and this structure lends itself well to image processing applications. All data in MATLAB is represented as double-precision, which makes the calculations more precise and interaction more convenient. However, it may also mean that MATLAB uses more memory and processing time than necessary. A large number of numerical algorithms and visualization options are available with the standard package. The Image Processing Toolbox provides a great deal of added functionality for image processing applications. It reads/writes several of the most common image formats; does convolution, FFTs, median filtering, histogram equalization, morphological operations, two-dimensional filter design, general nonlinear filtering, colormap manipulation, and basic geometrical manipulation. It also allows for a variety of display options, including surface warping and movies. MATLAB is an interactive environment, which makes interactive image processing and manipulation convenient. One can also add functionality by creating scripts or functions that use MATLAB s functions and other user-added functions. Additionally, one can add functions that have been written in C or Fortran. Conversely, C or Fortran programs can call MATLAB and MATLAB library functions. MATLAB is commercial software. More information on MATLAB and how to obtain it can be found through the homepage of The Mathworks, Inc., at http://www.mathworks.com/. PV-Wave PV-Wave is a general graphical/visualization and numerical analysis environment. It can handle images of arbitrary dimensionality 2-D, 3-D, and so on. The user can specify the data type of each data structure, which allows for flexibility but may be inconvenient for interactive work. PV-Wave contains a large collection of visualization and rendering options, including colormap manipulation, volume rendering, and animation. In addition, the IMSL library is available through PV-Wave. Basic image processing operations such as convolution, FFTs, median filtering, morphological operations, and contrast enhancement are included. Like MATLAB, PV-Wave is an interactive environment. One can create scripts or functions from the PV-Wave language to add functionality. It can also call C or Fortran functions. PV-Wave can be invoked from within C or Fortran too. PV-Wave is commercial software. More information on PV-Wave can be found through the homepage of Visual Numerics, Inc., at http://www.vni.com/.

58.2 Image Databases A huge number of image databases and archives are available on the Internet now, and more are continually being added. These databases serve various purposes. For the practicing engineer, the primary value of an image database is for developing, testing, evaluating, or comparing image processing and manipulation algorithms. Standard images provide a benchmark for comparing various algorithms. Furthermore, standard test images can be selected so that their characteristics are particularly suited to demonstrating the strengths and weaknesses of particular types of image processing techniques. In some areas of image processing no real standards exist, although de facto standards have arisen. In the discussion that follows, we provide pointers to some standard images, some de facto standards, and a few other databases that might provide images of value to algorithm work in image processing. We have deliberately steered away from images whose copyright is known to prohibit use for research purposes. However, some of the images in the list have certain copyright restrictions. Be sure to check any auxiliary information provided with the images before assuming that they are public domain. The images listed are in a variety of formats and may require conversion using one of the packages discussed previously such as netpbm. We list the databases according to two categories: (1) form and (2) content. By form, we mean that the images are organized according to the form of the image color, stereo, sequence, etc. By content, we mean that the images are grouped according to the image content faces, fingerprints, etc. 58.2.1 Images by Form Binary Images A set of standard CCITT fax test images has been made available for testing compression schemes. These are binary images that have come from scanning actual documents. They can be found at ftp://nic.funet.fi/pub/graphics/misc/test-images/ under ccitt[1-8].pbm.gz. Grayscale Images A collection of grayscale images can be obtained from ftp://ipl.rpi.edu/pub/image/still/canon/gray/. A compilation of de facto standard images can be found at http://www.sys.uea.ac.uk/research/resgroups/sip/images ftp/index.html. Note that the Lena image is copyrighted and should not be used in publications. Color Images A set of test images that were used by the JPEG committee in the development of the JPEG algorithm are available from ftp://ipl.rpi.edu/pub/image/still/jpeg/bgr/. These are 24-bit RGB images. Other 24-bit color images can be found at ftp://ipl.rpi.edu/pub/image/still/canon/bgr/. A set of miscellaneous images in JPEG and Kodak CD format can be found at http://www.kodak.com/digitalimages/samples/samples.shtml. Image Sequences Image sequences may be intended for study of computer vision applications or video coding. A huge set of sequences for computer vision applications are archived at http://www.ius.cs.cmu.edu/idb/. A set of sequences commonly used for video coding applications can be found at ftp://ipl.rpi.edu/pub/image/sequence/. Stereo Image Pairs

Stereo image pairs are available from http://www.ius.cs.cmu.edu/idb/. Texture Images A large set of texture images can be found at http://www-white.media.mit.edu/vismod/imagery/visiontexture/vistex.html. These images include textures from various angles and under different lighting conditions. Face Images The USENIX FACES database contains hundreds of face images in various formats. The database is archived at ftp://ftp.uu.net/published/usenix/faces/. Fingerprint Images Fingerprint images can be obtained from ftp://sequoyah.ncsl.nist.gov/pub/databases/data/. Medical Images A variety of medical images are available over the Internet. An excellent collection of CT, MRI, and cryosection images of the human body has been made available by the National Library of Medicine s The Visual Human Project. Samples of these images can be acquired at http://www.nlm.nih.gov/research/visible/visible human.html. A collection of over 3500 images that cover an entire human body is available via ftp and on tape by signing a license agreement. MRI and CT volume images are available from ftp://omicron.cs.unc.edu/pub/projects/softlab.v/chvrtd/. PET images and other modalities can be found in gopher://gopher.austin.unimelb.edu.au/11/images/petimages. Astronomical Images A collection of astronomical images can be found at https//www.univ-rennesl.fr/astro/astro.english.html. Hubble telescope imagery can be obtained from http://archive.stsci.edu/archive.html. Range Images Range images are available from http://www.eecs.wsu.edu/ĩrl/3ddb/rid/, along with a list of other sources of range imagery, and also from http://marathon.csee.usf.edu/range/database.html. The tools and databases discussed here should provide a convenient set of capabilities for the nonspecialist. The capabilities that are readily available are not static, however. Image processing will continue to become more and more mainstream, so we expect to see the development of image processing tools representing greater variety and sophistication. The advent of the World Wide Web will also stimulate further development and publishing of image databases on the Internet. Therefore, image processing capabilities will continue to grow and will be more readily available. The items discussed here are only a small sample of what will be available as time goes on.