IMAGESERVER: A SYSTEM FOR A DISTRIBUTED IMAGE PROCESSING APPLICATION BASED ON JAVA ADVANCED IMAGING Xinwen YU 1,2, xinwenyul@affrc.go.jp - Seishi Ninomiya 1, snino@narc.affrc.go.jp - Matthew Laurenson 1, matthewl@narc.affrc.go.jp - Zuorui Shen 3, ipmist@cau.edu.cn 1 Dept. Information and Technology, National Agriculture Research Center, Tsukuba, Japan 2 Dept. Biodiversity conservation, Southwest Forestry College, Kunming,650224, China 3 Dept. Entomology, China Agricultural University, Beijing, 100094, China Abstract: Recently, imaging has become an increasingly important subject in many fields, as well as an important field in networking computing since the release of Java Advanced Imaging (JAI) by Sun Microsystems. An ImageServer was developed using Java RMI and JAI technology. This ImageServer provides common functions for image processing which are very convenient for distributed imaging. In addition to the JAI image type, for example, PlanarImage, we also provide an output image of serialized BufferedImage type for client side. This means that the client side will be able to perform advanced image processing without installation of JAI on their machine, since JAI is an extension of Standard Java edition. An applet that demonstrates the performance of ImageServer was also developed using our ImageBeans, that are helpful in the development of client applications. Keywords: Java, distributed imaging, JAI, image processing 1. Introduction Imaging is becoming an increasingly important research subject in many agriculture fields. Image analysis methods, such as pattern recognition, characteristic measuring, shape evaluation and analysis, have been extensively used both in agricultural research and application. Researchers developed many useful algorithms for agricultural application, but these implementations of algorithms are only available to researchers themselves. As researchers in different fields have different research interests, they are unlikely to want to implement a wide range of image analysis algorithms. Therefore, the implementations of image analysis algorithms are distributed and dispersed in different areas, and are difficult for other people to make use of them. However, using a distributed system, both data and algorithms can be distributed (Morrow, 1999), a distributed imaging architecture will be a good solution for the above-mentioned situation. Distributed imaging is still quite new. There are some discussion rooms on the web (Palaniappan, 1998; Franz, Goller, 1998), but most focus on management of distributed image data (Takahiro, 1998; University of Alberta, 2000; Rosengarten, 2002), and are geo-science related. To our knowledge, there is no similar information available for distributed imaging in agriculture related areas to-date. The objective of this paper is to develop an ImageServer which will provide consistent access to different image processing and analysis methods for users developing imagingbased distributed computing applications, so that these applications may be run anywhere in the world. ImageServer is one module of a distributed imaging framework that will be 150
developed by our collaborators. Under this framework, the developer can create portable imaging softwares and researchers can make use of many implemented algorithms. As they will not have to implement various algorithms repeatedly, they can therefore concentrate on their objectives. 2. Approach As the name implies, ImageServer is a server application written in Java programming language. A distributed architecture is used in ImageServer to provide consistent access to imaging methods. Fig. 1 shows the structure of ImageServer. ImageServer consists of a control program and a base of image operators. The control program is responsible for analysing the request from client applications and dispatch requests to corresponding image operators. Image operators return processing results to the control program and then the control program returns the result to the client application. We also developed several small Java components called JavaBean, which will be very useful for connection to ImageServer (termed the ImageRequestBean in our project) and parameter collection (ImageBeans) necessary for image processing. The client application uses ImageRequestBean to make request for image processing methods when receiving any messages from ImageBeans, and then sends the request to ImageServer. ImageRequestBean, extended from ServerRequestBean which is a class of MetBroker (Laurenson et al., 2001), communicates with the ImageServer core, preferably using Java Remote Method Invocation (Java RMI), which is a Java implementation of distributed computation. Since Java RMI might be blocked by a firewall, we also developed an ImageServer servlet, which provides HTTP access to ImageServer for client application. The servlet itself communicates with ImageServer using Java RMI. Client Application A IMG Bean HTTP HTTP ImageServer servlet Java RMI Client Application B IMG Bean Java RMI ImageServer Image Operators Fig. 1. ImageServer structure Image operators, the core function module of ImageServer, are developed based on Java Advanced Imaging (JAI, Sun Microsystem). Because JAI is an extension package of Java 2 Standard Edition (J2SE), J2SE does not provide JAI in the library. Users must install a copy of JAI in their computing environment to make use of the functionality of JAI. According to our experience, many users are reluctant to have an installation of JAI. Furthermore, JAI is to some extent complicated, as it introduces many new concepts and 151
image types (e.g. TiledImage, PanarImage, ImageOp) that are difficult to understand and utilize in a short time. We therefore developed a set of wrapper classes to simplify the usage of JAI. These wrapper classes are the image operators in ImageServer. By using these wrapper classes, users do not need to know what JAI is, they can use the image class, BufferedImage, available in J2SE library, but still have the powerful functionality of JAI. We also added some useful operators we developed ourselves with the same form of these wrapper classes. In our structure, an image object will be transferred between client and server application, and Java demands object serialization when transferred via the network. but BufferedImage, which is the main image object in our project, is not serialized, we also developed two serialized version of BufferedImage, namely, PackedImage packing BufferedImage to byte array, and UnPackedImage decomposing BufferedImage to serialized objects. At present, ImageServer provides more than 16 operators, as shown in Table1, and we are adding still more operators to ImageServer. 3. Application We developed a simple viewing applet that allowed us to verify the performance of ImageServer. Because Java is a platform independent language, applications written in Java can be run on different platforms as long as JRE (Java Runtime Environment) is installed, and Java applet can be run on a web browser without any installation procedures, so the applet is helpful to demonstrate the functionality of ImageServer. The applet was developed using our ImageBeans, including an image display canvas (ImagePanelBean) and a control panel (composed of a collection of operation beans). We also provide some sample images listed on the right side of the applet. Users can click on the image name, and the applet will load the image from the server. Users can send processing requests by selecting or adjusting controllable options of ImageBeans, such as sliders or combo boxes, and ImageRequestBean will automatically make and send a request to ImageServer and then retrieve the processing result. Fig2 shows the screen-captured frames of the applet. Fig.2A.shows the layout of the applet, displaying an image without any application of image processing methods. The control panel is composed of 11 tabbed panes; each provides the functionality of controlling the behavior of the image operators. Fig. 2B shows the processing result of HIS conversionconverting the image from RGB color space to HIS (Hue, Intensity, Saturation) color space. Notice this page is a Chinese version. The internationalization of the applet is implemented by using the portable software ResourceServer (Laurenson, et al, 2002). Fig.2C displays a contrast enhancement result using the Window/Level operator. The yellow rectangle area on the tabbed pane is an assistant tool that user can click on a position in the rectangle or move the small red mark to a appropriate place to get the satisfactory result. Fig.2D displays the gradient and magnitude operation result by using the Prewitt method; there are 3 other methods available, Sobel, Freid-Chen and Roberts operators. Fig.2E shows a thresholding result. Users can click on the check box to display/remove a HistogramBean, which shows the histograms of pixel values of RGB channels, and is therefore helpful for selecting threshold values, using 3 sliders on the Threshold pane. Fig.2F shows the binarization result, using a user-specified thresholding value. When using this method, the HistogramBean is also available by clicking on the check box, to help the user select the thresholding value. There are 5 other adaptive binarization methods, available in the combo box. 152
Table 1. Operators available in ImageServer Operators Description AddConst adds a constant to every pixel of its corresponding band of the source A spatial operation that computes each output sample by multiplying Blur and sharp elements of a kernel with the samples surrounding a particular source sample. A spatial operation that computes each output sample by multiplying Convolve elements of a kernel with the samples surrounding a particular source sample. 16 kernels available. Window/Level A lookup table operation enables contrast enhancement. Gradient An edge detector which computes the magnitude of the image gradient vector in two orthogonal directions. 4 implementations available. RGB2Gray Changes to gray level image by band selecting or combining. HIS Converts to HIS image or gray level image by channel selecting of Hue, Intensity, and Saturation. A non-linear filter which is useful for removing isolated lines or pixels Median while preserving the overall appearance of an image. The filter is implemented by moving a mask over the image. 15 masks available. Scale Translates and resizes an image Maps the pixel values of an image from one range to another range by Rescale multiplying each pixel value by one of a set of constants and then adding another constant to the result of the multiplication. Rotate Rotates an image about a given point by a given angle, specified in radians Shear Shears an image either horizontally or vertically. Transpose Flips image in various ways. Threshold Maps all the pixels of the image whose value falls within a specified range to a specified constant. Binarize Thresholding a gray level image to produce a bi-level image. Morphological operators Several basic morphological operators were provided 153
Fig. 2. Screen shot of the applet showing processing results of several operators. 154
4. Discussion The architecture of ImageServer follows an approach developed with the weather data broker MetBroker(Laurenson et al, 2001) and the digital elevation model broker DEMBroker (Laurenson, et al, 2002), except that ImageServer does not connect to other databases. This architecture enables client applications to get consistent access to ImageServer and to request advanced image processing functionalities. We provided full documents to help potential client application developers understand the architecture and inner structure of ImageServer, including instructions for client application development and the documentation of all classes in the format JavaDoc format (Sun Microsystems). We will also open the source code of ImageServer in the near future. ImageServer communicates with the client application by Java RMI, but only Java applications are able to communicate using Java RMI. Although we developed a servlet that provides HTTP access for client applications, the interface is still difficult for the development of a client application. However, an XML-based protocol called Simple Object Access Protocol (SOAP) (Box, et al, 2000) provides similar functionality to Java RMI and is language-independent. Many programming languages, e.g. VB, Delphi and even Excel, have toolkits allowing them to communicate using SOAP. Therefore, we will develop a SOAP interface for ImageServer. This means that developers using language other than that of Java programming will also be able to develop client applications for ImageServer. Furthermore, with the SOAP implementation, we can easily provide distributed imaging Web Services, which use only HTTP protocol and are capable of bridging any operating system, hardware platform, or programming language, just as the web(newcomer, 2002). At present, the transferring speed of request object between client application and ImageServer is limited when using dial-up connection, because the request object enclosed an image object in it, but it is endurable when using Ethernet. We believe this will be improved as the band-width increases. 5. Acknowledgment This research is supported by the Japan Society for the Promotion of Science (JSPS). 155
REFERENCE Box, D. D. Ehnebuske, G. Kakivaya,, A., 2000, Simple Object Access Protocol (SOAP) 1.1, World Wide Web Consortium, URL:http://www.w3.org/TR/SOAP. Laurenson, M., A. Otuka and S. Ninomiya. 2001. Developing agricutlureal models using MetBroker mediation software. Journal of Agricultural Meteology. 58(1):1-9. Laurenson, M; Ushamani, A.G. and Ninomiya, S. 2002: ResourceServer A Localization System for Distributed Computing Applications. APAN Agriculture Working Group Meeting, Aug 26-28, Shanghai. (Remote presentation via video conference) http://agri-wg.jp.apan.net/2002shanghai/index.html. Laurenson, M, S. Ninomiya, 2002, DEMBroker Consistent Access for software Applications to Digital Elevation Models. Proceeding of AFITA2002, Beijing. Morrow, P., 1999, http://www.surfnetters.nl/ivana/reports/iee_distributed_imaging.html Newcomer, E., 2002, Understanding web services, XML, WSDL, SOAP, AND UDDI. Addison-Wesley. Niederl F., A. Goller, 1998, Method Execution on a Distributed Image Processing Back-end. PDP, Madrid. URL: http://www.icg.tu-graz.ac.at/~niederl/publications/pdp98 Palaniappan, K, Hasler, H., 1998, http://meru.cecs.missouri.edu/mvl/iiss/articles_backup/internet2_diss.htm Rosengarten, H., 2002, http://www.ziimaging.com/news/otherdocs/rosengar.pdf Sun Microsystem, http://java.sun.com/products/java-media/jai/ Takahiro E., et al., 1998, http://www.gisdevelopment.net/aars/acrs/1998/ts11/ts11005pf.htm University of Alberta, 2000, http://db.uwaterloo.ca/~ddbms/publications/multimedia/disimareport/disimafinalreport.htm 156