A New Concept in High Resolution Internet Image Browsing

Transcription

1 A New Concept in High Resolution Internet Image Browsing Denis Pitzalis, Ruven Pillay, Christian Lahanier C2RMF Centre de Recherche et de Restauration des Musées de France Palais du Louvre, Porte des Lions, 14, quai François Mitterand, Paris Cedex 1, France Abstract Various new imaging techniques have been used at the C2RMF, in order to digitize paintings and objects. These techniques are used for extensive and detailed non-destructive analysis of works of art in different ways and include multispectral imaging, panoramic viewing of objects, 3D laser scanning of painting surfaces and 3D photogrammetric modelling of objects. These techniques are valuable tools in the restoration and conservation arsenal, allowing us to perform regular and accurate monitoring of works of art, to measure their conservation state and compare these with previous analyses. These techniques, however, result in huge quantities of data, which needs to be visualized and disseminated in several ways. The resulting images must also be accessible to various partners around the world via the Internet, but the data needs to be protected and the vast data sizes involved require careful handling and management. This paper describes several new developments that have been made at the C2RMF by our team in order to make extremely high resolution images available on line. We will examine as a case study the use, the manipulation and the share of high resolution colorimetric images between members of the museum research community. The system that has been developed is a light-weight client-server architecture that efficiently streams image data to the client, allowing the user to quickly view very large images even over a slow Internet connection. These developments have been released as Open Source software in the IIPImage project. Keywords: IIP; Internet imaging; IIPImage; client-server; remote viewing 1 Introduction Advances in digital imaging have resulted in massively increased image resolutions and bit depths as well as in new techniques such as 3D imaging and topographical scanning. These new imaging techniques produce enormous quantities of data that need to be adequately managed, handled and exploited. Apart from the sheer volume of data, other advanced features such as colorimetric accuracy, high dynamic ranges (e.g. 16 bit images) and multi-channel output must also be considered. This problem has been particularly acute at the research and restoration centre at the Louvre Museum (the C2RMF - Centre de Recherche et Restauration des Musées de France) where a diverse range of imaging technologies have been applied in order to assist curators, restorers and researchers. In order to view such data, standard visualisation systems that simply transfer and load an entire image into memory are very inefficient. Not only are the scans large, but also colour calibrated, meaning that colour management must also be handled. The use of these advanced technologies has limited value if their results cannot be easily used or widely exploited. A key technical challenge is, therefore, how to efficiently disseminate this high quality data to people both internally and externally of the research centre. 2 The IIPImage Viewing System In order to handle this diversity of extremely large images, the Open Source IIPImage[1] system is used. IIPImage is a client-server system designed for the remote viewing of very high resolution images across the Internet. Its architecture gives it the peculiarity to be usable even over a slow dial up connection. The server can be launched as a plug-in that can work with a wide variety of web servers such as: Apache, Lighttpd or MyServer or any other FCGI-enabled http server. Images can be viewed via a feature-rich Java client, a Javascript client embedded within a web page or as JPEG image dynamically generated at the requested size and resolution by the server [2,3].

2 292 Pitzalis, Denis; Pillay, Ruven; Lahanier, Christian The system evolved from a prototype used in several European projects to view high resolution images of works of art across the Internet and has been greatly extended and restructured to handle the new imaging techniques described later in this paper. 2.1 Working Principle The system is fast and able to work remotely because the client only needs to download the portion of the whole image that is visible on the screen of the user at their current viewing resolution (Fig. 1). This means that it is also bandwidth and memory efficient as users do not need to be able to store or handle massive images on their local machine. Only the required parts of the whole image at the desired resolution need be sent. Figure 1: The real size of the image is much more bigger than the screen size The image tiles at the requested resolution are extracted from the source file by the server, dynamically compressed with JPEG (or lossless if desired) and sent to the client. The compression level used can be controlled by the client to optimize the transmission depending on the network environment (Fig. 2). This technique makes it possible to view extremely large images of several gigabytes in size in real-time over the Internet. Figure 2: Time-Line for an IIPImage client/server action 2.2 Data Storage Format In order for the system to be as efficient as possible, the images are stored in a multi-resolution tiled format, (Fig. 3) which allows the server to extract regions of the full image at different resolutions very quickly with

3 A New Concept in High-Resolution Internet Image Browsing 293 little processing overhead. Although a proprietary file format such as Flash Pix could have been used or a new one especially developed, it was decided to keep the format as open and interoperable as possible. The TIFF format is widely used and is flexible enough to permit this kind of structuring and encoding. Multiple resolutions can be stored within a single file in a pyramid-like structure using the standard TIFF multi-page functionality and each resolution can be tiled. A pyramidal image without compression uses only 33% extra space. However, the tiles can be optionally compressed using either a lossless compression such as Deflate or LZW, or a lossy compression such as JPEG. If space is an issue, JPEG compression is able to reduce the image file size by at least a factor of ten without a significant loss in visual quality [4]. When there are multiple views (e.g. multiple wavelengths or view angles etc), the different images are simply kept as separate files. Figure 3: Tiled Pyramidal format used by IIPImage 2.3 Colour Management and Bit Depth The classical 8-bit RGB colour space most commonly used today is insufficient for the richness of data acquired by high resolution cameras such as the CRISATEL [5] camera. Colour images are, therefore, stored in the device-independent CIELab colour space, which is able to describe colours that are outside the gamut of RGB. Furthermore, higher dynamic ranges (16-bit) are required to store the full dynamic range of imaging detail that would be invisible or saturated using only 8-bits. The IIPImage system is flexible enough to handle standard 8-bit RGB images, higher dynamic range 16bit images as well as colour spaces such as CIELab and generic multispectral images. Standard monitors are only able, however, to display 8-bits of information per pixel per channel in RGB colour space. In order to visualize 16-bit or CIELab images, the raw data needs to be first processed. In the case of 16-bit images, a contrast control allows the user to navigate within the extra, normally hidden data. For CIELab images, a dynamic conversion is performed by the client into the calibrated RGB space of the monitor. Eventually, it will be possible to use ICC profiles to calibrate the image to the user's monitor. 2.4 Internet Imaging Protocol The same group that in 1997 created the FlashPix image format, developed, at the same time, the IIP (Internet Image Protocol), a protocol aimed at Internet-based imaging and optimized for the FlashPix format. The tiled

4 294 Pitzalis, Denis; Pillay, Ruven; Lahanier, Christian pyramidal TIFF format used by us is similar to the FlashPix standard, but is a widely-used open format and is fully compatible with the protocol. The IIP protocol also allows for a certain amount of customisation which has allowed us to enhance certain features and add panoramic viewing, multispectral and surface topographic height handling to it. 3 Supported Image Types 3.1 Multispectral Viewing and Dynamic Spectral Reconstruction One of the advanced technologies used at the C2RMF is the multispectral CRISATEL camera system. Multispectral imaging enables us to accurately evaluate the surface physical state of a work of art. Having an accurate spectral record of the painting allows conservators to locate areas that may be invisible to the naked eye (Fig. 4) or that have been restored or retouched long ago[6]. By making regular scans of works of art, the gradual changes in the physical condition can be monitored and damage addressed quickly. The characterisation of materials used, the evaluation of the conservation state, the localization of previous restoration interventions and continuous monitoring are among the most important topics in the field of study and conservation of artworks. Figure 4: Lady Praying by Anonymous, Louvre Museum Painting Dep., Paris, inv. RF Blend between colour and infrared image [Invisible detail of a previous composition in the upper left corner] The CRISATEL multispectral imaging system is capable of producing scans at a resolution of 20,000 x 12,000 pixels each with a bit depth of 12 bits/pixel. Thirteen of these scans are taken at different wavelengths between 400nm and 1000nm (Fig. 5). This results in around 6 Gbytes of data per painting. After calibration and postprocessing of that data, the painting can be viewed either as a reconstructed colour image under a specified illuminant (eg D65) or the user can navigate through the various monochrome images representing the view at a certain wavelength. Arbitrary pairs of these images can be superimposed and blended together to allow subtle changes to be more clearly discerned (Fig. 5).This technique can also be used with other associated images, such as X-ray or fluorescence images.

5 A New Concept in High-Resolution Internet Image Browsing 295 Figure 5: Naked Woman in Landscape by Pierre Auguste Renoir (1883), Orangerie Museum, Paris, inv. RF [Spectral reconstruction from the 13 filter multispectral camera] Another important tool is the ability to dynamically reconstruct the spectra at any point on the image. It is possible to perform spectral reconstruction using the data from the CRISATEL camera with a high level of accuracy [7]. By comparing these reconstructed spectra to known pigment spectra, it is possible to perform rudimentary non-destructive pigment identification. 3.2 Panoramic Object Visualisation Another important imaging technique at the C2RMF arsenal is colorimetric 3D panoramic imaging. 3D imaging of objects and paintings offers a powerful new analytical tool to conservators, curators and art historians. Highresolution 3D image data contains information that can be used for display, comparison, measurement and ana lytical applications. A large collection of statues and objects has been scanned using the ACOHIR system [8]. These consist of a sequence of images of an object on a rotating turntable taken with a standard RGB camera. The turntable has a precision of about 1 micro degree and usually 36 images (one each 10 degrees) are taken at different angles. These images are colour calibrated using a MacBeth ColorChecker DC reference. The visua lisation system allows the user to navigate around the object and zoom in and link to areas of interest (Fig. 6). Figure 6: Nike between young people, Nicosthenes, Louvre Museum, DAGER dep., INV F102 [Two views of a Panoramic Object Visualisation]

6 296 Pitzalis, Denis; Pillay, Ruven; Lahanier, Christian 3.3 3D Surface Scan 3D laser scanning is based on the direct acquisition by multiple low-power lasers [9]. The system scans a small (less than 100µm diameter) white laser spot from three (RGB) laser sources over the complete surface of the painting. A triangulation based detection system simultaneously records the spatial measurements (x,y,z) and the colour reflectance (RGB) from the spot. The maximum lateral resolution is of 50µm with a depth resolution of 10µm. A unique feature of this technology is the ability to examine the surfaces and roughness of paintings as well as the overall shape of canvas and panel paintings. The shape data recorded by the scanner originates from the immediate surface of the paint layer, under the varnish, rather than from the varnish surface. This results in a detailed high-resolution recording of the surface relief or 3D structure of the paint layer from brush stroke details as well as from craquelure formations due to ageing (figures 7). In order to visualise this kind of data, the IIPImage system has to handle the extra surface data as well as the visual image. The extra data consists of height data for each pixel and also the vector describing the perpendicular to the plane of the surface. As with the visual images, this data is stored within tiled multiresolution TIFF files. With this extra data, the viewer can dynamically reconstruct the surface topography and can allow the user to change the direction of the light source (Fig. 8). This allows the user to more clearly distinguish surface features and, in essence, dynamically simulates raking light photography. The colour image is also able to be superimposed giving the user a unique view of the work of art. Figure 7: Surface RGB laser scan example [detail of the painting Lady Praying [ 4 Conclusions State of the art imaging techniques have been used at the C2RMF to non-destructively analyse and record the state of both paintings and objects in unprecedented levels of detail. Multispectral imaging provides ways to accurately measure surface reflectance, colour and perform some pigment and material identification. Panoramic imaging has been used to produce high resolution colorimetric images of statues and objects from many angles. While, laser scanning has been applied to the surface of paintings to measure with great accuracy the paint-layer roughness and surface relief. The use of such techniques has proved to be of great value and will help ensure that works of art are maintained in optimal condition. The data produced by these techniques has

7 A New Concept in High-Resolution Internet Image Browsing 297 required a new and flexible remote viewing system capable of handling ultra-high resolution images, colorimetric accuracy, extended bit depths, multi-channel data, spectral reconstruction, panoramic views and surface rendering. The IIPImage remote viewing system provides a flexible and efficient means to disseminate the results of these new imaging techniques and provides a powerful tool for advanced research and study of works of art. The system has been of great benefit to curators, researchers and restorers internally at the C2RMF. The efficient network capabilities also allow these images to be accessible externally and can potentially open up these fields of art and research to a far wider public around the world. We thank the European Union for their support given to the ACOHIR, ARTISTE, SCULPTEUR and CRISATEL projects. References [1] The IIPImage High Resolution Remote Image Viewing System. URL [2] MARTINEZ, K; CUPITT, J.; PERRY, S. High resolution Colorimetric Image Browsing on the Web. Computer Networks and ISDN Systems, Amsterdam : Elsevier Science Publishers, 1998, vol. 30, p [3] MARTINEZ, K.; CUPITT, J.; PERRY, S. Object browsing using the Internet Imaging Protocol. Computer Networks and ISDN Systems, Amsterdam : North-Holland Publishing, 2000, vol. 33, p [4] SAUNDERS, D.; CUPITT, J.; PILLAY, R.; MARTINEZ, K. Maintaining color accuracy in images transferred across the Internet. Color Imaging: Vision and Technology. Edit. by L. W. MacDonald, M.R. Luo : John Wiley, 1999, p [5] RIBÈS, A.; BRETTEL, H.; SCHMITT, F.; LIANG, H.; CUPITT, J.; SAUNDERS, D. Color and Spectral Imaging with the CRISATEL Aquisition System. The Digital Photography Conference. Rochester, USA, 2003, p [6] LAHANIER, C.; PILLAY, R.; BREUCKMANN, B.; COLANTONI, P.; PITZALIS, D. Multispectral Imaging and 3D Modelling of Canvas Painting, EVA Florence, Italy : Pitagora, 2005, p [7] RIBÈS, A., SCHMITT, F.; BRETTEL, H. Calibration and Spectral Reconstruction for an Art Painting Multispectral Acquisition System. CGIV Second European Conference on Color in Graphics, Imaging and Vision. Aachen, Germany, 2004, p [8] The ACOHIR Project. URL [9] TAYLOR, J.; BERALDIN, J. A.; GODIN, G.; COUMOYER, L.; RIOUX, M.; DOMEY, J. NRC 3D Imaging Technology for Museum and Heritage. Proceedings of The First International Workshop on 3D Virtual Heritage. Geneva, Switzerland, 2002.