Recent advances in multispectral colour imaging

Transcription

1 Recent advances in multispectral colour imaging Prof. Jon Y. Hardeberg, The Norwegian Color Research Laboratory, Gjøvik University College NOBIM 2006, Oslo, Norway, September 8, 2006 Outline Introduction Colour and metamerism Why multispectral? Acquisition System design how many channels Spectral reconstruction Reproduction Printer modeling Halftoning Conclusions and outlooks 2 Colour: Light, surface, eye + 3 Light s Spectral Distribution, l(λ) Surface s Spectral Reflectance, r(λ) Eye s Spectral Sensitivities s i (λ), i =1,2,3 Radiance, l(λ) Reflectance, r(λ) Sensitivity, s i (λ) S M L Wavelength, λ Wavelength, λ Wavelength, λ 1

2 Metamerism Metamers: Different light spectra having the same colour 4 Violet flower under tungsten light PC monitor tuned to match Metamerism: A curse and a blessing! RIT/MCSL Metamerism Not only visual metamerism Camera metamerism Different spectral reflectances give same RGB triplet Camera sensitivity visual sensitivity Camera metamerism visual metamerism Camera confounds what humans discriminate 5 Multispectral colour imaging Imaging based on spectral reflectance rather than only colour >3 channels At some point in the imaging chain, the visual appearance of the images are important 6 The way to go Inspiration Multispectral colour image acquisition system 2

3 Outline Introduction Colour and metamerism Why multispectral? Acquisition System design how many channels Spectral reconstruction Reproduction Printer modeling Halftoning Conclusions and outlooks 7 Acquisition: principles Monochrome CCD camera with several color filters Similar to Maxwell s first color slide (1861) Rapid & high spatial resolution Two variants Passive type: filters in optical path Active type: filters in light path 8 K Observed Scene Filter Wheel Monochrome CCD Camera Multispectral image Acquisition: principles Camera with Liquid Crystal Tunable Filter (LCTF) Easy tuning of spectral sensitivities 9 450nm 550nm 650nm = K Observed Scene Tunable filter Monochrome CCD Camera Multispectral image 3

4 Acquisition: Examples CRISATEL Camera EU Project with ENST (Paris), National Gallery (London), Lumiere Technology (Paris) and other partners Linear CCD array with optical filters and dedicated illumination system Very high resolution 10 Images from Schmitt et al, Proc. AIC Colour 05, pp , 2005 Acquisition: Examples The Norwegian Color Research Gjøvik University College Made from off-the shelf components Monochrome CCD camera Apogee Alta U32 Cooled, low-noise, 14bit LCTF Varispec from CRI, Inc Optics 11 System design How many channels? Spectral sampling at 10nm suggests

5 System design How many channels? PCA analysis suggests <20 13 Jon Y. Hardeberg, On the spectral dimensionality of object colours, Proc. CGIV, , 2002 System design How many channels? Tradeoff between cost and spectral precision Must consider acquisition noise But if you really want a number: 9 14 D. Connah, A. Alsam, J.Y. Hardeberg, Multispectral imaging: How many sensors do we need? Journal of Imaging Science and Technology, 50(1), Jan/Feb 2006 Spectral reconstruction Does the camera output give direct information about spectral reflectance? No typical sensitivity functions are far from nice sampling functions nm 450nm 500nm 550nm 600nm 650nm 700nm J.Y. Hardeberg, Filter Selection for Multispectral Color Image Acquisition, Journal of Imaging Science and Technology, 48(2): ,

6 Spectral reconstruction Common approach based on the inversion of a spectral model of the acquisition system 16 max c = l ( λ) r( λ) o( λ) φ ( λ) a( λ) dλ + ε k λ λmin R k k Spectral reconstruction Active research area Principal Eigenvector method J.Y. Hardeberg, F. Schmitt, H. Brettel, Multispectral Color Image Capture using a Liquid Crystal Tunable Filter, Optical Engineering, 41(10): , Comparison of spectral reconstruction methods Spectral reconstruction by polynomial regression Spectral reconstruction using convex bases D. Connah, J. Y.Hardeberg, S. Westland, Comparison of Spectral Reconstruction Methods for Multispectral Imaging, Proc. ICIP, 2004 D. Connah, J.Y. Hardeberg. Spectral recovery using polynomial models, SPIE Proc. 5667, 65-75, 2005 A. Alsam and D. Connah, Recovering Natural Reflectances with Convexity, Proc. AIC Colour 05, pp , 2005 Outline Introduction Colour and metamerism Why multispectral? Acquisition System design how many channels Spectral reconstruction Reproduction Printer modeling Halftoning Conclusions and outlooks 18 6

7 Spectral reproduction Reproduction of the spectral reflectance of the original scene The perfect reproduction intent Theoretically identical to original scene under all conditions Multi-channel inkjet system 19 J.Y. Hardeberg and J. Gerhardt, Characterization of an eight colorant inkjet system for spectral color reproduction, Proc. CGIV 2004, pp , 2004 Spectral reproduction Spectral printer model Typically Yule-Nielsen Modified Spectral Neugebauer Model Non-linear weighted sum of the Neugebauer Primaries (combinations of one or more primaries) 20 Inverse model needed Conversion from desired spectral reflectance to required colorant combination A. Alsam, J. Gerhardt, J.Y. Hardeberg, Inversion of the spectral Neugebauer printer model, Proc. AIC Colour 05, pp , 2005 Spectral reproduction Challenges Total ink limit Soaked paper with too much ink Spectral gamut limitations What to do if a spectral reflectance is not possible to reproduce with your system? 21 A.M. Bakke, I. Farup, and J.Y. Hardeberg, Multispectral Gamut Mapping and Visualisation: A first attempt, SPIE Proc. 5667, pp ,

8 Spectral reproduction Printers are binary devices Halftoning is needed Common approach: error diffusion Blue noise 22 Spectral reproduction Independent halftoning of several ink layers creates moiré problems Visually disturbing noise Blue-noise property lost Solution: Spectral vector error diffusion 23 J. Gerhardt, J.Y. Hardeberg, Spectral colour reproduction by vector error diffusion, Proc. CGIV, pp , 2006 J. Gerhardt, J.Y. Hardeberg, Reducing the error spreading in Spectral Vector Error Diffusion, To be presented at Electronic Imaging Symposium, San José, 2007 Spectral Vector Error Diffusion 24 spectral image λ (nm) x[] n + + e[ n] Select the primary as output value. Distance calculation filter - y[] n + binary image n colorants Error diffusion Floyd s error filter Error calculation 8

9 Results Tested on a set of 100 randomly chosen patches on a simulated printer Reproduced by The 100 ED patches under D65 channel-independent error diffusion Estimated spectral reflectance of patches Attempt to reproduce patches with these spectral reflectances by sved Evaluate differences in colour and spectral reflectance Evaluate visually 25 Results SED sved Results Spectral RMS error Average Maximum Colour difference E ab under D65 illuminant Average 1.68 Maximum 4.78 The 100 ED patches under D65 The 100 VED patches under D

10 Spectral Vector Error Diffusion 28 SED sved Update on work-in-progress: Conclusions and outlooks Multispectral colour imaging can overcome the problems of conventional metameric colour imaging Challenges Ease of use One-shot multispectral camera Spectral reproduction Challenges Reduce complexity of vector error diffusion calculations Spectral gamut limitations 29 Questions? 30 Further information: The cited papers My book: Acquisition and reproduction of color images: colorimetric and multispectral approaches Acknowledgements Ali Alsam, Arne Magnus Bakke, David Connah, Jérémie Gerhardt, Alamin Mansouri, 10