Multispectral stereo acquisition using 2 RGB cameras and color filters: color and disparity accuracy

Transcription

1 Multispectral stereo acquisition using 2 RGB cameras and color filters: color and disparity accuracy (a) and Bernhard Hill (b) (a) Institute of Imaging and Computer Vision (b) Research Group Color and Image Processing RWTH Aachen University 18. Workshop Farbbildverarbeitung, 28. September 212 1

2 Introduction Motivation Choice of optical filters Accuracy of 6-channel camera Stereo system and disparity estimation Comparison with RGB system Color accuracy of multispectral stereo system Conclusions 2

3 Trans. Motivation Filters 6-channel camera Disparity estimation RGB comparison Color accuracy Conclusions Monochrome sensor 7 narrowband color filters 7-channel camera Grey 7 successive images for 7 color channels High spectral accuracy Long acquisition time 4 λ [nm] 7 3

4 Sens. Transm. Motivation Filters 6-channel camera Disparity estimation RGB comparison Color accuracy Conclusions 6-channel camera RGB sensor 2 broadband color filters 1 4 λ [nm] 7 Filter 1 Filter 2 RGB.8 4 λ [nm] 7 Acquisition 1 4

5 Sens. Sens. Transm. Motivation Filters 6-channel camera Disparity estimation RGB comparison Color accuracy Conclusions 6-channel camera RGB sensor 2 broadband color filters 1 4 λ [nm] 7 Filter 1 Filter 2 RGB.8 4 λ [nm] λ [nm] 7 5 Acquisition 1 Acquisition 2

6 Sens. Sens. Sens. Transm. Motivation Filters 6-channel camera Disparity estimation RGB comparison Color accuracy Conclusions 6-channel camera RGB sensor 2 broadband color filters 1 4 λ [nm] 7 Filter 1 Filter 2 2 successive images for 6 color channels Lower color accuracy But faster acquisition.8 RGB 4 λ [nm] 7.8 Sensitivity of the 6-channel camera:.8 4 λ [nm] λ [nm] 7

7 Sens. Sens. Motivation Filters 6-channel camera Disparity estimation RGB comparison Color accuracy Conclusions 6-channel stereo camera 2 RGB cameras A different broadband color filter in front of each one.8 4 λ [nm] 7 RGB 1 image for 6 color channels Simultaneous acquisition of depth information and multispectral data.8 4 λ [nm] 7 7

8 Data for the selection Values obtained with RGB camera simulated Halogen light source Natural objects Munsell color chips RGB Curves of standard filters from various manufacturers (GamColor, Lee, Edmund Optics, Schneider Kreuznach, Schott) 354 spectra from Vrhel dataset DuPont color chips 8

9 Simulation of camera values Camera sensit. RGB camera, filters, light source Vrhel data set Normally distributed noise with std. dev. 1% Relative noise 6 ideal camera responses Integration Quantization 8 bits 6 simulated output responses 9

10 Quantitative evaluation Using the sensitivities of the camera and the 6 simulated camera values: Wiener estimation to reconstruct the acquired spectra Color difference CIEDE2 calculated between the colors of reference Vrhel spectra and those of spectra reconstructed with the 6 camera values Filter pair selected: Low color difference Transmission reaching at least.4 1

11 Channel sens. Channel sens. Motivation Filters 6-channel camera Disparity estimation RGB comparison Color accuracy Conclusions Selected optical filters Results: GamColor filters 14 Broadway Rose and 57 Light Green Yellow,7 B1 B2 G2 GamColor R1,8 B1 B2 G1 G2 R1 Infitec R2 G1 R2 4 7 λ [nm] 4 7 λ [nm] Also dichroic Infitec filters, splitting each channel R, G and B in two halves. 11

12 Measurement of color accuracy RGB RGB camera 2 filters consecutively Color checker SG acquired Color difference CIEDE2 between reference and acquired colors Comparison of 2 filter sets (GamColor and Infitec) 12

13 Color accuracy of 6-channel camera Results with Infitec filters Color difference Acquired 7.23 Reference Accurate color acquisition possible Filter set Minimum Mean Median Maximum GamColor Infitec

14 Sens. Sens. Motivation Filters 6-channel camera Disparity estimation RGB comparison Color accuracy Conclusions Stereo 6-channel camera 2 cameras: different positions, different spectral information Similarity measure: mutual information Possible contrast inversions between color channels Illumination difference,8 4 λ [nm] 7 RGB,8 4 λ [nm] 7 14

15 Disparity estimation Block matching on rectified images Left image Disparity Right image 15

16 Multiscale approach Disparity estimation Subpixel accuracy required to merge both images for accurate color information: Only sought in last step Approximation with 2 nd order polynomial on 3*3 neighborhood around pixel position of maximum 16

17 Estimated disparity Disparity map pixel 15 pixels Reconstruction 17

18 Estimated disparity Disparity map pixel 15 pixels Reconstruction 18

19 Stereo-multispectral system Multispectral to RGB system RGB 19

20 Multispectral to RGB system Stereo-multispectral system Easily modified to RGB stereo system RGB Disparity estimated with multispectral system can be compared to values obtained with RGB system 2

21 Disparity from both systems Disparity from RGB pixel 15 pixels Disparity from multispectral 21

22 Comparison of disparity maps Difference Median value:.35 pixel pixel 5 pixels 22

23 Color accuracy of multispectral stereo system RGB 2 RGB cameras in a stereo configuration Different filter in front of each camera Color checker SG acquired Color difference CIEDE2 Comparison of 2 filter sets (GamColor and Infitec) 23

24 Color accuracy of multispectral stereo system Results with Infitec filters Filter set Minimum Mean Median Maximum GamColor Infitec

25 Color accuracy of multispectral stereo system Results with Infitec filters Color acquisition still accurate in stereo configuration Filter set Minimum Mean Median Maximum GamColor Infitec Filter set Minimum Mean Median Maximum GamColor Infitec Mono Stereo 25

26 Conclusions Advantages of 6-channel camera: faster acquisition of multispectral image. With stereo multispectral system: even depth information possible Optimal selection of color filters with simulation of camera responses Algorithm for disparity estimation for cameras acquiring different spectral information: results comparable with RGB stereo system 6-channel camera in mono or stereo configuration suitable for accurate color imaging 26

27 Channel sens. Channel sens. Motivation Filters 6-channel camera Disparity estimation RGB comparison Color accuracy Conclusions Outlook Dichroic filters: avoid blind spots in the wavelength e.g. by utilizing one filter in front of one camera and no filter in front of the other camera.8 B1 B2 G1 G2 R1 R2.9 B2 B1 G2 G1 R2 R1 4 7 λ [nm] 4 7 λ [nm] Acquisition of other objects whose reflectance distribution has a higher angular dependence 27

28 Thank you for your attention! 28