Parameter Estimation in Bayesian High-Resolution Image Reconstruction With Multisensors

Transcription

1 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL 12, NO 12, DECEMBER Parameter Estimation in Bayesian High-Resolution Image Reconstruction With Multisensors Rafael Molina, Member, IEEE, Miguel Vega, Javier Abad, and Aggelos K Katsaggelos, Fellow, IEEE Abstract In this paper, we consider the estimation of the unknown parameters for the problem of reconstructing a high-resolution image from multiple undersampled, shifted, degraded frames with subpixel displacement errors We derive mathematical expressions for the iterative calculation of the maximum likelihood estimate of the unknown parameters given the low resolution observed images These iterative procedures require the manipulation of block-semi circulant (BSC) matrices, that is block matrices with circulant blocks We show how these BSC matrices can be easily manipulated in order to calculate the unknown parameters Finally the proposed method is tested on real and synthetic images Index Terms Bayesian methods, high-resolution image reconstruction, parameter estimation I INTRODUCTION HIGH-RESOLUTION images can, in some cases, be obtained directly from high precision optics and charge coupled devices (CCDs) However, due to hardware and cost limitations, imaging systems often provide us with only multiple low resolution images and in addition, there is a lower limit as to how small each CCD can be, due to the presence of shot noise [1] and the fact that the associated signal to noise ratio (SNR) is proportional to the size of the detector [2] Low resolution images are common in many imaging applications, such as remote sensing, surveillance and astronomy For example, the optical imaging camera on the Hubble Space Telescope (HST), the Wide Field Planetary Camera 2 (WFPC2) is composed of four separated pixel CCD cameras, one of which, the planetary camera (PC), has an image scale of 0046 /pixel, while the other three arranged in a chevron around the PC have a scale of /pixel (see [3] for details) Over the last two decades research has been devoted to the problem of reconstructing a high-resolution image from multiple undersampled, shifted, degraded frames with subpixel displacement errors Since the early work by Tsai and Huang [4], researchers, primarily within the engineering community, have focused on formulating the high resolution problem as a reconstruction (see [5] for a comprehensive review) or a recognition Manuscript received October 15, 2002; revised April 10, 2003 This work was supported by the Comisión Nacional de Ciencia y Tecnología under Contract TIC The associate editor coordinating the review of this manuscript and approving it for publication was Dr Mario A T Figueiredo R Molina and J Abad are with the Departamento de Ciencias de la Computación e IA Universidad de Granada, Granada, Spain ( rms@decsaiugres) M Vega is with the Departamento de Lenguajes y Sistemas Informáticos Universidad de Granada, Granada, Spain A K Katsaggelos is with the Department of Electrical and Computer Engineering, Northwestern University, Evanston, IL USA Digital Object Identifier /TIP one (see [6] [8], see also [9]) The astronomical community has also being working on the high resolution problem and has made available the Drizzle method to reconstruct high resolution images (see [3]) However, as reported in [5], not much work has been devoted to the efficient calculation of the reconstruction or to the estimation of the associated parameters Bose and Boo [10] use a block semi-circulant (BSC) matrix decomposition in order to calculate the maximum a posteriori (MAP) reconstruction, Chan et al ([11] [13]) and Nguyen ([14] [16]) use preconditioning, wavelets, as well as BSC matrix decomposition The efficient calculation of the MAP reconstruction is also addressed by Ng et al ([17], [18]) and Elad and Hel-Or [19] To our knowledge only the works by Bose et al [20], Nguyen [15], [16], [21], [22] and to some extend [13] and [23] [25] address the problem of parameter estimation Furthermore, in those works the same parameter is assumed for all the low resolution images, although in the case of [20] the proposed method can be extended to different parameter for low resolution images (see [26]) In this paper we use the general framework for frequency domain multi-channel signal processing developed by Katsaggelos et al in [27] and Banham et al in [28] (a formulation that was also obtained later by Bose and Boo [10] for the high resolution problem) to tackle the parameter estimation in high resolution problems With the use of BSC matrices we show that all the matrix calculations involved in the parameter maximum likelihood estimation can be performed in the Fourier domain The proposed approach can be used to assign the same parameter to all low resolution images or make them image dependent We also show that the results are extensions of maximum likelihood estimation for single channel restoration problems The rest of the paper is organized as follows The problem formulation is described in Section II The high resolution prior image model and the process to obtain the low resolution images from the high resolution one is described in Section III The application of the Bayesian paradigm to calculate the maximum a posteriori high resolution image and to estimate the parameters is described in Section IV Experimental results are described in Section V Finally, Section VI concludes the paper The paper also contains two appendices on the application of EM algorithms to high resolution problems and the matrix manipulations needed to calculate the MAP estimate and the parameters II PROBLEM FORMULATION Consider a sensor array with sensors (a sensor is, for instance, a CCD camera), where each sensor has /03$ IEEE

2 1656 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL 12, NO 12, DECEMBER 2003 Fig 1 Correspondence between high and low resolution pixels pixels and the size of each sensing element is Our aim is to reconstruct an high resolution image, where and, from lowresolution observed images Fig 1 shows a visual description of the problem formulation for and Note that in order for our goal make sense we need to assume that the original high-resolution scene is bandlimited to wavenumbers and along the horizontal and vertical directions, respectively In this case the original high-resolution image can be reconstructed with and samples along the horizontal and vertical directions, respectively (see [29]) To maintain the aspect ratio of the reconstructed image we consider the case where, for simplicity we also assume that is an even number Each observed undersampled image is a shifted, downsampled version of the high-resolution image In the ideal case, the low resolution sensors are shifted with respect to each other by a value proportional to (note that if the sensors are shifted by values proportional to the high-resolution image reconstruction problem becomes singular) However, in practice there can be small perturbations around those ideal locations (see [30] for a formulation without perturbations) Thus, the horizontal and vertical displacements and of the sensor with respect to the reference sensor are given by (see Fig 1) where and denote respectively the normalized horizontal and vertical displacement errors We assume that and with The normalized horizontal and vertical displacement may be assumed to be known (see [10] and [18] for details) An approach where the displacements are assumed unknown and are estimated simultaneously with the high-resolution image is presented in [23] [25] III IMAGE AND DEGRADATION MODELS Let be the high resolution image and the observed low resolution image from the sensor, Our goal is to (1) reconstruct from using the Bayesian paradigm The first step with this paradigm is the definition of a prior distribution, a probability distribution over high resolution images, It is here where we incorporate information on the expected structure of It is also necessary to specify the probability distribution of the observed low resolution image if were the true high resolution image These image and high to low resolution degradation models depend on the unknown parameters and, that have to be estimated In order to apply the Bayesian paradigm to this problem we define next our image and high to low degradation models A Image Model Our prior knowledge about the smoothness of the original high resolution image makes it possible to model the distribution of by a simultaneous autoregression (SAR) [31] Thus, where the entries of the matrix,, are equal to 1 if cells and are spatial neighbors (pixels at distance one) and zero otherwise and is just less than 025 in order to model smoothness and result in a positive definite matrix This model is characterized by where Note that the parameter measures the smoothness of the high resolution image Assuming a toroidal edge correction, the eigenvalues of the matrix are,, Then the SAR distribution is given by where, From the regularization point of view, the SAR model imposes conditions on the second differences of the image B Model for Obtaining the Low-Resolution Observed Images The process to obtain the observed low resolution image by the sensor,, from can be modeled as follows (see Fig 1 for the correspondence between the high and low resolution image pixels) First, is obtained This image represents a blurred version of the original high-resolution image, according to where is an matrix and may have different forms In [10], [18], is associated to the blurring function (2) (3) (4) (5) (6)

3 MOLINA et al: PARAMETER ESTIMATION IN BAYESIAN HIGH-RESOLUTION IMAGE RECONSTRUCTION WITH MULTISENSORS 1657 with From now on, given an column vector, we will denote by the column vector given by (7) (17) where and In [32] has the form otherwise (8) note that in this case,,,, the normalized horizontal and vertical displacement errors in (1) satisfy, and, Let and now be the 1-D downsampling matrices defined by (9) (10) where is the identity matrix, is the unit vector whose nonzero element is in the position, denotes the Kronecker product operator and the transpose operator Then for each sensor the discrete low-resolution observed image can be written as IV BAYESIAN ANALYSIS The steps we follow in this paper to estimate the parameters, and, and the original image are as follows: 1) Step I: Estimation of the Parameters: and are first selected as where (18) (19) 2) Step II: Estimation of the Original Image: Once the parameters have been estimated, the estimation of the original image,, is selected as the image satisfying where has been defined in (5) (11) which produces (20) (12) (21) denotes the 2D downsampling matrix and is modeled as independent white noise with variance If denotes the matrix where (22) then we have (13) (14) where We denote by the sum of the upsampled low-resolution images, that is, Then (15) Note that we are using maximum likelihood for estimating the parameter and maximum a posteriori (MAP) for estimating Furthermore, although steps I and II are separated, the iterative scheme to be proposed performs both estimations simultaneously The estimation process we are using could be performed within the so called hierarchical Bayesian approach (see [33]) by including priors on the unknown parameter and vector However, the possibility of incorporating additional knowledge on them by means of gamma or other distributions will not be discussed here (see [33] and [34]) Let us examine the estimation process in detail Fixing and and expanding the function (16) around,wehave (23) where and (24)

4 1658 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL 12, NO 12, DECEMBER 2003 Therefore the same the same (25) We then have Fig 2 Original high resolution image We now differentiate with respect to and so as to find the conditions which are satisfied at the maxima We have Note that refers here to a value and should not be confused with that refers to a vector B) Since (26) (29) (27) The following algorithm is proposed for the simultaneous estimation of the parameters and the high resolution image by using (26) and (27) we have that and (30) Algorithm 1 1) Choose and 2) Compute using (21) with and 3) For a) Calculate and by substituting and in the left hand side of (26) and (27) b) Compute using (21) with and 4) Go to 3 until is less than a prescribed bound A number of comments are now made regarding these equations A) If the same parameter is used for some low resolution observations, (26) and (27) become easier to solve In particular if all the noise variances are assumed to be the same, that is,, (27) becomes (28) (31) So we see that the maximum likelihood estimate (mle), satisfies, (32) which means that a fraction of the observations are used to calculate the misfit to the prior model, and another proportion is used for the misfit to the high to low resolution process (, ) C) Algorithm 1 is, in fact an EM-algorithm [35] with complete data and incomplete data Steps 3a and 3b iteratively increase (see Appendix I for details) D) We note that in order to find the MAP estimate we need to invert and in order to estimate the parameters we have to calculate for the values of and In Appendix II it is shown how these calculations can be performed in an efficient way in the frequency domain by utilizing properties of SBC matrices [27]

5 MOLINA et al: PARAMETER ESTIMATION IN BAYESIAN HIGH-RESOLUTION IMAGE RECONSTRUCTION WITH MULTISENSORS 1659 (a) (b) (c) (d) (e) (f) Fig 3 Experiment 1, 30 db case (a) Zero-order hold for g, (b) best bilinear interpolated image, (c) initial high resolution image, (d) estimated high resolution image with the proposed method, (e) estimated high resolution image with GCV, and (f) estimated high resolution image with L-curve E) Finally, we also note that a very interesting low to high resolution problem occurs when low resolution observations are available We are currently working on this problem, and preliminary results can be found in [36] and [37] V EXPERIMENTAL RESULTS A number of simulations have been performed with the proposed algorithm over a set of images We present results with

6 1660 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL 12, NO 12, DECEMBER 2003 TABLE I SUMMARY OF RESULTS FOR THE THREE DIFFERENT LOW RESOLUTION IMAGE SETS EACH COLUMN SHOWS STATISTICS FOR THE TEN SIMULATIONS two images evaluating the performance of the proposed method under different noise conditions and comparing it with other existing approaches for estimating the parameters For all the experiments, the criterion was used for terminating the iteration We set in all our experiments, where has been defined in (15) The performance of the restoration algorithms was evaluated by measuring the signal to noise ratio (SNR) improvement denoted by and defined by (33) where and are the original and estimated high resolution images, respectively We compared our proposed algorithm with the Generalized Cross-Validation (GCV) [16] and L-curve [20] methods Both methods estimate the high resolution image using (34) where is selected using the GCV [16] or L-curve [20] methods, the respective values of will be denoted by and See [38] for a description of GCV and other methods for choosing the regularization parameter in image restoration problems Note that the L-curve and GCV methods estimate a single parameter This corresponds to the low to high resolution problem where the same noise variance is assumed for all observed low resolution images (see (11)) Note also that selecting the high resolution image according to (34) is the same as finding the solution of (20) with and A first experiment was devoted to test the performance of the proposed method under different noise conditions on the low resolution images The image,, shown in Fig 2 was blurred using the blurring function defined in (8) with, which produced the image Then was downsampled with obtaining the sixteen low resolution images (35) Gaussian noise with the same variance, that is,, was added to each low resolution image to obtain three sets of sixteen low resolution images The noise variance for each set of images was set to 16129, 1444 and 144, respectively, thus obtaining an SNR of approximately 10 db, 20 db and 30 db in the low resolution images of each image set Fig 3 shows reconstructions for the 30 db SNR case Fig 3(a) depicts the zero-order hold upsampled image (see (11)) Each low resolution image in each set was bilinearly interpolated to obtain a image The best one in terms of is shown in Fig 3(b) The starting image,, is shown in Fig 3(c) Fig 3(d) depicts the estimated high resolution image by the proposed algorithm when sixteen noise parameters are considered The high resolution image obtained by the proposed method with the same noise parameter for the sixteen low resolution observations, see (28), is almost identical and it is not shown here Figs 3(e) and 3(f) show the images obtained by estimating the high resolution image using the parameter values selected by the GCV and L-curve methods, respectively From

7 MOLINA et al: PARAMETER ESTIMATION IN BAYESIAN HIGH-RESOLUTION IMAGE RECONSTRUCTION WITH MULTISENSORS 1661 TABLE II STATISTICS OF THE ESTIMATED NOISE VARIANCES AND REGULARIZATION PARAMETER FOR THE TEN SIMULATIONS OF THE THREE LOW RESOLUTION IMAGE SETS WITH ONE NOISE PARAMETER these figures, it is clear that the proposed method provides the visually best reconstructions In order to validate the proposed parameter estimation algorithm on a number of simulations, ten realizations of the noise were generated for each noise level Table I shows the mean values of for the different methods under consideration From this table it is clear that the proposed method improves the SNR even in the case of severe noise although higher improvements are obtained as the noise decreases The improvement in SNR obtained by the proposed algorithm is greater than the ones obtained by the GCV and L-curve procedures, resulting in less noisy results as previously commented In Table I we have included the obtained by the proposed method when using only one and sixteen different noise parameters The results are almost identical in both cases, so validating the estimation process Table I also shows the standard deviation of the ten obtained by our proposed method with one and sixteen noise parameters and the number of iterations needed The estimated image model parameters and their standard deviations, in brackets, for the three sets of images with one noise parameter were equal to, and, respectively The corresponding estimated mean noise parameters for the low resolution images are presented in Table II together with their standard deviations Examining these tables we conclude that the proposed method produces accurate estimates of the low resolution noise variances The results of the estimation of sixteen noise parameters, one for each low resolution image, were also very close to their real values In order to compare the proposed method with the GCV and L-curve approaches we have included in Table II the equivalent value of obtained by our method when estimating only one noise parameter We can see from this table that both GCV and L-curve obtain a smaller regularization parameter and therefore noisier reconstructions Fig 4 shows the evolution of the for the three low resolution image sets Note that most of the improvement was obtained in the first few iterations Each iteration took 155 seconds on a Pentium IV 1700 A second experiment was performed to test the proposed algorithm when different noise variances are used on the low resolution observations The original image in Fig 2 was blurred and downsampled as in the previous experiment and Gaussian noise was added to each low resolution image to obtain a set Fig 4 SNR improvement evolution with the number of iterations for the three low resolution image sets TABLE III SNR AND NOISE VARIANCES FOR LOW RESOLUTION IMAGES OF THE SECOND EXPERIMENT of sixteen low resolution images with different noise characteristics, with SNRs of 10 db, 20 db or 30 db randomly selected (see Table III) Again, we generated ten realizations of the noise Fig 5(a) depicts the zero-order hold upsampled image The bilinear interpolation is shown in Fig 5(b) Fig 5(c) and 5(d) depict the results obtained by the proposed algorithm estimating one and sixteen noise parameters, respectively, and Fig 5(e) and 5(f) show the results with the GCV and L-curve estimation procedures, respectively From these images it is clear that the proposed method outperforms all other reported methods, for both cases of one and sixteen noise parameters estimation The best visual result is obtained when one noise parameter is estimated for each low resolution image The obtained by the proposed algorithm estimating one and sixteen noise parameters and, also, by the GCV and L-curve methods are shown in Table IV We can see that even when the proposed algorithm estimates only one noise parameter the results are better than those obtained by the GCV and

8 1662 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL 12, NO 12, DECEMBER 2003 (a) (b) (c) (d) (e) (f) Fig 5 (a) Zero-order hold for g, (b) best bilinear interpolated image, (c) estimated high resolution image (one noise parameter), (d) estimated high resolution image (sixteen noise parameters), (e) estimated high resolution image with GCV, and (f) estimated high resolution image with L-curve L-curve methods The improvement in SNR obtained by the proposed algorithm with sixteen noise parameters clearly represents the best result The number of iterations required by the proposed algorithm to satisfy the convergence criterion are also shown in this table Table IV also shows the estimated values of by the proposed method when estimating one noise parameter, and the GCV and L-curve From these figures it is clear that both GCV and L-curve methods obtain significantly lower than the proposed method The mean value of the image model parameter estimated by the proposed algorithm is, with standard deviation 31, when estimating one noise parameter, and

9 MOLINA et al: PARAMETER ESTIMATION IN BAYESIAN HIGH-RESOLUTION IMAGE RECONSTRUCTION WITH MULTISENSORS 1663 (a) (b) (c) (d) Fig 6 (a) Original high resolution image, (b) zero order hold of low resolution observed image g, (c) Initial high resolution image, and (d) estimated high resolution image TABLE IV 1 FOR THE SECOND EXPERIMENT TABLE V MEANS OF THE ESTIMATED NOISE PARAMETERS FOR THE SECOND EXPERIMENT,SEE TABLE III IN BRACKETS THEIR STANDARD DEVIATIONS FOR THE TEN REALIZATIONS, with standard deviation 10, when estimating sixteen noise parameters Table V shows the mean noise variance parameters, and their corresponding standard deviations in brackets, estimated by the proposed algorithm This table shows that accurate estimates are obtained by the proposed model although small variance values are slightly overestimated when their corresponding low resolution image is close in terms of shifts to other low resolution images with larger noise variances A value of, with standard deviation 061, was obtained when estimating one noise parameter This value is close to the mean of the noise variances in the low resolution images In a third experiment we also tested the proposed method on low resolution images with different noise variances First, the set of low resolution images was obtained by blurring and downsampling the original image depicted in Fig 6(a) following the same procedure as in the previous experiments Then, to each low resolution image, Gaussian noise was added to obtain at random degraded images with SNR 20, 30, or 40 db The noise variances utilized are shown in Table VI Fig 6(b) depicts zero-order hold of the observed low resolution image We ran the proposed method starting with the initial image (Fig 6(c)) obtaining at convergence the estimated high resolution image shown in Fig 6(d) For the resulting image the SNR improvement was 1163 db and the estimated image model parameter The estimated variances of the degrading noise are shown in Table VII

10 1664 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL 12, NO 12, DECEMBER 2003 TABLE VI SNR AND NOISE VARIANCES FOR LOW RESOLUTION IMAGES IN FIG (6b) TABLE VII ESTIMATED NOISE VARIANCES FOR HIGH RESOLUTION IMAGE IN FIG (6d) Fig 7(a) depicts versus number of iterations Note that the vertical axis is plotted in logarithm scale From this plot it is clear that the method converges fast, needing only a few iterations to obtain a good estimation of the image, validating, this way, the theoretical results Fig 7(b) shows the signal to noise ratio improvement,, versus the number of iterations It is clear that increases monotonically and that most of the improvement is obtained in the first four iterations VI CONCLUSIONS A new method to estimate the unknown parameters in a high resolution image reconstruction problem has been proposed Using BSC matrices we have shown that all the matrix calculations involved in the parameter maximum likelihood estimation can be performed in the Fourier domain The approach followed can be used to assign the same parameter to all low resolution image parameters or to make them image dependent We have also shown that the results are extensions of maximum likelihood estimation for single channel restoration problems The proposed method has been validated experimentally APPENDIX I EM ALGORITHM APPLIED TO THE RESOLUTION PROBLEM In this section, we show how the EM-algorithm can be used to estimatetheunknownparametersinthelowtohighreconstruction problem under consideration Let be the vector containing the unknown parameters Note that in our high resolution problem we are dealing with an exponential family since we have (36) (a) where (37) (38) and is a scalar function depending only on the complete data We note that the expectation of the sufficient statistic is given by (39) (b) Fig 7 (a) Convergence criterion versus iterations plot and (b) SNR improvement versus iterations plot for the image shown in Fig 6(d) By comparing the real and estimated degradation model parameters (see Tables VI and VII, respectively) we conclude that the proposed model produces good estimates for all parameters although, again, small variance values are overestimated when their corresponding low resolution images are close in terms of shifts to other low resolution images with larger noise variances and so we have and Furthermore (40) (41) (42)

11 MOLINA et al: PARAMETER ESTIMATION IN BAYESIAN HIGH-RESOLUTION IMAGE RECONSTRUCTION WITH MULTISENSORS 1665 and (43) The EM-algorithm for this family requires the maximization with respect to (see [39]) lexicographically ordered components,, If in (46) we write, with, and, we have (44) whose unique solution is provided by steps 3a and 3b of algorithm 1 APPENDIX II CALCULATING THE MAP AND PARAMETERS In this appendix we show how with known to efficiently calculate the corresponding MAP,, that is, how to obtain the solution of and so Let us consider the convolution process given by (45) or (50) (46) (51) By lexicographically ordering the signals in (46) and we have (47) where and are vectors and is a convolution matrix of size We will assume that is a circular convolution operator (see however [18] for the issues when using circulant approximations in high-resolution problems) We examine how to calculate as the sum of convolutions Each convolution will involve a circulant convolution matrix of size and a vector of size Proposition 1: Let be the circular convolution process defined in (47) and the downsampling matrix defined in (12), then (48) where is the circular convolution matrix defined by (49) Proof: First, note that for an column vector,, is the column vector with It is interesting to note that, Let be the matrix defined by (52) Note that is a permutation matrix, see [40], and therefore Since and, we have from the above proposition [see (48)] Property 1: and so we have the following: Property 2: See (54) at the top of the next page Furthermore, from property 1 we have the following Property 3: (53) (55)

12 1666 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL 12, NO 12, DECEMBER 2003 (54) Let us now proceed to solve (45) First, we rewrite this equation as (56) Then, we use property 2 on, properties 1 and 3 on each term of the form and property 3 on each term of the form to obtain that (56) can be written as (57) where and are circular convolution matrices In order to solve the system in (57) we only need to apply Fourier transform to each equation of the form (58) in that system, since in the Fourier domain and become diagonal matrices Then, our problem becomes the solution of systems of equations each one of size It is interesting to note that a similar decomposition approach to the one used here is followed in [28] and [10] to solve restoration and high-resolution problems Finally in order to estimate the parameters we have to calculate and To do so we only need to take into account that and then apply Fourier transforms as we did to calculate the MAP REFERENCES [1] K Aizawa, T Komatsu, and T Saito, A scheme for acquiring very high resolution images using multiple cameras, in Proc IEEE Conf Audio, Speech, Signal Processing, vol 3, 1992, pp [2] H Stark and P Oskoui, High-resolution image recovery from imageplane arrays, using convex projections, J Opt Soc A, vol 6, no 11, pp , 1989 [3] A S Frutcher and R N Hook, Drizzle: a method for the linear reconstruction of undersampled images, Pub Astron Soc Pacific, vol 114, pp , 2002 [4] R Y Tsai and T S Huang, Multiframe image restoration and registration, in Advances in Computer Vision and Image Processing, R Y Tsai and T S Huang, Eds New York: JAI, 1984, vol 1, pp [5] S Borman and R Stevenson, Spatial resolution enhancement of lowresolution image sequences A comprehensive review with directions for future research, in Laboratory for Image and Signal Analysis Notre Dame, IN: Tech Rep, Univ Notre Dame, 1998 [6] S Baker and T Kanade, Limits on super-resolution and how to break them, in Proc IEEE Conf Computer Vision and Pattern Recognition, vol 2, 2000, pp [7], Super-resolution: reconstruction or recognition?, in IEEE-EURASIP Workshop on Nonlinear Signal and Image Processing, 2001 [8] W T Freeman, E C Pasztor, and O T Carmichael, Learning low-level vision, Int J Comput Vis, vol 40, no 1, pp 25 47, 2000 [9] D P Capel and A Zisserman, Super-resolution from multiple views using learnt image models, in Proc IEEE Conf Computer Vision and Pattern Recognition, vol 2, 2001, pp [10] N K Bose and K J Boo, High-resolution image reconstruction with multisensors, Int J Imag Syst Technol, vol 9, pp , 1998 [11] R H Chan, T F Chan, M K Ng, W C Tang, and C K Wong, Preconditioned iterative methods for high-resolution image reconstruction with multisensors, Proc SPIE, vol 3461, pp , 1998 [12] R H Chan, T F Chan, L Shen, and S Zuowei, A wavelet method for high-resolution image reconstruction with displacement errors, in Proc Int Symp Intelligent Multimedia, Video and Speech Processing, 2001, pp [13] R H Chan, T F Chan, L X Shen, and Z W Shen, Wavelet Algorithms for High-Resolution Image Reconstruction, Tech Rep, Dept Math, Chinese Univ Hong Kong, 2001 [14] N Nguyen and P Milanfar, A wavelet-based interpolation-restoration method for superresolution, Circuits, Syst, Signal Process, vol 19, pp , 2000 [15] N Nguyen, Numerical Algorithms for Superresolution, PhD dissertation, Stanford Univ, Stanford, CA, 2001 [16] N Nguyen, P Milanfar, and G Golub, A computationally efficient superresolution image reconstruction algorithm, IEEE Trans Image Processing, vol 10, pp , 2001 [17] M K Ng, R H Chan, and T F Chan, Cosine transform preconditioners for high resolution image reconstruction, Linear Algebra Applicat, vol 316, pp , 2000 [18] M K Ng and A M Yip, A fast MAP algorithm for high-resolution image reconstruction with multisensors, Multidimen Syst Signal Process, vol 12, pp , 2001 [19] M Elad and Y Hel-Or, A fast super-resolution reconstruction algorithm for pure translational motion and common space invariant blur, IEEE Trans Image Processing, vol 10, pp , 2001 [20] N K Bose, S Lertrattanapanich, and J Koo, Advances in superresolution using L-curve, in Proc IEEE Int Symp Circuits Systems, vol 2, 2001, pp [21] N Nguyen, P Milanfar, and G Golub, Blind superresolution with generalized cross-validation using Gauss-type quadrature rules, in Proc 33rd Asilomar Conf Signals, Systems, Computers, vol 2, 1999, pp

13 MOLINA et al: PARAMETER ESTIMATION IN BAYESIAN HIGH-RESOLUTION IMAGE RECONSTRUCTION WITH MULTISENSORS 1667 [22] N Nguyen, P Milanfar, and G H Golub, Efficient generalized cross-validation with applications to parametric image restoration and resolution enhancement, IEEE Trans Image Processing, vol 10, pp , 2001 [23] B C Tom, N P Galatsanos, and A K Katsaggelos, Reconstruction of a high resolution image from multiple low resolution images, in Super- Resolution Imaging, S Chaudhuri, Ed Norwell, MA: Kluwer, 2001, ch 4, pp [24] B C Tom, A K Katsaggelos, and N P Galatsanos, Reconstruction of a high-resolution image from registration and restoration of low-resolution images, in Proc IEEE Int Conf Image Processing, vol 3, 1994, pp [25] B C Tom and A K Katsaggelos, Reconstruction of a high-resolution image by simultaneous registration, restoration, and interpolation of low-resolution images, in Proc IEEE Int Conf Image Processing, vol 2, 1995, pp [26] M Belge, M E Kilmer, and E L Miller, Simultaneous multiple regularization parameter selection by means of the L-hypersurface with applications to linear inverse problems posed in the wavelet domain, Proc SPIE, 1998 [27] A K Katsaggelos, K T Lay, and N P Galatsanos, A general framework for frequency domain multi-channel signal processing, IEEE Trans Image Processing, vol 2, pp , 1993 [28] M R Banham, N P Galatsanos, H L Gonzalez, and A K Katsaggelos, Multichannel restoration of single channel images using a waveletbased subband decomposition, IEEE Trans Image Processing, vol 3, pp , 1994 [29] A K Jain, Fundamentals of Digital Image Processing Englewood Cliffs, NJ: Prentice-Hall, 1989 [30] R R Schultz and R L Stevenson, Extraction of high resolution frames from video sequences, IEEE Trans Image Processing, vol 5, pp , 1996 [31] B D Ripley, Spatial Statistics New York: Wiley, 1981 [32] D Rajan and S Chaudhuri, An MRF-based approach to generation of super-resolution images from blurred observations, J Math Imag Vis, vol 16, pp 5 153, 2002 [33] R Molina, A K Katsaggelos, and J Mateos, Bayesian and regularization methods for hyperparameter estimation in image restoration, IEEE Trans Image Processing, vol 8, pp , 1999 [34] N P Galatsanos, V Z Mesarovic, R Molina, A K Katsaggelos, and J Mateos, Hyperparameter estimation in image restoration problems with partially-known blurs, Opt Eng, vol 41, pp , 2002 [35] A P Dempster, N M Laird, and D B Rubin, Maximum likelihood from incomplete data, J R Statist Soc B, vol 39, pp 1 38, 1972 [36] J Mateos, R Molina, and A K Katsaggelos, Bayesian high resolution image reconstruction with incomplete multisensor low resolution systems, in Proc IEEE Int Conf Acoustic, Speech, Signal Processing, 2003 [37] J Mateos, M Vega, R Molina, and A K Katsaggelos, Bayesian image estimation from an incomplete set of blurred, undersampled low resolution images, in Proc 1st Iberian Conf Pattern Recognition and Image Analysis (IbPRIA2003), 2003 [38] N P Galatsanos and A K Katsaggelos, Methods for choosing the regularization parameter and estimating the noise variance in image restoration and their relation, IEEE Trans Image Processing, vol 1, pp , 1992 [39] G J McLachlan and T Krishnan, The EM Algorithm and Extensions New York: Wiley, 1997 [40] G H Golub and C F Van Loan, Matrix Computations Baltimore, MD: Johns Hopkins Univ Press, 1991 Rafael Molina (M 88) was born in 1957 He received the degree in mathematics (statistics) in 1979 and the PhD degree in optimal design in linear models in 1983 He became Professor of computer science and artificial intelligence at the University of Granada, Granada, Spain, in 2000 His areas of research interest are image restoration (applications to astronomy and medicine), parameter estimation in image restoration, low to high image and video, and blind deconvolution Dr Molina is a member of SPIE, the Royal Statistical Society, and the Asociación Española de Reconocimiento de Formas y Análisis de Imágenes (AERFAI) Miguel Vega was born 1956 in Spain He received the Bachelor Physics degree from the Universidad de Granada, Granada, Spain, in 1979 and the PhD degree from the Universidad de Granada in 1984 He was a Staff Member ( ) and Director ( ) of the Computing Center facility of the Universidad de Granada He has been a Lecturer since 1987 in the ETS Ing Informática of the Universidad de Granada (Departemento de Lenguajes y Sistemas Informáticos) He teaches software engineering His research focuses on image processing (multichannel and superresolution image reconstruction) He has collaborated at several projects from the Spanish Research Council Javier Abad was born in Granada, Spain, in 1968 He received his degree in computer science from the University of Granada in 1991, and recently received the Ph D degree in wavelet image restoration with applications to super-resolution problems He has been Assistant Professor since 1991 at the Department of Computer Science and Artificial Intelligence of the University of Granada His research interests are image and video processing, including multichannel image restoration, image and video recovery and compression, and super-resolution from (compressed) stills and video sequences Aggelos K Katsaggelos (F 98) received the Diploma degree in electrical and mechanical engineering from the Aristotelian University of Thessaloniki, Greece, in 1979 and the MS and PhD degrees both in electrical engineering from the Georgia Institute of Technology, Atlanta, in 1981 and 1985, respectively In 1985, he joined the Department of Electrical and Computer Engineering at Northwestern University, Evanston, IL, where he is currently Professor, holding the Ameritech Chair of Information Technology He is also the Director of the Motorola Center for Communications During the academic year, he was an Assistant Professor at Department of Electrical Engineering and Computer Science, Polytechnic University, Brooklyn, NY Dr Katsaggelos is a Fellow of the IEEE, an Ameritech Fellow, a member of the Associate Staff, Department of Medicine, at Evanston Hospital, and a member of SPIE He is a member of the Publication Board of the IEEE PROCEEDINGS, the IEEE Technical Committees on Visual Signal Processing and Communications, and Multimedia Signal Processing, Editorial Board Member of Academic Press, Marcel Dekker: Signal Processing Series, Applied Signal Processing, and Computer Journal He has served as editor-in-chief of the IEEE Signal Processing Magazine ( ), a member of the Publication Boards of the IEEE Signal Processing Society, the IEEE TAB Magazine Committee, an Associate editor for the IEEE TRANSACTIONS ON SIGNAL PROCESSING ( ), an area editor for the journal Graphical Models and Image Processing ( ), a member of the Steering Committees of the IEEE TRANSACTIONS ON IMAGE PROCESSING ( ) and the IEEE TRANSACTIONS ON MEDICAL IMAGING ( ), a member of the IEEE Technical Committee on Image and Multidimensional Signal Processing ( ), and a member of the Board of Governors of the IEEE Signal Processing Society ( ) He is the editor of Digital Image Restoration (New York: Springer-Verlag, Heidelberg, 1991), coauthor of Rate Distortion Based Video Compression (Norwell, MA: Kluwer, 1997), and coeditor of Recovery Techniques for Image and Video Compression and Transmission (Norwell, MA: Kluwer, 1998) He is the coinventor of eight international patents, and the recipient of the IEEE Third Millennium Medal (2000), the IEEE Signal Processing Society Meritorious Service Award (2001), and an IEEE Signal Processing Society Best Paper Award (2001)