ReconstructionofaHighResolutionImagefromMultipleDegraded DepartmentofElectricalEngineeringandComputerScience Mis-RegisteredLowResolutionImages McCormickSchoolofEngineeringandAppliedScience Email:briant@eecs.nwu.edu,aggk@eecs.nwu.edu BrianC.TomandAggelosK.Katsaggelos Evanston,IL60208-3118 NorthwesternUniversity anddisplacedbysub-pixelshiftswithrespecttoareferenceframe.therefore,theproblemcanbedividedinto imagesofsuchhighresolutionbyjustusinghardware(highprecisionopticsandchargecoupleddevices(ccds)). Instead,imageprocessingapproachescanbeusedtoconstructahighresolutionimagefrommultiple,degraded,low threesub-problems:registration(estimatingtheshifts),restoration,andinterpolation.noneofthemethodswhich appearedintheliteraturesolvetheregistrationandrestorationsub-problemssimultaneously.thisissub-optimal, resolutionimages.itisassumedthatthelowresolutionimageshavebeensubsampled(thusintroducingaliasing) Inapplicationsthatdemandhighlydetailedimages,itisoftennotfeasibleorsometimespossibletoacquire Abstract sincetheregistrationandrestorationstepsareinter-dependent.basedonpreviousrestorationandidentication conditionalmean(restoredimages)simultaneously.inaddition,theregistrationandrestorationsub-problemsare workusingtheexpectation-maximization(em)algorithm,theproposedapproachestimatesthesub-pixelshiftsand castinamulti-channelframeworktotakeadvantageofthecross-channelinformation.experimentalresultsshow Keywords:Expectation-Maximizationalgorithm,sub-pixelregistration,multi-channelrestoration,multiresolution thevalidityofthismethod. tobe50m[1].thisboundisduetothepresenceofshotnoisethatisunavoidableintheimagingsystem.in[14] bydecreasingthepixelsizeontheccd,thereisalowerboundprescribinghowsmallthepixelscanbe,considered greaterthanthatofhdtv)cannotbeaccommodatedbycurrentccdtechnology.whileresolutioncanbeincreased havingtoolowansnrtobeuseful. itisarguedthattheassociatedsignal-to-noiseratio(snr)ofadetectorisproportionaltothesizeofthedetector,if apoissonnoiseprocessisassumed.therefore,decreasingthesizeofthepixelspastacertainpointwillyieldimages Inthenearfuture,thedemandforveryhighdenition(VHD)resolutions(suchasthatof35mmlm,whichis 1.INTRODUCTION theblursarenotknown,thentheycanbeestimatedbyslightlymodifyingtheproposedalgorithm.thenalstep, showninfig.1.letgilr;i=1:::p,betheobserved,degraded,misregistered,lowresolutionimages,wherepisthe (registration,restoration,andinterpolation)canbesolvedindependently.fortheregistrationsub-problem,mort sub-pixeldisplacementsmustbeaccuratelyestimated(registration).additionally,theseimagesmustberestored.if interpolation,usesalloftheresultingimages,filr;i=1:::ptoobtainthedesiredsamplinglattice. associatedwiththem,wherethesubscriptsdenotethedirectionofdisplacement(eitherthexorydirections).these numberofframesavailable.lettherstframebethereferenceframe.allotherframeshaveashift,(ix;iy);i6=1, Inordertosolvethehighresolutionproblem(obtainingfHRfromgiLR;i=1;:::;P),thethreesub-problems Insuchcases,imageprocessingtechniquesmaybeused.Towardsthisend,threesub-problemsmustbesolved, andsrinathusedamaximumlikelihoodformulationtoestimatethesub-pixelshifts[12].forrestoration,srinivas andsrinathusedamulti-channelformulationinordertousethecrosscorrelationtermsanddegradations[13]. Theyalsousedaparametricmodelforthecorrelations,inordertoincorporatetheshifts(assumedknown)intothe covariancematrix.
However,theapproachisnotoptimalbecausethesub-problemsareinterdependent.Asecondapproachisto combinetwo(orevenallthree)sub-problemstogether.theearliestcomprehensiveworkwasdonebytsaiand Huang[18],wheretheylookedatallthreesub-problems.However,theimagesweredegradation-andnoise-free, sothatrestorationwasnotnecessary.morerecently,kimandbose[10]solvedtherestorationandinterpolation sub-problemstogether,usingarecursivealgorithm,assumingthedisplacementshiftswereknown.in[15]thehigh resolutionproblemwassolvedbyextendingtwopreviouslyproposedapproachesinordertoaccountforsensornoise, butagain,theshiftsareassumedtobeknown(orestimatedbyanotheralgorithm). Adistinguishingfeatureoftheproposedalgorithmfromallofthepreviouscomprehensiveapproachesisthat thersttwosub-problemsaresolvedsimultaneously(seefig.1).thecombinationofthersttwosub-problems ismotivatedbythefactthattherestorationandregistrationsub-problemsareinter-dependent.clearly,accurate estimatesoftheshiftsdependupontheimageshavingaslittledegradationandnoiseaspossible,whileatthesame time,goodrestorationresultsdependupontheshiftsbeingaccuratelyknown.inordertosimultaneouslyregister andrestoretheimages,theexpectation-maximization(em)algorithmwillbeused,basedonpreviouswork[11,17]. Oneadvantageofthisalgorithmisthatamulti-channelframeworkisused.Suchaframeworkusesthecrosschannel informationcontainedinthecorrelationsanddegradations,andyieldsbetterresultsthanindependentlyrestoring eachimage[3,13]. Thispaperisorganizedasfollows.InSec.2,theproblemofsimultaneouslyperformingtheregistrationand restorationstepswillbeformulatedandsolvedusingtheemalgorithm.insec.3,theinterpolationstepwillbe covered.insec.4,experimentalresultsandpracticalconsiderationswillbediscussed,andsec.5concludesthis paper.2.registrationandrestorationusingtheemalgorithm Considerthedegradationmodel glr=dflr+v; (1) wheredrepresentstheblurmatrix,vtheadditivenoise,andglrandflrarerespectivelytheobservedand originallowresolutionimages,resultingfromthestackingofp-channelimages.eachobservedlowresolutionframe, gilr;i=1;:::;p,islexicographicallyordered,andthemthcomponentofglrisap1vector,withelementsbeing gilr(m).[11,9].assumingthattheblurisknownandthattheobservedlowresolutionframesaredisplacedby sub-pixelshiftswithrespecttoareferenceframe,thetaskathandisto1)ndanimageascloseaspossibletoflr insomesense,and2)estimatethesub-pixelshifts.inthispaper,amaximumlikelihood(ml)approachwillbe used,baseduponourpreviousworkofmulti-channelimagerestorationandbluridentication[17].flrisassumed tobegaussian,uncorrelatedwithv.glrisalsogaussianwithprobabilitydensityfunction fg(g)=2?dfdh+v?12exp?12gt?dfdh+v?1g; (2) wherevandfarethecovariancematricesofthenoiseprocessandtheoriginalimage,respectively,andthe subscriptslrhavebeendropped.takingthelogarithmofeq.(2)andignoringconstantterms,themaximization ofthelog-likelihoodfunctionbecomestheminimizationofthefunctionl('),givenby L(')=logDFDH+V+gT?DFDH+V?1g; (3) where'isthesetofvariablesofinterest.byusingtheexpectation-maximization(em)algorithmforthelinear Gaussiancase[2,11],thefunctiontobeminimizedbecomes L(';'(k))=logjFj+logjVj+trn??1 F+DH?1 VD(k) Fjgo +(k)h Fjg??1 F+DH?1 VD(k) Fjg?gH?1 VD(k) Fjg+(k)H FjgDH?1 Vg+gH?1 Vg; (4)
wheretr(a)denotesthetraceofa,thesuperscript(k)denotesthekthiteration,ahisthehermitianofa,fjgis theconditionalcovariance,andfjgistheconditionalmean,whichyieldstherestoredimage. [4,13]willbeused,givenby Inordertoincorporatethesub-pixelshiftsintothelikelihoodfunction,theseparablecovarianceimagemodel (k) (k) Fjg=(k) FD(k)HD(k)(k) F?(k) FD(k)HD(k)(k) FD(k)H+(k) cov(f(i;j);f(k;l))=2fji?kj FD(k)H+(k) 1jj?lj V?1g V?1D(k)(k) F; (6) (5) where2f;1;2arethevarianceoftheimageandthecorrelationcoecientsinthexandydirections,respectively. Eachlowresolutionframeismodeledas Theparametersetnowbecomes'=nfi covariancematrixcanbewrittenas LR(m;n)=f1 LR(m+ix;n+iy);i=1;:::;P;m;n=0;:::;N?1: LR;1;2;2f;ix;iyoforallframesi=1;:::;P.UsingEqs.(7)and(8),the (8) wherew=wwip,wherewisthetransformationmatrixofsizennconstructedfrom2ddftkernels, withthe(i;j)thelementsofr1;r2;sequaltoi?j andipistheidentitymatrixofsizepp. semi-blockcirculant,andcanbediagonalizedusingkroneckerproductformulasof[4],sothat F=2fR1R2S; F=WFW?1; 1;i?j 2;ix?jx 1iy?jy 2,respectively.ItcanbeveriedthatXis (10) (9) rowofr1,or where1=wr1w?1isthecorrespondingdiagonalmatrixofr1,whoseentriesaregivenbythedftoftherst UsingtheKroneckerproductformulasandEq.(10),Eq.(9)canbewritteninthefrequencydomainas F=2f?WR1W?1?WR2W?1S =2f12S; 1(m)=N?1 Xn=0n1wmn N; (11) wherej(m;n)=logjv(m;n)j+trn?2f1(m)2(n)s?1 wherewn=exp??j2 L(';'(p))=PN2log?2f+PNlogj1j+PNlogj2j+N2logjSj+N?1 N,andsimilarly2=WR2W?1.Eq.(4)cannowbewritteninthefrequencydomainas Xm=0N?1 Xn=0J(m;n); (13) (12) +HD(m;n)?1 V(m;n)D(m;n)(p) Fjg(m;n)o+1 +1 N2GH(m;n)?1 M(p)H Fjg(m;n)HD(m;n)?1 V(m;n)D(m;n)M(p) V(m;n)D(m;n)M(p) V(m;n)G(m;n) Fjg(m;n)M(p)H Fjg(m;n)+ Fjg(m;n)o N22ftrnh(1(m)2(n)S)?1 N2GH(m;n)?1 V(m;n)G(m;n); (14)
whereg;mfjgarethedftsofg;fjg,respectively,andv;fjg;dareobtainedbydiagonalizingv;fjg;d, channelcase respectively.ineq.(14),itisassumedthatthenoiseiszeroacrosschannels,andwhitewithinchannels,sothat ThegradientofLwithrespectto',denotedbyrL,canthenbefound.ItcanbeshownthatforthegeneralP @2f=PN2 @L2f?14ftr?1 V(m;n)=264210:::0 1?1 2S?1Fjg+1 00:::2P375: 02:::0. N2MFjgMHFjg; (15) @1=PN"?N2N?1 @L+1 2fN?1 Xm=0N?1 Xn=0@?1 1?N1+N?1 2@1?1 1(m) Xm=0wmN 1?1wmN#+N2 2(n)trS?1B(m;n) jsj@fjsjg @1; (17) (16) where @ix=n2@logjsj @LB(m;n)=Fjg(m;n)+1 @ix+trn?2f12s?1b(m;n)o;i=1;:::;p; N2MFjg(m;n)MHFjg(m;n): (18) casewhenp=2.instead,numericalmethodsmustbeusedtoestimate'(k+1).inthispaper,thesteepestdescent explicitequationsfor'(k+1),theaboveequationsaresetequaltozeroandsolved.unfortunately,whileclosedform approachisused.theiterativeequationfortheparametersetis unchangedfrom[17],duetothefactthattheyaredecoupledfromtherestoftheparameterset.inordertoobtain expressionscanbeobtainedfor2f;2i,eqs.(17)and(18)cannotbesolvedfor1;ix,respectively,eveninthesimple ThepartialsofLwithrespecttoeachofthenoisevariances,21;:::;2P,arenotshown,becausetheyremain (19) wherehisthehermitianofl(h=@2l whereisthestepsize,givenby '(k+1)='(k)?rl; =rlhrl rlhhrl; (20) tobefound,sincehissymmetric,andthenoisevariancesarecompletelydecoupledfromtherestoftheparameter byinterchangingixwithiy,1with2,and2with1ineq.(18).likewise,bymakingthesamesubstitutions, set. @2canbefoundfromEq.(17).Finally,incalculatingthehermitian,H,only@2L @LItisworthwhiletonotethatsinceixappearswith1inexactlythesamewayasiydoeswith2,@L @'2). @1@2;@2L @1@2f;@2L @1@ix;@2L @iycanbefound @1@iyneed (21) highresolutionimage.firstofall,theformationofthelowresolutionimagesfromthehighresolutionimagecanbe modeledasfllr(m;n)=xn1xn2fhr(n1;n2)sinwx?lx+mt0x?n1tx Finally,therestoredlowresolutionimagesgivenbytheconditionalmeanneedtobeinterpolateduptothedesired 3.INTERPOLATION Wx(lx+mT0x?n1Tx)sinWy?ly+nT0y?n2Ty Wy?ly+nT0y?n2Ty; (22)
andwyaredenedby where(t0x;t0y)and(tx;ty)arethesamplingintervalsinthexandydirections,respectivelyforeachframe,andwx Byusingthesameorderingof[9],Equation(22)canbewritteninmatrixvectorformas Wx=2 flr=()fhr; Tx;Wy=2 Ty: (23) where fhr=fhr(0)fhr(1):::fhr(n2?1); flr=flr(0)flr(1):::flr(n2?1) (25) (24) InEq.(24),()istheinterpolationoperatorbetweenthelowandhighresolutiongrids,givenby fhr(m)[fhr(4lxlym)fhr(4lxlym+1):::fhr(4lxlym+4lxly?1)]t: flr(m)f1lr(m)f2lr(m):::fp(m)t ()=2640;00;1:::0;N2?1 N2?1;0N2?1;1:::N2?1;N2?1375PN24LxLyN2; 1;01;1:::1;N2?1..... (26) and=[1x;1y;:::;px;py]. Eachsub-blockof()isi;j=264 1i;j(0) 2i;j(0) 1i;j(1)::: 2i;j(1)::: 1i;j(4LxLy?1) 2i;j(4LxLy?1) (27) where ìj(k)=sinc2`x+in2lx?jn2lx?k sinc2?ỳ+mod(i;n)2ly?mod(j;n)2ly?mod(k;2ly) Pi;j(0). Pi;j(1):::... Pi;j(4LxLy?1)375P4LxLy;. 2Ly : (28) InEq.(29),bxcdenotesthegreatestintegerofx,andmod(x;y)xmoduloy.Furthermore,therelationships wherebistheparametercontrollingconvergenceaswellastherateofconvergence,andatdenotesthetransposeof betweenthesamplingperiodofthelowresolutionframeandthatofthehighresolutionframeweretakentobe A.Inthispaper,Qistakentobethe2DLaplacian.Theregularizationparameter,,istheinverseofthevariance iterationisused iterativeapproach,similartotheonein[6,8],isalsofollowedhereinsolvingeq.(24).inotherwords,thefollowing Tx=2Lx,T0y T0x ofthehigh-passedlowresolutionimage,qflr,sincethehigh-passedimagebehavesmorelikeagaussianthanthe TheinversionofEq.(24)isanill-posedproblem,duetothelargesizeandtheerrorsin().Aregularized Ty=2Ly. originalimage[5]. f(k+1) HR=f(k) HR+bT()fLR??T()()+QTQf(k) HR; (30)
fourlowresolutionimageswereextracted,eachofsize128128,andeachdisplacedbydierentamounts.the (normalized)sub-pixelshiftswere(0:0;0:25);(0:75;0:0);(0:5;0:75),wheretherstframewastakenashavingzero a55gaussianpsf,andgaussiannoiseaddedtoeachframe,suchthateachframehadansnrof40db.these theleft.thesefourlowresolutionframeswerethenblurredindependently(i.e.,nocrosschanneldegradations)with displacement(thereferenceframe).figure2depictsthesefourframes,withthereferenceframebeingtheimageon blurredimagesareshowninfig.3.throughouttheseexperiments,itisassumedthattheblursareknown.figure Inthefollowingexperimentalresults,theoriginalairplaneimage(size512512)wasused.Fromthisimage, 4.EXPERIMENTALRESULTS wordsontheplaneremainindecipherable. factthatmanyoftheexistingregistrationalgorithmsassumeoriginal(thoughpossiblynoisy),unblurredimages. 4showsthe(undegraded)rstframeofthissequencebilinearlyinterpolatedupto256256.Notethatallofthe Therestorationwasdonebyusing[17],wherenoknowledgeaboutthesub-pixelshiftswasused.Eachlowresolution becametheiterativewienerlter.forcomparisonpurposes,itisassumedthattheshiftsareknownexactlyforthe wasrestoredindependently,sothatnocross-correlationtermswereused.sincetheblurswereassumedknown,this interpolationstep,thus,oeringa\bestcase"scenariofortheirrapproach.intheeventthattheshiftsarenot neouslyrestoringandregistering(srr)theimagesusingthealgorithmpresentedinsec.2.theestimatedandtrue known,registrationalgorithmssuchastheonein[12]maybeused. Forindependentregistrationandrestoration(IRR),therestorationstepwasperformedrst.Thisisduetothe valuesfortheshiftsareshownintable1.thetopfourframesoffigure5showtherestoredframeswithindependent improvedbyincorporatingthesub-pixelshifts.generallyspeaking,thebottomfourframesareconsistentlysharper proposedalgorithm,theimprovementinsnr(isnr)fortheithframe,denotedbyi,isdenedby registrationandrestoration(irr),whilethebottomfourframesarethoseimagesrestoredbythesrrapproach. thantheircounterparts(thetopframes).thisiscanbeseeninthe\f16"ontheplane'stail.inaddition,the ringingaroundtheplaneismorenoticeableinthetopframes.inordertoobtainaquantitativeevaluationonthe Bycomparingthewordsontheairplane,itisevidentthattherestorationofthelowresolutionframesisgreatly Theseindependentlyrestoredandregistered(IRR)resultswerecomparedwiththeresultsobtainedbysimulta- where^filristheithframeoftherestored(lexicographicallyordered)lowresolutionimage.table2liststheisnrs forallfourframes. (iterativewienerlter)frames.theseresultsconrmthepremisethattheregistrationandrestorationstepsare inter-dependentproblems.themaindierencebetweentheirrandsrrframesisthatthesrrrestoredimages NotethattheISNRfortheSRRframeswasconsistentlybetween1.5to2.0dBhigherthanthatoftheIRR i=kfilr?gilrk areslightlysharper,attheexpenseofhavingmorenoisepresent.themainadvantage,however,isthatthewordson kfilr?^filrk; (31) theairplanearesharper,whichwewillseeyieldsamuchsharperinterpolatedimage.inthesrralgorithm,thetrue andsrrapproaches,respectively,goingfromlefttoright.inobtainingthesrrimage,thetruesub-pixelshifts valuesforthenoisevariancesforeachofthefourframeswere0:1602;0:1599;0:1575;0:1645,andthecorresponding givenintable1wereused.inalloftheaboveexperiments,lambdawassetto0.00,sincetheimageswerenotvery 256256,isshowninFig.6.ThenexttwoimagesinFig.7correspondtotheinterpolatedframesusingtheIRR estimatedvarianceswere0:1971;0:1941;0:1943;0:1992. noisy(40dbsnr).incaseswherenoiseismoresignicant,regularizationshouldbeused. interpolated,usingtheexactvaluesfortheshifts.afterthreeiterations,theresultanthighresolutionimage,size Eq.30.Theinitialvaluesforf(0) Next,thelowresolutionrestoredframesareinterpolateduptothedesiredhighresolutionsamplinglatticeusing HRweresettozero,andb=0:8.First,thefouroriginalframesfromFig.2were separablecovariancemodel,thesub-pixelshiftscouldbetakenintoaccountinthelikelihoodfunction.however, Inthispaper,thetwosub-problems,restorationandregistration,havebeensolvedsimultaneously.Byusingthe 5.CONCLUSION
theinitialconditions(0) theyappearinthelikelihoodfunctionsimilarly. algorithm,theinitialconditionsplayasignicantroleindeterminingtheresults.onepointtoconsideristhatif previousworkusingtheemalgorithmneededtobemodiedtomaximizethisfunctionwithrespecttotheshifts. Inparticular,asteepestdescentmethodwasemployedtoperformthemaximization(M)step.Withanydescent sincemoreinformation(intheformoftheadditionallowresolutionframes)wasincorporatedintotheinterpolation step.inaddition,therestoredimagesusingthesrrapproachoeredalmost1.5dbimprovementovertheirr areexpectedifallthreesub-problemsaresolvedsimultaneously.thisisthesubjectofcurrentresearch. case.itshouldbekeptinmind,however,thathighlyaccurateestimatesofthedisplacementsareverydicultto ndfromtheobserveddataalone.whilethispapercombinedthersttwosub-problemstogether,improvedresults Experimentalresultsdemonstratedthattheapproachoersmoreresolutionthanbilinearinterpolationalone, 1=(0) 2,and(0) x=(0) y,thenthenalestimatesofxandywillbeexactlythesame,since [1]K.Aizawa,T.Komatsu,andT.Saito,\ASchemeforAcquringVeryHighResolutionImagesUsingMultiple ThisworkwassupportedinpartbytheSpaceTelescopeScienceInstitute. 6.ACKNOWLEDGEMENTS [2]M.FederandE.Weinstein,\ParameterEstimationofSuperimposedSignalsUsingtheEMAlgorithm,"IEEE [3]N.P.GalatsanosandR.T.Chin,\DigitalRestorationofMultichannelImages,"IEEETrans.Acoust.,Speech, Cameras,"IEEEProc.ICASSP-92,SanFrancisco,vol.III,pp.289-292,1992. Trans.Acoust.,Speech,SignalProcessing,vol.ASSP-36,pp.477-489,April1988. SignalProcessing,vol.ASSP-37,No.3,pp.415-421,March1989. 6.REFERENCES [6]A.K.Katsaggelos,\IterativeImageRestorationAlgorithms,"OpticalEngineering,vol.28,no.7,pp.735-748, [5]M.G.Kang,AdaptiveIterativeImageRestorationAlgorithm,Ph.D.thesis,NorthwesternUnivsity,June1994. [4]A.K.Jain,FundamentalsofDigitalImageProcessing,PrenticeHall,1988. [10]S.P.Kim,N.K.Bose,andH.M.Valenzuela,\RecursiveReconstructionofHighResolutionImageFromNoisy [9]A.K.Katsaggelos,K.T.Lay,andN.P.Galatsanos,\AGeneralFrameworkforFrequencyDomanMulti-Channel [7]A.K.Katsaggelos,\AMultipleInputImageRestorationApproach,"J.VisualCommun.ImageRepresent., [8]A.K.Katsaggelos,J.Biemond,R.M.Mersereau,andR.W.Schafer,\ARegularizedIterativeImageRestoration SignalProcessing,"IEEETrans.ImageProcessing,vol.2,no.3,pp.417-420,July1993. Algorithm,"IEEETrans.SignalProcessing,vol.39,no.4,pp.914-929,April,1991. July1989. UndersampledMultiframes,"IEEETrans.Acoust.,Speech,SignalProc.,vol.38,no.6,pp.1013-1027,June vol.1,pp.93-103,sept.1990. [12]M.S.MortandM.D.Srinath,\MaximumLikelihoodImageRegistrationwithSubpixelAccuracy,"SPIE,vol. [11]K.T.Lay,\MaximumLikelihoodIterativeImageIdenticationandRestoration,"Ph.D.thesis,Departmentof [13]C.SrinivasandM.D.Srinath,\AStochasticModel-BasedApproachforSimultaneousRestorationofMultiple [14]H.StarkandP.Oskoui,\High-resolutionImageRecoveryfromImage-PlaneArrays,UsingConvexProjections," EECS,NorthwesternUniversity,1991. 1990. [15]A.M.Tekalp,M.K.Ozkan,M.I.Sezan,\High-ResolutionImageReconstructionFromLower-ResolutionImage 974,pp.38-45,1988. [16]A.TikhonovandV.Arsenin,SolutionofIll-PosedProblems,JohnWileyandSons,1977. MisregisteredImages,"SPIE,vol.1360,pp.1416-1427,1990. 1992. SequencesandSpace-VaryingImageRestoration,"IEEEProc.ICASSP-92,SanFrancisco,vol.3,pp.169-172, J.Opt.Soc.Amer.A,vol.6,no.11,pp.1715-1726,Nov.1989.
[17]B.C.TomandA.K.Katsaggelos,\Multi-ChannelImageIdenticationandRestorationUsingtheExpectation- [18]R.Y.TsaiandT.S.Huang,\MultiframeImageRestorationandRegistration,"inAdvancesinComputerVision andregistration,vol.1,imagereconstructionfromincompleteobservations,t.s.huang,ed.,pp.317-339,jai MaximizationAlgorithm,"SPIE,vol.2298,July1994. Press,1984. Table1:TrueandEstimatedValuesfor(ix;iy),SimultaneousRegistrationandRestoration (2x;2y)(0.00,0.25)(0.1435,0.4166) (3x;3y)(0.75,0.00)(0.7934,0.1887) (4x;4y)(0.50,0.75)(0.6527,0.8527) TrueValueEstimatedValue Table2:ImprovementSNR(dB)fororiginalandmodiedEM frameioriginalemmodiedem 1234 3.3289 2.9704 3.2669 2.6686 5.0318 4.8350 4.4606 4.0446
g1lrg2lrg3lr- CombinedRegistration&Restoration gplr-. Determinethe shiftsofeach Find(ix;iy)- frame.-andnoisefrom Restoration Removethe degradation eachframe f1lr- fplr- f2lr- f3lr-. resolutionimage Interpolateup Interpolation toahigh fhr fhr - Figure1:Blockdiagramofconstructingahighresolutionframefrommultiplelowresolutionframes Figure2:Subsampled128128frames Figure3:Degradedlowresolutionframes
Figure4:Frame#1ofFig.2,bilinearlyinterpolatedto256256 Figure5:Restoredlowresolutionframes,IRR(top)andSRR(bottom)
Figure6:Interpolatedimagefromtheoriginallowresolutionframes Figure7:InterpolatedimagesfromtheIRR&SRRrestoredlowresolutionframes