Near Light Correction for Image Relighting and 3D Shape Recovery

Transcription

1 Near Light Correction for Iage Relighting and 3D Shae Recovery Xiang Huang, Marc Walton, Greg Bearan and Oliver Cossairt Northwestern University, Evanston, IL, USA ANE Iage, Pasadena, CA, USA Abstract In this aer, we roose a near-light illuination odel for iage relighting and 3D shae recovery Classic ethods such as used by the oular RTI software fro Cultural Heritage Iaging assue that lighting is infinitely far away fro the scene However, this constraint is iossible to achieve in ractice: light sources cannot be too far away fro the scene due to sace and ower constraints This causes non-unifor illuination artifacts due to variations in the distance between the light source and oints in the scene We correct this effect to rovide uch ore uniforly lit iages that yield ore aealing iage for relighting alications Furtherore, we use our near-light odel for ore accurate hotoetric stereo calculations of surface norals, eliinating the otato-chi shaed surface reconstruction error that results fro violating the far-light assution We verify our odel with both free-for cature using hand-held flash as the illuination source, and cature using LED lights ounted on a doe shaed surface Keywords Reflectance Transforation Iaging RTI), Polynoial Texture Ma PTM), HSH, Near Light Calibration, Photoetric Stereo, Shae fro Shading, 3D Surface Shae Reconstruction, Iage Relighting I INTRODUCTION The aearance of a wor of art results fro the hysical interaction of light with its constituent aterials To forally cature a colete record of aearance one should easure all the essential coonents of its light-transort function: the total interaction of the aterials and icrostructure corising a wor of art with all ossible incoing/illuinating light rays, easured by all ossible outgoing/observable light rays [1] Ebedded in the light-transort function are each fixed fraction of incident light the artwor will absorb, reflect, refract, scatter, and transit fro its surface, for all ossible incidentlight locations, directions, wavelengths, and olarizations It also contains the directional fractions of incident light that will scatter beneath the object s surface and re-eerge in different directions fro various neighboring oints While these easureents are theoretically ossible, the enorous size of such data and the tie required to cature the is largely iractical As a result, the light-transort function is tyically undersaled by taing only a few iages of an artwor with varying lighting conditions rather than directly easuring all the ossible interactions between light and object One exale of such a technique is Reflectance Transforation Iaging RTI), originally nown as Polynoial Texture Maing PTM) Malzbender [2] first roosed RTI as a way to interactively change the lighting conditions of a digital iage By interolating ultile iages of an object, each with different illuination angles fro a fixed caera osition, an active hoto ay be roduced with easy controls that encourage exloration to see vanishingly-subtle features In the last decade the art conservation counity has becoe increasingly interested in using RTI for ore closely exaining artwors through relighting As a relighting technique, RTI has rovided visually coelling ways to interactively exlore surface relief and discover subtle surface features otherwise issing or indiscernible in ordinary hotos or by direct visual insection [3], [4] The freely available viewer software fro Cultural Heritage Iaging CHI) [5] can also exaggerate surfaces, ixel-by-ixel, to deict the toograhy ore clearly, and to coute estiates of surface noral vectors via hotoetric stereo or fro the PTM interolation equation itself However, while current RTI ethods offer conservators a owerful exloratory tool, the any systeatic aroxiations inherent to the technique liit its use to qualitative assessents of aearance Quantitative deterination of surface norals could be very useful for art conservation For instance, surface norals easured over tie can detect and a shae changes to an object within a useu environent More accurate surface norals ight hel deterine the sequences of brush stroes or in alication to a wor of art to hel understand how an artist ade the object As one ste towards quantitative surface noral estiations, here we address a fundaental liitation of the RTI odel; the assution that the whole object is lit fro the sae illuination angle with the sae illuination intensity across the entire field of view This requireent is rarely et in real-life exeriental conditions because the light would need to be laced infinitely far away fro the object and can t be satisfied due to sace constraints of ost worsaces, and a very owerful light to illuinate fro such a large distance The isatch between the lighting odel and real exeriental conditions has been docuented to roduce erroneous surface noral estiations, a otato-chi shae estiation error when the surface norals are integrated, and non-unifor illuination effects in relighting that we call the sot-light effect [6], [7], [8] A Contributions In this aer, we resent an aroach to reove deendence on this far light assution Using our technique we relight iages and erfor surface reconstructions using lighting in close roxiity to the object, either a atrix of LED lights ounted to a sall half eter doe or a handheld flash light source ositioned about one eter away Our new ethod offers the following contributions: Our ethod allows lighting direction estiation without the need for a irror ball laced in the scene, as ileented in ost in RTI exeriental setus Our ethod requires just a flat atte surface aroxiately Labertian) such as a color-checer calibration target to be laced in the scene Our ethod estiates not only the 2D lighting direction but the true 3D osition of light sources By estiating 3D light ositions, we can ore accurately solve for the surface noral and albedo for

2 all oints in the scene With our near-light odel, the recovered 3D surface shae is exet fro the otato-chi shae errors encountered in traditional RTI catures By coensating each ixel s iage intensity according to its distance to lighting often called flatfielding ) we reove the sot-light effect brightercenter, darer-border) often encountered in RTI catures We rovide freely available software written in Matlab that ileents our algorith on iage sets catured for RTI to create standard RTI files that can be viewed in the RTI viewer freely available fro CHI [5] II PREVIOUS WORK There are two coutation ethods for reresenting RTI iages using the CHI RTI Builder and Viewer software suites [5] The olynoial texture aing PTM) version, originally roosed by Malzbender [2], uses a olynoial basis function for light intensity interolation, and the heisherical haronics HSH) version [9] reduces the directional bias Surface norals are calculated differently for each reresentation The PTM version fits the ixel intensity to a local bi-quadratic function of light angles and then finds the noral by setting the derivative to zero, which has the effect of finding the direction of the brightest ixel [10] For the HSH version, three lighting directions are chosen to generate a set of three relit hotograhs fro the fitted HSH fitted data Conventional hotoetric stereo is then alied to using the chosen lighting directions and synthesized ixel intensities CHI software users can create norals as in RTI Viewer for both coutation ethods and exort those iages at JPG files for later use Integrating the RTI-derived norals to create a lofted 3D surface results in large scale surface cuing our otatochi surface) [7], [8], esecially at the edges coared to the center To correct for the large scale bending, MacDonald et al [8] easured additional deths of a few surface oints to bound the surface during reconstruction While this solution is feasible, it requires user intervention and its accuracy is difficult to characterize Instead, we see a ore rinciled solution that does not involve further interaction or easureent of the object As MacDonald et al [8] originally observed, violation of the far-light assution will result in norals that are inaccurate for very low satial frequencies and this situation is quite coon for RTI cature As shown in Figure 1, a tyical RTI freefor cature set u can easily is-estiate light angles with errors that san 19 degree fro the left iage border to the right Errors in the estiated light direction will roduce incorrect noral estiation, which will in turn roduce errors in integrated deth In this aer, we correct these errors by using a near-light odel Photoetric stereo is a well established couter vision technique that often used to recover surface shae fro iage intensity The original forulation by Horn [11] assued lights are infinitely far away, the caera is orthograhic, and the object surface is Labertian and convex ie no shadows or inter-reflections) Since hotoetric stereo was originally introduced, several researchers have sought to generalize the technique for ore ractical caera, surface and lighting odels Belhueur et al [12] discovered that with an orthograhic caera odel and uncalibrated lighting, the object s surface can be uniquely deterined to within a bas-relief abiguity Paadhiitri and Favaro et al [13] recently ointed out that this abiguity is resolved under the ersective caera odel Fig 1: Potato-chi shaed surface calculated fro tyical RTI data In this exale the object is 50 c wide, sooth and uniforly flat, and the light is 150 c above the object The center of the object has an incident illuination angle of 90 degrees, but the left and right border of the central scan line have incident illuination angles of 805 degrees and 995 degrees resectively, with an error range of 19 = 2 atan25/150) degrees fro left to right The erroneous lighting angles cause is-estiated surface norals in RTI, which causes the otato-chi shaed surface Several researchers have also sought to relax the Labertian reflectance assution and incororate effects such as secular highlights and shadows New techniques have been introduced based on non-labertian reflectance odels [14], [15], [16], or sohisticated statistic ethods to autoatically filter nonlabertian effects [17], [18], [19] However, less attention has been aid to relaxing assutions on the lighting odel Several other researchers [20], [21] recently investigated reoving the far-light assution to irove the accuracy of hotoetric stereo Others [22] further considers non-isotroic illuinations In this aer, we devise a near-light odel and introduce a new fast otiization ethod to solve an energy iniization roble and obtain ore accurate light osition and surface noral estiates Our ethod corrects for inaccuracies in the conventional RTI cature rocess, roducing ore accurate relighting results, surface noral estiates, and 3D odels fro catured data, and yet it reoves the troublesoe requireent to cature a irror ball iage within each artwor hotograh III IMAGE INTENSITY FORMATION MODEL OF NEAR LIGHT SOURCE We consider the hysical odel of light transort as shown in Figure 2 The light rays are eitted fro a source, reflect fro an object surface, then finally reach a caera The ixel intensities easured at the caera sensor deend on three coonents: the light source, the object shae and reflectance, and the caera ixel s intensity resonse and lens vignetting Each are discussed individually Both RTI and conventional hotoetric stereo assue each light source is infinitely far away, eg sun light, and illuinates fro direction ˆl = ˆl x, ˆl y, ˆl z ), where the hat ato ˆl denotes noralized unit-length) vector This far-light assution is widely used to silify iage foration equations: light incident at any scene oint arrives with the sae angle and ower However, in ractice it is not ossible to lace lights far fro the scene due to sace and ower constraints Thus, a near-light odel is necessary to ore accurately describe the non-unifor light distribution incident on the scene Given an isotroic oint light source with ower e at osition l = l x, l y, l z ) note the hat -free l is not a unit vector), and each scene oint = x, y, z ) that illuinates fro direction l, the irradiance will fall by the square of distance:

3 IV AN OPTIMIZATION SCHEME FOR NEAR LIGHT POSITION AND SURFACE NORMAL ESTIMATION The near light odel in Equation 2 of Section III describes the ixel intensity given the light osition and ower, and the object shae and albedo In this section, we solve the inverse roble: given the observed ixel intensity, coute the light and object araeters using our near-light odel We gather a series of iages with fixed caera osition illuinated fro different light ositions We do a least-square fitting using the odel for ixel intensities fro Equation 2 and the easured intensity A solution for lighting ositions is found by iniizing the following energy function for all N ixels in K iages: Fig 2: Light transort fro the oint light source, reflected by the object surface, and sensed by the caera Figure courtesy of Trucco and Verri [23] e l 2 Unlie the distant light odel, the light direction and energy is not unifor across all scene oints Light incident on an object s surface is reflected according to surface aterial s bidirectional reflectance distribution function BRDF) As in conventional hotoetric stereo, we assue the surface is Labertian In the far-light odel, reflected light intensity falls by cosine of lighting angle: R = ˆn ˆlea = ˆn T ˆlea, where ˆn = ˆn x, ˆn y, ˆn z ) denotes the unit-length surface noral and a denotes albedo, the reflectivity at oint In our near-light odel, the reflected light energy taes the ore colicated for: R = ˆn T We refer to the quantity fl, ) = ˆn T l ) l 3 ea l ) l 3 1) as the light encil, which deends entirely on the scene geoetry, naely the object shae and light osition The light encil consists of two ters The first ter ˆn T l ) l reresents the cosine result of the dot-roduct between the light angle and surface noral as in the far- light odel The 1 second coonent l describes the squared distance fall 2 off of light energy Assuing a linear caera odel, the intensity easured at a ixel I is roortional to the aount of light reflected at the corresonding oint R : I = ˆn T l ) l 3 ea η = ˆn T l ) l 3 ea 2) The fractional coefficient η is due to caera vignetting and deends on ixel location: defines as 10 at the center of the iage and saller at the border Vignetting is artly caused by a natural illuination fall-off nown as the cosine fourth law: the vignetting coefficient is roortional to the four ower of the cosine of the angle between the rincial ray and otical axis Vignetting can also be caused by blocing of lighting rays fro aertures within a coound lens, and often stronger for large-aerture and wide-angle lenses For a fixed caera osition, albedo and vignetting ters cannot be not searated, we can only estiate their cobined effect a = a η El, e, ˆn,, a ) =, ˆn T l ) l 3 e a I 3) In the energy iniization, we have N K observed ixel values, and 4K + 3N indeendent araeters to solve More secifically, we need to solve 3K araeters for lighting ositions l, K araeters for lighting intensity e, 2N araeters for surface norals with unit-nor ˆn, and N araeters for cobined albedo and vignetting a Note that the 3D surface osition can be integrated fro estiated surface norals ˆn [24] In the case of catured rgb color iages, we have 3N K observed ixel values and 6K + 5N indeendent araeters 3K instead of K araeters for rgb light intensity, 3N instead of N araeters for rgb albedo ties vignetting) Directly solving the energy iniization roble defined in Equation 4 is generally rohibitive, as it is non-convex with illions of araeters 4K + 3N, where the ixel nuber N for ost odern caeras is in the range of 10 illion We use an alternating iniization forulation that iteratively solves two subrobles: one sall scale non-convex otiization to estiate light osition given albedo and noral, and another linear least squares otiization to find albedo and noral given 3D light osition The stes are as follows: 1) Given the cobined albedo and vignetting a, surface noral n and 3D scene oint, we calibrate light osition l and ower e 2) Given the lighting osition l, ower e, and 3D scene oint, we coute the albedo a and noral n The albedo and noral can be couted for each ixel by solving a least squares roble siilar to conventional hotoetric stereo: arg in a,ˆn, a ˆn T l l I where the distance coensated ixel intensity, 4) I = I l 2 e 5) After couting the noral, the surface shae can be integrated fro the noral using the ethod fro Agrawal et al [24] 3) We iteratively solve robles 1) and 2) siilar to the wor by Paadhiitri et al [21] The details of erforing ste 2) are siilar to conventional hotoetric stereo Ste 3) is siilar to the iterative rocedure introduced by Paadhiitri et al [21] In ste 1),

4 Paadhiitri et al [21] roose to initialize the scene as a lanar object n = [0, 0, 1] with constant effective albedo a = 1 They erfor an initial brute force search within a one eter cubic volue with a 10c ste to deterine light ositions in order to get an initial estiate for the light ositions However, we found in ractice that this ethod tends to fail because the effective albedo a = a η isn t constant due to vignetting, which results in an erroneous estiation of light osition Instead, we roose a new otiization rocedure for estiating the 3D light ositions that eliinates deendence on albedo and vignetting ie the ter a in Equation 4) Rather than erforing a brute force search for 3D lighting ositions, we derive an objective function that can be solved using gradient decent or quasi-newton s otiization ethod and is found to be robust to initialization value As a result, our light estiation is uch ore robust to sources of noise and odeling error coon in RTI catures To derive our new objective function, we first transfor Equation 2 for the oint under lighting to a l 3 = I ˆn T l ) e 1 = I gl, )e 1, where the inverse of light encil fl, ) 1 is defined as gl, ) = l 3 ˆn T l ), 6) which urely deends on the geoetric roerties of the scene: the light osition and object shae Equation a = I gl, )e 1 holds for all = 1, 2,, K lights for a given oint Those K easureents are equal in theory, but slightly different fro each other in ractice due to noise and odeling error We find the best lighting and surface shae araeters that iniize the variance of the exected a fro each lighting instance, for each ixel, by iniizing the following objective function: D = 1 N = 1 1 NK I gl, )e 1 1 K I gl, )e 1 ) 2 I gl, )e 1 I gl, )e 1 I gl, )e 1 The variance of a is noralized by its total ower to eliinate bias towards sall albedo values and closer lights The above objective function Dl, e, ˆn, ) has iniu value of 0 and axiu value of 1 1/K It reaches a iniu of 0 when i gl, )e 1 = i n gl n, )e 1 n for all, n 1, 2,, K It reaches a axiu of 1 1/K when the ratio I gl, )e 1 / axi gl, )e 1 ) for all 1, 2,, K has K 1 zeros, which occurs when one of the lights is at actually at finite distance fro the scene, but erroneously couted to be a very far distance away 7) With the new objective function, the light osition and intensity can be udated by the Broyden-Fletcher-Goldfarb-Shanno BFGS) quasi-newton otiization ethod, with the gradient as D l = 2 NK D e 1 = 2 NK Q = Q 3l ) l 2 Q e I gl, )e 1 I gl, )e 1 I gl, )e 1 I gl, )e 1 ) ˆn ˆn T l ) I gl, )e 1 I gl, )e 1 8) 1 In theory, all ixels in the catured iages can be used to erfor this otiization In ractice, we use only a sall nuber generally less than five ercent of all ixels), generated by downsaling the iage As a result, 3D light estiation can be erfored in only seconds on an ersonal desto with Intel R Xeon R E GHz CPU Note that our new fraewor still requires initial values for the cobined albedo and vignetting and surface shae in ste 1), and light osition and ower in ste 2) We initially assue the scene is flat, so that ˆn = 0, 0, 1) T, and = u, v, 0) where u, v) are ixel coordinates We have found that our algorith converges for a large variety of initial light ositions In all our exerients, we initialized all lights to have the sae ower and osition l = [w/2, h/2, axw, h)], where w, h are the width and height of the iage in ixels While our algorith does not exlicitly require surface shae to be nown ahead of tie, in ractice, any scenes often have a relatively flat surface as bacground that can be used to further constrain the lighting otiization V EXPERIMENTAL RESULTS We now show how our new near light odel can irove the results of iage relighting in RTI and also the surface noral reconstruction Calibrated light ositions are shown in Figure 3 for a lighting doe with Canon 5D Mar III caera and a rie lens of 20 For reference, coarisons are shown using triangulation with ultile sherical balls laced in the scene The 3D light ositions for the doe are catured faithfully with our technique while significant errors are resent when triangulating with ultile irror balls Fig 4: a) Freefor cature b) doe cature

5 a) Calibrated light osition fro 5 irror sheres b) Our calibrated light osition fro one iece of aer Fig 3: A Coarison of 3D light osition estiation using triangulation fro ultile irror balls left) and the ethod roosed in this aer right) In the left figure, the light ositions of 81 doe lights obtained by least square error triangulation fro five irror sheres laced in the center, to left, to right, botto left, botto right of the scene In the right figure, the light ositions of the sae setu is estiated fro just a iece of white atte rinting aer The light ositions fro the irror balls are subject to large triangulation errors for lights near the to of the doe Our technique, however, roduces highly accurate estiates of 3D light osition A Iage Relighting using PTM or HSH with Near Light Once we have accurately estiated the 3D location of light sources during RTI cature, we can use this to generate ore accurate relighting results The PTM or HSH iage relighting ethods assue iages are catured under distant light so that each ixel is lit fro sae lighting angle with sae energy If iages are catured using near lights but the 3D location of light sources is nown, it is still ossible to generate accurate relighting results using these ethods The PTM and HSH techniques relight an iage under a novel lighting direction by interolating basis iages catured fro a fixed caera and a set of nown lighting directions PTM stores six coefficients c = [c 0, c 1, c 2, c 3, c 4, c 5 ] T for each ixel, and coutes ixel intensity I fro a novel illuination direction ˆl as I = c 0 lx 2 + c 1 ly 2 + c 2 l x l y + c 3 l x + c 4 l y + c 5 = hˆl) T c, where the biquadratic olynoial hˆl) = [lx, 2 ly, 2 l x l y, l x, l y, 1] HSH is siilar to PTM, but uses Heisherical Haronics functions instead of olynoial functions as basis function hˆl) After caturing a set of iages with K nown lighting direction K larger than the nuber of coefficients in c ), the coefficients of HSH or PTM can be couted by solving an over-deterined linear syste of equations: I = c 0 l 2 x + c 1 l 2 y + c 2 l x l y + c 3 l x + c 4 l y + c 5 = hˆl) T c hˆl 1 ) hˆl 2 ) hˆl K ) c = I 1 I 2 I K 9) With near light sources, catured iages suffer fro a nonunifor illuination artifact that roduces a sot light effect: regions of the object further away fro the light source will receive weaer illuination and aear darer For exale, in the doe cature setu used in this aer, scene oints near the border receive as little as 10% of the illuination received at the center To correct for non-unifor illuination, we coute a corrected or re-lit iage I so that each ixel I aears to be lit fro a distant light source as shown in Figure 5 We first correct for the distance-squared attenuation to ixels in all K catured iages The corrected iages are then used to fit the coefficients of a relighting interolation function in a siilar way to Equation 9 However, unlie PTM or HSH where each ixel has the sae interolation lighting atrix H, we create a ixel deendent H atrix by relacing light direction ˆl with l ) l, as shown in Equation 10 h l1 ) l 1 ) h l2 ) l 2 ) h l K ) l K ) c = I 1 I 2 I K 10) Figure 5 shows a coarison between relighting results with and without our near-light correction for two RTI catures of rints by the artist Paul Gauguin, housed within the eranent collection at the Art Institute of Chicago The iages were catured using our doe illuination, which roduces a severe sotlight effect in the raw catured data After 3D light osition estiation and near-light correction is alied, the relit iages aear uniforly lit and are generally uch ore visually leasing B Surface Noral and Shae Reconstruction RTI easures only one light direction at a lace near the object using a irror ball, and alies the sae light direction for any oint of the object; this does not atch the near-light odel for close illuination The isatching light angles cause erroneous surface noral estiates which yield otatochi surface shae when integrated To deterine the accuracy of our technique in easuring 3D surface shae, we use flat atte aer as a ground truth lanar surface Figure 6 shows coarisons between raw data catured using our lighting doe, and corrected data using our 3D light estiation technique Before correction, 3D surface reconstructions exhibit a severe bending, resulting in a otato chi lie aearance due to large global errors in surface shae estiation After our correction is alied, the 3D surface aears nearly flat, indicating it has been reconstructed with uch higher quality We further test 3D reconstruction quality for a cature of a woodbloc rint by Paul Gauguin housed at the Art

6 Institute of Chicago, as shown in Figure 7 A siilar bending is 3D reconstruction without correcting for the near-light effect shows the failiar otato chi bending so that surface details are difficult to resolve However, after alying our near-light correction, the 3D reconstruction of the woodbloc aears uch flatter As a result, inute details in the carving of the woodbloc can now be discerned fro the reconstruction VI CONCLUSION We have develoed a novel autoatic ethod to estiate the 3D location of light sources for catured hotoetric stereo and RTI data We have shown how to incororate our results into a near-light odel for ore accurate 3D surface reconstructions and show results for both ground truth data, as well as RTI catures of woodblocs and rints fro the Art Institute of Chicago s collection of the wor of Paul Gauguin Our wor ay have direct iact on cultural heritage iaging alications by roviding further stes towards inexensive and accurate 3D surface easureent techniques that are accessible to curators and conservation scientists around the world Our 3D light estiation and RTI generation software will be released as a freely available MATLAB acage that will integrate sealessly into the worflow of any curators or conservators currently using RTI software The inut to our software will be a set of hotos fro a tyical RTI cature setu and the outut will be an RTI file directly loadable in RTI Viewer software VII FUTURE WORK In the future, we want to investigate ore accurate hysical odels for light sources Currently we assue lights are isotroic and light ower is eitted uniforly across all angles In ractice, eitted ower usually exhibits soe degree of angular deendence esecially for systes such as the doe lighting used for exerients in this aer Soe recent wors odel the non-isotroic effect of light sources [25], [22] We are also interested in develoing ethods to erfor accurate 3D surface estiation using ore sohisticated odels of aterial reflectance In addition, we would lie to incororate non-linear global illuination effects such as shadows and ulti-bounce inter-reflections within the scene VIII ACKNOWLEDGEMENT This roject was undertaen at the Northwestern University / Art Institute of Chicago Center for Scientific Studies in the Arts NU-ACCESS) NU-ACCESS is funded through a generous grant fro the Andrew W Mellon Foundation Suleental suort is rovided by the Materials Research Center, the Office of the Vice President for Research, the McCoric School of Engineering and Alied Science and the Deartent of Materials Science and Engineering at Northwestern University REFERENCES [1] K Kutulaos, Light Transort Analysis for 3D Photograhy, Sixth International Conference on 3-D Digital Iaging and Modeling 3DIM 2007), no 3di, , 2007 [2] T Malzbender, D Gelb, and H Wolters, Polynoial texture as, in Proceedings of SIGGRAPH 2001, Annual Conference Series New Yor, New Yor, USA: ACM Press, 2001, [Online] Available: htt://ortalacorg/citationcf?doid= [3] M Mudge, T Malzbender, C Schroer, M Lu, R Art, and R Fields, New Reflection Transforation Iaging Methods for Roc Art and Multile-Viewoint Dislay, in VAST, 2006 [4] G Earl, K Martinez, and T Malzbender, Archaeological alications of olynoial texture aing : analysis, conservation and reresentation, Journal of Archaeological Science, 1 11, 2010 [Online] Available: htt://dxdoiorg/101016/jjas [5] Cultural Heritage Iaging: Reflectance Transforation Iaging RTI), 2013 [Online] Available: htt://culturalheritageiagingorg/technologies/rti/indexhtl [6] L Macdonald and S Robson, Satial Calibration of an Illuination Doe Conventional coystand, Tech Re, 2011 [7] M Elfarargy, A Rizq, M Rashwan, B Alexandrina, and P O Box, 3D Surface Reconstruction Using Polynoial Texture Maing, in Lectures Notes in Couter Science, no 1, 2013, [8] L W Macdonald, Reresentation of Cultural Objects by Iage Sets with Directional Illuination, no 45 c, 43 56, 2015 [9] S Y Elhabian, H Rara, and A a Farag, Towards accurate and efficient reresentation of iage irradiance of convex-labertian objects under unnown near lighting, Proceedings of the IEEE International Conference on Couter Vision, , 2011 [10] G Pala, M Corsini, P Cignoni, R Scoigno, and M Mudge, Dynaic shading enhanceent for reflectance transforation iaging, Journal on Couting and Cultural Heritage, vol 3, no 2, 1 20, 2010 [11] B K P Horn, Obtaining shae fro shading inforation, The Psychology of Couter Vision, , 1975 [Online] Available: htt://dlacorg/citationcf?id=93877 [12] P N Belhueur, D J Kriegan, and A L Yuille, The Bas-Relief Abiguity, IJCV, vol 35, no 1, 33 44, 1999 [13] T Paadhiitri and P Favaro, A new ersective on uncalibrated hotoetric stereo, Proceedings of the IEEE Couter Society Conference on Couter Vision and Pattern Recognition, , 2013 [14] a Hertzann and S Seitz, Shae and aterials by exale: a hotoetric stereo aroach, 2003 IEEE Couter Society Conference on Couter Vision and Pattern Recognition, 2003 Proceedings, vol 1, 1 8, 2003 [15] N Alldrin, T Zicler, and D Kriegan, Photoetric stereo with nonaraetric and satially-varying reflectance, 26th IEEE Conference on Couter Vision and Pattern Recognition, CVPR, 2008 [16] D B Goldan, B Curless, A Hertzann, and S M Seitz, Shae and satially-varying BRDFs fro hotoetric stereo, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol 32, no 6, , 2010 [17] L Wu, A Ganesh, B Shi, Y Matsushita, Y Wang, and Y Ma, Robust hotoetric stereo via low-ran atrix coletion and recovery, Lecture Notes in Couter Science including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinforatics), vol 6494 LNCS, , 2011 [18] S Iehata, D Wif, Y Matsushita, and K Aizawa, Robust hotoetric stereo using sarse regression, in Proceedings of the IEEE Couter Society Conference on Couter Vision and Pattern Recognition, vol 1, no 1, 2012, [19] M Zhang, Robust surface noral estiation via greedy sarse regression, PhD dissertation, 2014 [Online] Available: htt://suitsfuca/ite/13719 [20] A Wetzler, R Kiel, A M Brucstein, and R Mecca, Close-Range Photoetric Stereo with Point Light Sources, in nd International Conference on 3D Vision, 2014, [Online] Available: htt://ieeexloreieeeorg/ldocs/eic03/wraerht?arnuber= [21] T Paadhiitri, P Favaro, and U Bern, Uncalibrated Near-Light Photoetric Stereo, in Proceedings of the British Machine Vision Conference, 2014, 1 12 [22] Y Quéau and J-d Durou, Soe Illuination Models for Industrial Alications of Photoetric Stereo, 2015, QCAV [23] E Trucco and A Verri, Introductory Techniques for 3-D Couter Vision Prentice Hall PTR, 1998 [24] A Agrawal, R Rasar, and R Chellaa, What is the Range of Surface Reconstructions fro a Gradient Field? in ECCV, 2006, [25] J Par, S N Sinha, and Y-w Tai, Calibrating a Non-isotroic Near Point Light Source using a Plane, in CVPR, 2014,

7 a) Originally Catured Iage I b) Light Intensity Mas I c) Corrected Iage I d) Originally Catured Iage II e) Light Intensity Mas II f) Corrected Iage II g) Originally Catured Iage I h) Light Intensity Mas I i) Corrected Iage I j) Originally Catured Iage II ) Light Intensity Mas II l) Corrected Iage II Fig 5: Relighting coarisons for two wors by Paul Gauguin housed at the Art Institute of Chicago, a woodbloc to), and a transfer rint botto) The concession nuber for the woodbloc is , and for the rint A coarison is shown between the raw catured iages left) and after the near-light correction technique introduced in this aer right) We use the calibrated light osition to coute the light attenuation due to the distance squared fall-off The inverse of this attenuation as iddle) is used to roduce relit iages with even illuination right) The corrected iages loo uniforly lit and ore visually leasing

8 60 z ) x ) a) 4 of 81 catured iages b) Noral: far-light odel c) 3D surface: far-light d) Red scanline of the surface 60 z ) x ) e) 4 of 81 corrected iages f) Noral: near-light odel g) 3D surface: far-light h) Red scanline of the surface Fig 6: Exeriental coarison between far-light and near-light odels for surface noral and 3d shae reconstruction We use a flat atte aer as a ground truth test object The first colun shows the catured and corrected iages The osition of the red dot in the circle aroxiates lighting direction The second colun shows the surface noral The ground truth should be unifor, but the noral fro the distant light odel has a large error esecially at the border The third colun shows the reconstructed 3d surface The last colun shows a horizontal scan line of the recovered deth a The near-light odel results are close to ground truth, while the far-light odel results in a large otato chied distortion a) 1 of 81 catured iages b) Reconstructed 3D surface using a far-light c) Reconstructed 3D surface using our near-light odel odel Fig 7: Coarison of far-light and near-light odels for 3d shae reconstruction The wor of art shown is a woodbloc roduced by artist Paul Gauguin, housed at the Art Institute of Chicago The concession nuber of the artwor is a) One of the catured iages showing the near-light illuination effect b) A 3D reconstruction without correcting for the near-light effect The woodbloc aears to be bent lie a otato chi so surface details are difficult to resolve c) A 3D reconstruction using our ethod to correct for the near-light effect The woodbloc is now flat and details of the carving can be discerned fro the reconstruction