On the relationship between radiance and irradiance: determining the illumination from images of a convex Lambertian object

Transcription

1 2448 J. Opt. Soc. Am. A/ Vo. 18, No. 10/ October 2001 R. Ramamoorthi and P. Hanrahan On the reationship between radiance and irradiance: determining the iumination from images of a convex Lambertian object Ravi Ramamoorthi and Pat Hanrahan Department of Computer Science, Stanford University, Gates Wing 3B-386, Stanford, Caifornia Received October 30, 2000; revised manuscript received March 8, 2001; accepted March 12, 2001 We present a theoretica anaysis of the reationship between incoming radiance and irradiance. Specificay, we address the question of whether it is possibe to compute the incident radiance from knowedge of the irradiance at a surface orientations. This is a fundamenta question in computer vision and inverse radiative transfer. We show that the irradiance can be viewed as a simpe convoution of the incident iumination, i.e., radiance and a camped cosine transfer function. Estimating the radiance can then be seen as a deconvoution operation. We derive a simpe cosed-form formua for the irradiance in terms of spherica harmonic coefficients of the incident iumination and demonstrate that the odd-order modes of the ighting with order greater than 1 are competey annihiated. Therefore these components cannot be estimated from the irradiance, contradicting a theorem that is due to Preisendorfer. A practica reaization of the radiance-fromirradiance probem is the estimation of the ighting from images of a homogeneous convex curved Lambertian surface of known geometry under distant iumination, since a Lambertian object refects ight equay in a directions proportiona to the irradiance. We briefy discuss practica and physica considerations and describe a simpe experimenta test to verify our theoretica resuts Optica Society of America OCIS codes: , INTRODUCTION This paper presents a theoretica anaysis of the reationship between incoming radiance and irradiance. Radiance and irradiance are basic optica quantities, and their reationship is of fundamenta interest to many fieds, incuding computer vision, radiative transfer, and computer graphics. Physicay, we are interested in anayzing the properties of the ight fied generated when a homogeneous convex curved Lambertian surface of known geometry refects a distant iumination fied. A Lambertian surface refects ight proportiona to the incoming irradiance, so anaysis of this physica system is equivaent to a mathematica anaysis of the reationship between incoming radiance and irradiance. The specific question of interest to us in this paper is the estimation of the incident radiance from the irradiance, i.e., estimation of the ighting from observations of a Lambertian surface. We are abe to derive a cosed-form formua in terms of the spherica harmonic coefficients for the irradiance and thereby to show that odd modes of the ighting with order greater than 1 cannot be estimated. The rest of this paper is organized as foows. In Section 2, we briefy discuss some previous work. In Section 3, we introduce the mathematica and physica preiminaries. In Section 4, we obtain the equation for the irradiance in terms of spherica harmonic coefficients. Section 5 appies this resut to the main probem of this paper, recovering the input radiance from the irradiance, and derives the main resuts of this paper. Section 6 briefy discusses some practica considerations and describes a simpe experimenta verification. We discuss some of the broader impications of our resuts and concude the paper in Section PREVIOUS WORK The radiance-from-irradiance probem as discussed in this paper is addressed by Preisendorfer 1 in his treatise on hydroogic optics. He considers the recovery of radiance, given irradiance at a surface orientations. Preisendorfer s concusion is that irradiance and radiance are equivaent and that irradiance can be inverted to give the input radiance. By deriving a simpe cosed-form formua, we wi show that this assertion is not true. More recenty, Marschner and Greenberg 2 have considered the inverse ighting probem, assuming curved Lambertian surfaces, and have proposed a practica soution method. However, they have noted the probem to be i conditioned and have made extensive use of reguarization. By deriving a cosed-form formua for the case of convex objects, we can expain the i conditioning observed by Marschner and Greenberg 2 and propose aternative agorithms. Sato et a. 3 use shadows instead of curvature to recover the ighting in a scene, and they aso estimate the refectance parameters of a panar surface. Inverse probems in transport theory have aso been studied in other areas such as radiative transfer and neutron scattering. See McCormick 4 for a review. However, to the best of our knowedge, these researchers have not addressed the specific radiance-from-irradiance probem treated here, /2001/ $ Optica Society of America

2 R. Ramamoorthi and P. Hanrahan Vo. 18, No. 10/October 2001/J. Opt. Soc. Am. A 2449 wherein we use the varying refected radiance over a curved Lambertian surface to estimate the input ighting. The work presented here is reated to efforts in many different areas of computer graphics and vision and is ikey to be of interest to these communities. Within the context of rendering surfaces under distant iumination (referred to as environment mapping within computer graphics), many previous authors such as Mier and Hoffman 5 and Cabra et a. 6 have quaitativey described refection as a convoution and have empiricay demonstrated that a Lambertian bidirectiona refectance distribution function (BRDF) behaves ike a ow-pass fiter. In computer vision, simiar observations have been made by many researchers such as Haddon and Forsyth 7 and Jacobs et a. 8 Our main contribution is in formaizing these previous quaitative resuts by deriving anaytic quantitative formuas reating the incoming radiance to the irradiance. Athough our goas are different, our work is aso reated to efforts in the object recognition community to characterize the appearance of a Lambertian surface under a possibe iumination conditions, and it is ikey that our resuts wi be appicabe to this probem. Behumeur and Kriegman 9 have theoreticay described this set of images in terms of an iumination cone, whie empirica resuts have been obtained by Epstein et a. 10 In independent work simutaneous with our own, Basri and Jacobs 11 have described Lambertian refection as a convoution and have appied the resuts to face recognition. To derive an anaytic formua for the input iumination, we must anayze the properties of the refected ight fied from a homogeneous Lambertian surface. The ight fied 12 is a fundamenta quantity in ight transport and therefore has wide appicabiity for both forward and inverse probems in a number of fieds. A good introduction to the various radiometric quantities derived from ight fieds that we wi use in this paper is provided by McCuney. 13 Light fieds have been used directy for rendering images from photographs in computer graphics, without considering the underying geometry, 14,15 or by parameterizing the ight fied on the object surface. 16 In previous work, we 17 have performed a theoretica anaysis of two-dimensiona or fatand, ight fieds, which is simiar in spirit to the anaysis in this paper for threedimensiona Lambertian surfaces. To derive our resuts, we wi represent quantities by using spherica harmonics In previous work, D Zmura 21 has quaitativey anayzed refection as a inear operator in terms of spherica harmonics. Basis functions have aso been used in representing BRDFs for computer graphics. A number of authors have used spherica harmonics, whie Koenderink and van Doorn 25 have described BRDFs by using Zernike poynomias. surface refecting a distant iumination fied. For the physica system, we wi assume that the surfaces under consideration are convex, so they may be parameterized by the surface orientation, as described by the surface norma, and so that interrefection and shadowing can be ignored. Aso, surfaces wi be assumed to be Lambertian and homogeneous, so the refectivity can be characterized by a constant abedo. We wi further assume here that the iuminating ight sources are distant, so the iumination or the incoming radiance can be represented as a function of direction ony. This aso means that the incident iumination does not depend directy on surface position but depends ony on surface orientation. Finay, for the purposes of experimenta measurement, we wi assume that the geometry of the object and its ocation with respect to the camera are known, so that we can reate each image pixe to a particuar ocation on the object surface. The notation used in the paper is isted in Appendix A. A diagram of the oca geometry of the situation is shown in Fig. 1. We wi use two types of coordinates. Unprimed goba coordinates denote anges with respect to a goba reference frame. On the other hand, primed oca coordinates denote anges with respect to the oca reference frame, defined by the oca surface norma and an arbitrariy chosen tangent vector. These two coordinate systems are reated simpy by a rotation, and this reationship wi be detaied shorty. B. Refection Equation In oca coordinates, we can reate the irradiance to the incoming radiance by Ex Lx, i, i cos i d, (1) where E is the irradiance, as a function of position x on the object surface, and L is the radiance of the incident ight fied. As noted in Subsection 3.A, primes denote quantities in oca coordinates. The integra is over the upper hemisphere with respect to the oca surface norma. Practicay, we may observe the irradiance by considering the brightness of a homogeneous Lambertian refector. For a Lambertian surface with constant refectance, we can reate the refected radiance to the irradiance by Bx Ex, (2) 3. PRELIMINARIES A. Assumptions Mathematicay, we are simpy considering the reationship between the irradiance, expressed as a function of surface orientation, and the incoming radiance, expressed as a function of incident ange. The corresponding physica system is a curved convex homogeneous Lambertian Fig. 1. Loca geometry. Quantities are primed because they are a in oca coordinates.

3 2450 J. Opt. Soc. Am. A/ Vo. 18, No. 10/ October 2001 R. Ramamoorthi and P. Hanrahan where B is the radiant exitance (i.e., brightness) as a function of position x on the object surface, is the surface refectance, which ies between 0 and 1, and E is the irradiance defined in Eq. (1). In computer graphics, the radiant exitance B is usuay referred to by the oder term radiosity. Note that the radiant exitance B is simpy a scaed version of the irradiance E and is exacty equa to E if the Lambertian surface has refectance 1. Furthermore, for a Lambertian object, the refected radiance is the same in a directions and is therefore directy proportiona to the radiant exitance, being given by B/. In the rest of this paper, we wi anayze Eq. (1), reating the irradiance E to the incident radiance L. It shoud be understood that the practicay measurabe quantities for a Lambertian surface the radiant exitance B or the corresponding refected radiance are simpy scaed versions of the irradiance and can be derived triviay from E and the surface refectance by using Eq. (2). We now manipuate Eq. (1) by performing a number of substitutions. First, the assumption of distant iumination means that the iumination fied is homogeneous over the surface, i.e., independent of surface position x, and depends ony on the goba incident ange ( i, i ). This aows us to make the substitution L(x, i, i ) L( i, i ). Second, consider the assumption of a curved convex surface. This ensures that there is no shadowing or interrefection, so the irradiance is due ony to the distant iumination fied L. This fact is impicity assumed in Eq. (1). Furthermore, since the iumination is distant, we may reparameterize the surface simpy by the surface norma n. Equation (1) now becomes En L i, i cos i d. (3) The goa of this paper is to determine what we can earn about the incident radiance L from measuring the functiona dependence of the irradiance E on the surface norma n. To proceed further, we wi parameterize n by its spherica anguar coordinates (,, ). Here (, ) define the anguar coordinates of the oca norma vector, i.e., Note that oca and goba coordinates are mixed. The ighting is expressed in goba coordinates, since it is constant over the object surface when viewed with respect to a goba reference frame, whie the transfer function A cos i is expressed naturay in oca coordinates. To anayze this equation further, we wi need to appy a rotation corresponding to the surface orientation (,,) in order to convert the ighting into oca coordinates. C. Rotating the Lighting We have assumed that the incoming ight fied remains constant (in a goba frame) over the object. To convert to the oca coordinates in which ( i, i ) are expressed, we must perform the appropriate rotation on the ighting. Let L( i, i ) be the goba incoming radiance and (,,) the parameters corresponding to the oca surface orientation. We define R,, to be a rotation operator that rotates ( i, i ) into goba coordinates ( i, i ). R,, can be expressed in terms of the standard Euer-ange representation and is given by R,, R z ()R y ()R z (), where R z is a rotation about the Z axis and R y is a rotation about the Y axis. Refer to Fig. 2 for an iustration. It is easy to verify that this rotation correcty transforms the oca coordinates (0, 0) corresponding to the oca representation of the Z axis, i.e., the surface norma to the goba coordinates (,) corresponding to the goba representation of the surface norma. The reevant transformations are given beow: R,, R z R y R z, i, i R,, i, i, L i, i L(R,, i, i ). (6) Note that the anguar parameters are rotated as if they were a unit vector pointing in the appropriate direction. It shoud aso be noted that this rotation of parameters is equivaent to an inverse rotation of the function, with R 1 being given by R z ()R y ()R z (). Finay, we can pug Eqs. (6) into Eq. (5) to derive n sin cos, sin sin, cos ; (4) defines the oca tangent frame, i.e., rotation of the coordinate axes about the norma vector. For isotropic surfaces those where there is no preferred tangentia direction, i.e., where rotation of the tangent frame about the surface norma has no physica effect the parameter has no physica significance, and we have therefore not expicity considered it in Eq. (3). We wi incude for competeness in the ensuing discussion on rotations but wi eventuay eiminate it from our equations after showing mathematicay that it does in fact have no effect on the fina resuts. Finay, for convenience, we wi define a transfer function A( i ) cos i. With these modifications, Eq. (3) becomes E,, L i, i A i d. (5) Fig. 2. Diagram showing how the rotation corresponding to (,, ) transforms between oca (primed) and goba (unprimed) coordinates.

4 R. Ramamoorthi and P. Hanrahan Vo. 18, No. 10/October 2001/J. Opt. Soc. Am. A 2451 E,, L(R,, i, i )A i d. (7) Note that this equation is essentiay a convoution, athough we have a rotation operator rather than a transation. The irradiance can be viewed as a convoution of the incident iumination L and the transfer function A cos i. Different observations of the irradiance E, at points on the object surface with different orientations, correspond to different rotations of the transfer function (since the oca upper hemisphere is rotated), which can aso be thought of as different rotations of the incident ight fied. In Section 4, we wi see that this integra becomes a simpe product when transformed to spherica harmonics, further stressing the anaogy with convoution. Our goa wi then be to deconvove the irradiance in order to recover the incident iumination. 4. SPHERICAL HARMONIC REPRESENTATION We now proceed to construct a cosed-form description of the irradiance in terms of spherica harmonic coefficients. Spherica harmonics are the anaog on the sphere to the Fourier basis on the ine or the circe. The spherica harmonic Y,m is given by Y,m, N,m P,m cos expim, (8) where N,m is a normaization factor. In the above equation, the azimutha dependence is expanded in terms of Fourier basis functions. The dependence is expanded in terms of the associated Legendre functions P,m. The indices obey 0 and m. Thus there are 2 1 basis functions for given order. Inui et a. 18 is a good reference for background on spherica harmonics and their reationship to rotations. We begin by expanding the ighting in goba coordinates in terms of spherica harmonics: L i, i 0 L,m Y,m i, i. (9) m We wi then transform to oca coordinates by appying the appropriate rotations. A. Rotation of Spherica Harmonics Let us now buid up the rotation operator on the spherica harmonics. We wi use notation of the form R()Y,m () to stand for a rotation by the rotation operator R of the parameters of the spherica harmonic Y,m. For instance, R z Y,m i, i Y,m (R z i, i ). (10) First, from the form of the spherica harmonics, rotation about z is simpe. Specificay, R z Y,m i, i Y,m i, i expimy,m i, i. (11) Rotation about y is more compicated and is given by R y Y,m i, i m D m,m Y,m i, i, (12) where D isa(2 1) (2 1) matrix that tes us how a spherica harmonic transforms under rotation about the Y axis, i.e., how to rewrite a rotated spherica harmonic as a inear combination of a the spherica harmonics of the same order. The important thing to note here is that the m indices are mixed a spherica harmonic after rotation must be expressed as a combination of other spherica harmonics with different m indices. However, the indices are not mixed; rotations of spherica harmonics with order are composed entirey of other spherica harmonics with order. Finay, we can combine the above two equations, with a simiar resut for the z rotation by, to derive the required rotation formua: R,, Y,m i, i D m,m R z R y R z Y,m i, i m D,, Y m,m,m i, i,,, D m,m expimexpim. (13) In terms of group theory, the matrix D can be viewed as the 2 1-dimensiona representation of the rotation group SO(3), with the interesting dependence being encapsuated by D. Equation (13) is simpy the standard rotation formua for spherica harmonics. Since the transfer function A cos i has no azimutha dependence, terms with m 0 wi vanish when we perform the integration in Eq. (7). Therefore we wi be most interested in the coefficient of the term with m 0, i.e., D m,0 D m,0 ()exp(im). It can be shown that this is simpy equa to 4/(2 1)Y,m (, ). This resut wi be assumed in Eq. (23). Another way to derive the resut just stated without appeaing to the properties of the matrix D is to reaize that we simpy want the coefficient of the term with no azimutha dependence. At i 0, the rotated function is determined ony by the term with m 0. In fact, it can be shown (Jackson, 19 Eq. 3.59) that, with D m,0 equa to the desired coefficient, we have R,, Y,m 0, i 2 1 1/2 D m,0,,. 4 (14) Noting that R,, Y,m (0, i ) corresponds to evauating Y,m at the oca Z axis or norma, i.e., goba coordinates of (, ), we see that D m,0 4/(2 1) 1/2 Y,m (, ). B. Representing the Transfer Function We switch our attention now to representing the transfer function A( i ) cos i. Since an object refects ony the upper hemisphere, A( i ) is nonzero ony when cos i 0. The transfer function A( i ) 0 over the

5 2452 J. Opt. Soc. Am. A/ Vo. 18, No. 10/ October 2001 R. Ramamoorthi and P. Hanrahan ower hemisphere where cos i 0. We may refer to the transfer function as the camped-cosine function, since it is equa to the cosine function over the upper hemisphere but is camped to 0 when cos i 0 over the ower hemisphere. We wi need to use many formuas for representing integras of spherica harmonics, for which a reference 20 wi be usefu. First, we expand the transfer function in terms of spherica harmonics without azimutha dependence: A i cos i n0 A n Y n,0 i. (15) The coefficients are given by /2 A n Y n,0 i cos i sin i d i. (16) 20 The factor of 2 comes from integrating 1 over the azimutha dependence. It is important to note that the imits of the integra range from 0 to /2 and not, because we are considering ony the upper hemisphere. The expression above may be simpified by writing in terms of Legendre poynomias P(cos i ). Putting u cos i in the above integra and noting that P 1 (u) u and that Y n,0 ( i ) (2n 1)/(4) 1/2 P n (cos i ), we obtain A n 2 2n 1 1/2 1 P n up 1 udu. (17) 4 0 To gain further insight, we need some facts regarding the Legendre poynomias. P n is odd if n is odd and even if n is even. The Legendre poynomias are orthogona over the domain [1, 1], with the orthogonaity reationship being given by 1 2 P a up b u 1 2a 1 a,b. (18) From this, we can estabish some resuts about Eq. (17). When n is equa to 1, the integra evauates to haf the norm above, i.e., 1/3. When n is odd but greater than 1, the integra in Eq. (17) vanishes. This is because for a n and b 1, we can break the eft-hand side of Eq. (18), by using the oddness of a and b, into two equa integras over [1, 0] and [0, 1]. Therefore both of these integras must vanish, and the atter integra is the righthand integra in Eq. (17). When n is even, the required formua is given by manipuating Eq. (20) in Chap. 5 of MacRobert. 20 Putting it a together, we have A n n 2. A pot of A n for the first few terms is shown in Fig. 3, and approximation of the camped cosine by spherica harmonic terms as n increases is shown in Fig. 4. C. Spherica Harmonic Version of the Refection Equation We now have the necessary toos to write Eq. (7) in terms of spherica harmonics. Substituting Eqs. (13) and (15), we obtain E,, n0 T n,,m i 0 0 m m L,m A n D m,m expimexpimt n,,m, 2 i 0 Y,m i, i Y n,0 i, i sin i d i d i. (20) Note that we have summed over a indices and have removed the restriction on the integra to the upper hemisphere, because that restriction has now aready been foded into the coefficients A n. By orthonormaity of the spherica harmonics, 2 i 0 i 0 Y,m i, i Y n,0 i, i sin i d i d i,n m,0. (21) Therefore terms that do not satisfy n, m 0 wi vanish. Making these substitutions in Eq. (20), we obtain E,, 0 m1 L,m A D m,0 expim. As noted in Subsection 4.B, it can be shown that 4 D m,0 expim 1/2 2 1 (22) Y,m,. (23) n 1: A n /3, n 1, odd: A n 0, n even: A n 2 2n 1 4 1/2 1 n/21 n 2n 1 n! 2 n n!/2 2. (19) We can determine the asymptotic behavior of A n for arge even n by using Stiring s formua. The bracketed term goes as n 1/2, which cances the term in the square root. Therefore the asymptotic behavior for even terms is Fig. 3. The soid curve is a pot of A n versus n. It can be seen that odd terms with n 1 have A n 0. Aso, as n increases, the coefficients rapidy decay.

6 R. Ramamoorthi and P. Hanrahan Vo. 18, No. 10/October 2001/J. Opt. Soc. Am. A 2453 E,m 5/2, since A 2. The transfer function A acts as a ow-pass fiter, causing the rapid decay of highfrequency terms in the ighting. We are now ready to answer the question motivating this paper: To what extent can we estimate the incoming radiance or iumination distribution given the irradiance at a orientations or surface normas. This can be viewed as a probem of deconvoution. Equation (26) makes it trivia to formuate a cosed-form soution. Fig. 4. Successive approximations to the camped-cosine function by adding more spherica harmonic terms. For n 2, we aready get a very good approximation. With this reation, we can write 1/2 4 E,, 0 A m 2 1 L,m Y,m,. (24) To compete the expansion in terms of spherica harmonics, we first drop the dependence of E. We can see that the right-hand side of the equation above does not depend on, as required by physica considerations, since has no physica significance. Then we compete the expansion in terms of spherica harmonics by aso expanding the irradiance E(, ), i.e., E, 0 E,m Y,m,. (25) m We can now equate coefficients to obtain the refection equation in terms of spherica harmonic coefficients: E,m 4 1/2 2 1 A L,m. (26) D. Discussion Equation (26) states that the standard direct iumination integra in Eq. (7) can be viewed as a simpe product in terms of spherica harmonic coefficients. This is not reay surprising, considering that Eq. (7) can be interpreted as showing that the irradiance is a convoution of the incident iumination and the transfer function, with different observations E corresponding to different rotations of the incident ight fied. Since Eq. (26) is in terms of spherica harmonic coefficients, it can be viewed in signa processing terms as a fitering operation; the output ight fied can be obtained by fitering the input ighting by using the camped-cosine transfer function. Since A vanishes for odd vaues of 1, as seen in Eqs. (19), the irradiance has zero projection onto oddorder modes, i.e., E,m 0 when 1 and odd. In terms of the fitering anaogy, since the fiter corresponding to A cos i destroys high-frequency odd terms in the spherica harmonic expansion of the ighting, the corresponding terms are not found in the irradiance. Further, for arge even, the asymptotic behavior of 5. RADIANCE FROM IRRADIANCE In this section, we discuss the inverse ighting probem. We want to find L,m given the functiona dependence of the irradiance on the surface norma. A practica reaization is that we want to find the incident iumination from observations of a homogeneous convex curved Lambertian surface. From Eq. (26), it is trivia to derive a simpe cosed-form reation: L,m 2 1 1/2 E,m. (27) 4 A We wi have difficuty soving for L,m ony if for a m, E,m and A both vanish, in which case the right-hand side cannot be determined. We have aready seen in Subsection 4.E that this happens when is odd and 1. This brings us to our main resut. Theorem 1. In genera, it is not possibe to recover the odd-order spherica harmonic modes with order 1 ofa distant radiance distribution from information about the irradiance at every surface orientation. In practica terms, observations of a homogeneous convex curved Lambertian surface do not determine the odd-order modes with order 1 of the incoming distant iumination fied. This theorem is simpe to understand in terms of signa processing. The fiter A has no ampitude aong certain modes, and it annihiates the corresponding ighting coefficients when convoved with the incident iumination. Therefore a deconvoution method cannot recover the corresponding origina components of the ighting. A stronger version of the theorem is that adding a perturbation to the incident ight fied consisting ony of odd modes with order 1 does not change the irradiance. Theorem 2. A perturbation of the incident distant iumination fied consisting entirey of a inear superposition of odd-order spherica harmonic modes with order 1 has no effect on the irradiance at any surface orientation and hence no effect on the refected radiance from, or on the appearance of, a convex curved Lambertian object. As a simpe exampe, consider adding a perturbation L Y 3,m ( i, i ) for any m to the incident ighting L. We consider the change in irradiance E at a point with surface norma in spherica coordinates (, ). To evauate this, we need to rotate the ighting into the oca coordinate frame. We know that for spherica harmonics, rotation does not change the order but merey mixes the indices. Since the cosine in the irradiance integra does not have azimutha dependence, the ony term that can

7 2454 J. Opt. Soc. Am. A/ Vo. 18, No. 10/ October 2001 R. Ramamoorthi and P. Hanrahan affect the irradiance is the term without azimutha dependence, i.e., Y 3,0 in oca coordinates. The precise coefficient of this term is not important, and for simiar reasons, we ignore the factor of 2 that comes from integrating over the azimutha ange. We can simpy denote C(, ) as a constant factor and write /2 E, C, Y 3,0 i cos i sin i d i. 0 (28) Substituting for Y 3,0, ignoring the premutipying constant that we absorb into C, and then substituting u cos i, we obtain E, C, 0 /2 cos i sin i d i C, 0 C, 0 C, cos3 i cos i u3 u u du u4 u du u50 C, u30 0. (29) To make matters concrete, we can now caim that the foowing two ighting functions are equivaent in that they produce the same irradiance at a surface orientations and therefore cannot be distinguished by observations of a convex Lambertian surface. The constants have been chosen to ensure that the ighting remains nonnegative everywhere: L 1 i, i 1, L 2 i, i cos3 i cos i. (30) A. Comparison with Preisendorfer Preisendorfer 1 (Vo. 2, pp ) concudes that radiance and irradiance are equivaent, with radiance aways being recoverabe from measurements of irradiance at a surface orientations. His argument is that a positive sum ighting perturbation must resut in a positive norm change in the integrated irradiance over a surface orientations. In fact, he shows that the norm of the change in the integra of the irradiance is proportiona to the sum of the ighting perturbation. Therefore he concudes that any ighting perturbation must resut in a corresponding perturbation of the refected ight fied. He negects to consider that whie overa, the ighting must be nonnegative, it is possibe to devise a zero-sum perturbation to the ighting, since the perturbation can have both positive and negative components, with the ony physica requirement being that the net ighting is nowhere negative. Indeed, a the odd-order modes, incuding the exampe above, have zero sum, since their integra with Y 00, the constant term, must be 0 by orthogonaity. Nevertheess, the condition that the ighting must be positive to be physicay reaizabe is important and provides a further constraint on aowabe perturbations. We wi see that because of the constraint of positivity, there are severa important specia cases where the irradiance distribution does in fact uniquey determine the radiance or input iumination distribution. However, in genera, as evidenced by the exampe above, the irradiance distribution fais to competey specify the radiance, as summarized in Theorems 1 and 2. B. Constraining the Lighting to be Positive We have so far not considered the physica requirement that the ighting be everywhere positive. To be physicay reaizabe, any perturbations of the incident iumination must be sma enough that the ighting remains nonnegative everywhere. Here we wi show how this constraint somewhat restricts the set of aowabe perturbations perturbations that do not affect the irradiance. Our anaysis is fairy straightforward, and we eave for future work a more compete characterization. We start by enumerating two important properties that aowabe perturbations must satisfy. Here we wi use L to denote a perturbation. 1. We know that an aowabe perturbation must be constructed of odd-order spherica harmonic modes. These modes have the property of being odd over the sphere the vaue of a function is negated at the antipoda point on the sphere. This can be written as L i, i L i, i. (31) 2. The remaining condition is that the perturbation s projection onto order-1 modes must be 0 (since the order-1 modes can be recovered). The order-1 modes are simpy the inear terms x, y, and z. These modes are inear and odd their vaue negates at the antipoda point. Therefore, in conjunction with condition 1, an aowabe perturbation must have zero inear moment over any hemisphere since the antipoda hemisphere is negated for both the function and the order-1 mode. Condition 2 is important in showing that for a directiona source, there is no aowabe nonzero norm perturbation. We first consider aowabe vaues of a perturbation, as described by the foowing emma. Lemma 1. If L is the true vaue of the incident iumination, the vaues of an aowabe perturbation L satisfy L i, i L i, i L i, i. (32) The norm of the maximum aowabe perturbation therefore satisfies L i, i maxl i, i,l i, i. (33) The first inequaity in reation (32) hods triviay to maintain positivity of L. The second inequaity in reation (32) foows from the need to maintain positivity at the antipoda point, since L( i, i ) L( i, i ). This simpe emma eads to a number

8 R. Ramamoorthi and P. Hanrahan Vo. 18, No. 10/October 2001/J. Opt. Soc. Am. A 2455 of coroaries for various specia cases, where we show that increasingy compex ighting conditions admit no nonzero norm perturbation. Coroary 1. Aowabe perturbations L( i, i ) must be 0 at points where the true ighting vaue L( i, i ) and the true ighting vaue for the antipoda point L( i, i ) are both 0. In particuar, if the true ighting L is everywhere 0, there is no aowabe perturbation. This foows triviay from reation (33). Note that if the true ighting is a singe directiona source, this condition forces the aowabe perturbation to be 0 everywhere except at the source and its antipoda point. Coroary 2. If the true ighting is a singe directiona source, there is no nonzero norm aowabe perturbation, and therefore the true radiance can be recovered from the irradiance distribution at a surface orientations. From the discussion beow Coroary 1, the aowabe perturbation can have a nonzero (negative) vaue ony at the source with a corresponding negation (positive vaue) at the antipoda point. A perturbation L at the source and L at the antipoda point yieds a nonzero moment with respect to one or both of cos i and sin i the order-1 modes. In other words, since we are on a sphere, one of x, y, orz has to be nonzero, so the inear moments cannot vanish. Note that by the second required property, these moments must vanish in an aowabe perturbation. Therefore a perturbation consisting of a negative spike with an equa positive spike at the antipoda point cannot be constructed by using ony oddorder spherica harmonics with order 1. In particuar, this aows Coroary 2 to be extended to the case of two antipoda sources, since an aowabe perturbation must have the same form a negative spike at one of the sources aong with an equa positive spike at the other source. A much stronger statement is found beow. Coroary 3. If the true ighting consists of two distinct directiona sources or a nondegenerate arrangement of three directiona sources, there is no aowabe nonzero norm perturbation. From the above discussions, we know that aowabe perturbations can be nonzero ony at the sources and their antipoda points. The condition of vanishing moments with respect to the inear order-1 modes eads to a set of three simutaneous equations requiring us to choose perturbation intensities at the sources so that the moments in x, y, and z vanish. This is not possibe for nondegenerate configurations of three sources uness a perturbation intensities are 0. It is aso not possibe for any two distinct sources, since we can reparameterize so that one of the sources is on the Z axis. To make a inear moments vanish, the other source woud need to have x y 0, i.e., be antipoda at Z, in which case, by the argument beow Coroary 2, there is no aowabe perturbation. The coroary does not extend to more than three directiona sources. For four directiona sources, we may set the (negative) perturbation arbitrariy at one of the sources to fix the scae. We can then sove the vanishing moment equations for the three remaining perturbations, and the condition for a nonzero norm perturbation is that a three perturbations at the sources are negative, as required by reation (32). With more than four directiona sources, we have much more freedom in choosing the perturbations, and it is ikey that there aways exist aowabe perturbations if the sources are far enough apart, i.e., do not satisfy the conditions of Coroary 4 beow. Our primary consideration in this paper has been continuous ighting distributions or area sources. For these sources (which can be thought of as made up of an infinite number of directiona sources), Coroary 3 ceary does not appy. Coroary 4. If there exists a hemispherica region in which the true ighting L 0 everywhere, i.e., a the sources are stricty confined to one hemisphere, there is no aowabe nonzero norm perturbation. We can aways reparameterize so that a the sources ie in Z for instance, and since L 0 over this hemisphere (since L at the antipode is 0), the inear moment over Z must be negative, vioating the condition that the inear moments of the perturbation must vanish. This coroary may have impications for natura ighting when the upper hemisphere is the major contributor. Note that the sources must be stricty confined to one hemisphere a coection of sources at the equator or any other great circe does not satisfy the requirements of the coroary. In Section 6, we wi consider practica issues with ighting recovery. It wi be shown that regardess of the theoretica resuts derived here, inverse ighting is in practice poory conditioned, so even in cases where a unique soution exists, a numerica agorithm is unikey to find it. Nevertheess, the resuts of this set of coroaries tes us that it wi be hepfu to try to expicity ensure positivity of the recovered soutions in any numerica agorithm; this makes possibe the soution of some cases or reduces the norm of the maximum aowabe perturbation that are otherwise ambiguous by Theorems 1 and PRACTICAL CONSIDERATIONS This section briefy discusses some practica considerations with respect to the radiance-from-irradiance probem and describes a simpe experiment verifying the formuas derived. Equation (26) and the discussion in Subsection 4.E considering asymptotic forms show that the coefficients of the irradiance fa off rapidy, i.e., E,m 5/2 L,m. This indicates that in practice the inverse ighting probem is very poory conditioned. In fact, we can expicity write out numericay the first few terms for the irradiance: E 0, L 0,0, E 1,m 2.094L 1,m, E 2,m 0.785L 2,m, E 3,m 0,

9 2456 J. Opt. Soc. Am. A/ Vo. 18, No. 10/ October 2001 R. Ramamoorthi and P. Hanrahan E 4,m 0.131L 4,m, E 5,m 0, E 6,m 0.049L 6,m. (34) We see that aready for E 4,m, the coefficient is ony approximatey 4% of what it is for E 0,0. Therefore in rea appications where surfaces are ony approximatey Lambertian and there are errors in measurements we expect to robusty measure the irradiance ony up to order 2, and this is the maximum order at which we can recover the incident iumination. Since there are 2 1 indices (vaues of m, which range from to ) for order, this corresponds to nine coefficients for 2: one term with order 0, three terms with order 1, and five terms with order 2. Note that the singe order-0 mode Y 0,0 is a constant, the three order-1 modes are inear functions of the Cartesian coordinates in rea form, they are simpy x, y, and z and the five order-2 modes are quadratic functions of the Cartesian coordinates. Therefore the irradiance or, equivaenty, the refected ight fied from a convex Lambertian object can be we approximated as a quadratic poynomia of the Cartesian coordinates of the surface norma vector. Enforcement of positivity constraints and consideration of error metrics based on higher-order derivatives may improve the resuts somewhat and make them physicay more pausibe, but it wi sti be virtuay impossibe to recover higher-order coefficients of the ighting. A. Discussion Thus, even though in theory we can recover a the even modes of the ighting, in practice we expect to recover ony the first nine coefficients of the ighting modes up to order 2. For practica purposes, the irradiance in genera can be characterized by ony its first nine coefficients; the others vanish or are too sma to be accuratey measured. Thus the Lambertian BRDF acts as a very owpass fiter, passing through ony the first nine coefficients of the ighting, with the irradiance effectivey restricted to being at most quadratic in the Cartesian coordinates of the surface norma vector. These observations hep expain the resuts of Marschner and Greenberg. 2 In that paper, an attempt was made to sove the inverse ighting probem, treating the surfaces as Lambertian. The authors noted that the probem appeared i conditioned and not amenabe to accurate soution. Therefore they had to rey heaviy on a reguarizing term that preserved the smoothness of the soution. The resuts in this paper show why the probem is i conditioned and suggest a different reguarization scheme. We can assume the highfrequency ighting coefficients to be inaccurate, so we do not attempt to recover them, and we merey set them to 0. This indicates that a spherica harmonic basis is idea for recovering the ighting. The fact that the irradiance is sufficienty sowy varying across the surface that it can be described by so few parameters has impications for a number of research areas, a fact that is expored in Section 7. Since Lambertian surfaces are a reasonaby cose approximation to many rea-word objects and are a widey used approximation in computer graphics and vision, we expect this resut to have wide appicabiity. B. Experimenta Verification We describe a simpe experiment to verify the resuts of the paper. Using a camera mounted on a spherica gantry, we took a few caibrated gray-scae images of a Tefon sphere with known radius and position from different viewing positions using the same iumination primariy from a distant ceiing ight and an umbrea amp. The ighting was measured in high dynamic range by using an amost perfecty specuar mirror sphere (gazing ba). The spherica harmonic coefficients of the ighting were then computed. We were abe to compute B(, ), the radiant exitance as a function of the surface norma, on the Tefon sphere by discarding specuarities and averaging measurements of the same surface ocation on the Tefon sphere as seen from different viewing directions. We then directy used Eq. (27) to determine the first nine spherica harmonic coefficients of the ighting. These coud then be compared with those obtained from the ighting measured by using the mirror sphere. Since we did not know the reative refectances of the Tefon and mirror spheres or, equivaenty, the precise scaing factor reating the radiant exitance B to the irradiance E there was a scae factor that we did not recover. Therefore we uniformy scaed one set of ighting coefficients to be abe to make meaningfu comparisons with the other. Figure 5 shows resuts from our experimenta test. In (A), we show an image of the mirror and Tefon spheres. We see that the image of the Tefon sphere is a ow-pass fitered version of the ighting, retaining essentiay none of the high-frequency content that is visibe in the image of the mirror sphere. Image (B) shows the highresoution rea iumination distribution, as recovered from the mirror sphere. Images (C) (F) compare recovered and rea iumination distributions. Note that since we do not expicity enforce positivity, there are some darker negative regions in images (C) (F). Image (F) represents a faied attempt; it is incuded to show the futiity of attempting to recover the higher-order modes of the ighting. However, a comparison of images (C) and (D) indicates that the first nine coefficients of the ighting can be we recovered from observation of a curved convex Lambertian surface. Thus we see that we are abe to recover the first nine coefficients of the ighting. But, as predicted by our theory, we fai to recover higher-order coefficients. Tabe 1 shows a numerica comparison of rea and recovered ighting coefficients (in rea form) for the first two orders of spherica harmonics. The second coumn shows the (scaed) irradiance coefficients E,m. These are divided by the third coumn, as per Eq. (27), to obtain the recovered ighting coefficients L,m, given in the penutimate coumn. The fina coumn has the rea ighting coefficients, as found by using a gazing sphere. We see that the rea and recovered vaues match cosey. We aso see that the irradiance coefficients are ower for 2 than for 0or 1, since there is greater attenuation by the BRDF fiter (A is smaer). Finay, Tabe 1 shows

10 R. Ramamoorthi and P. Hanrahan Vo. 18, No. 10/October 2001/J. Opt. Soc. Am. A 2457 Fig. 5. (A) One of the photographs of the mirror sphere (eft) and the Tefon sphere (right), (B) the rea ighting as recovered by using the mirrored sphere, (C) the ighting obtained by considering ony the first nine coefficients of (B), i.e., up to order 2, (D) the recovered ighting obtained by cacuating the first nine coefficients of the ight from the radiant exitance of the Tefon sphere, (E) the rea ighting up to order 4, (F) an attempt to recover the ighting up to order 4 by aso cacuating the nine order-4 modes. Images (B) (F) are visuaizations obtained by unwrapping spherica coordinates of the ighting. i ranges over [0, ] uniformy from top to bottom, and i ranges over [0, 2] uniformy from eft to right. The zero of the ighting is the gray coor used for the background of image (B). Tabe 1. Comparison of Recovered and Rea Lighting Coefficients (,m) E,m 4/(2 1) 1/2 A L,m (Rec.) L,m (Rea) (0,0) (1,1) (1,0) (1,1) (2,2) (2,1) (2,0) (2,1) (2,2) (3,3) (3,2) (3,1) (3,0) (3,1) (3,2) (3,3) the irradiance coefficients E,m for 3. According to the theory, these coefficients shoud be identicay 0, since A 3 0. We see that the experimenta vaues are indeed very cose to CONCLUSIONS We have presented a theoretica anaysis of the reationship between radiance and irradiance. We have shown that the operation of refection is anaogous to convoution of the iumination and a camped-cosine function and have derived a simpe cosed-form formua in terms of spherica harmonic coefficients. We have further demonstrated that the camped cosine or, equivaenty, the Lambertian BRDF acts as a very ow-pass fiter, making deconvoution to recover the ighting difficut. In fact, we have demonstrated that odd-order modes of the incident iumination with order 1 cannot be recovered from the irradiance, i.e., from observation of a Lambertian surface. In other words, the radiance-from-irradiance probem is i posed or ambiguous. We have aso presented evidence showing that in practica terms, the refected ight fied from a Lambertian surface is characterized ony by spherica harmonic modes up to order 2, and we can

11 2458 J. Opt. Soc. Am. A/ Vo. 18, No. 10/ October 2001 R. Ramamoorthi and P. Hanrahan therefore reiaby estimate ony the first nine coefficients of the incident iumination. These resuts confirm some previous empirica resuts and aso open up the possibiity of nove agorithms for probems in many areas of computer graphics and vision. For instance, in the genera context of inverse rendering to recover ighting and BRDFs, our resuts suggest the use of a spherica harmonic basis as a suitabe representation, with reguarization obtained by imiting the number of modes used. In rendering images in computer graphics with compex iumination represented by environment maps, our resuts indicate that for argey diffuse surfaces, an accurate ighting description is not necessary. Efficient agorithms might resut from rendering in frequency space, considering ony the first few spherica harmonic coefficients of the ighting. For object recognition under varying iumination, our resuts indicate that the space of a possibe images of an object can be easiy described by a sma basis set of images, corresponding to the owest-order modes of the ighting. Our work may aso have appications in visua perception. Since our resuts indicate that ighting cannot ead to rapid variation of intensity over a Lambertian surface, such variation must be because of secuarity or texture, and this resut may be usefu in expaining how one can perceive these quantities independenty of the iumination. Further work must be done on deveoping this theory for non-lambertian surfaces and in considering other effects such as shadows and interrefections. Further practica work is required on the probems just mentioned. We beieve that the techniques deveoped in this paper are of fundamenta interest and may provide a firm theoretica foundation for nove agorithms in many different research areas. APPENDIX A: NOTATION L Incoming radiance L,m Coefficients of spherica harmonic expansion of L E Irradiance E,m Coefficients of spherica harmonic expansion of E B Radiant exitance Surface refectance A Cosine of oca incident ange A cos i A Coefficients of spherica harmonic expansion of A i ( i ) Incident eevation ange in oca (goba) coordinates i ( i ) Incident azimutha ange in oca (goba) coordinates o ( o ) Outgoing eevation ange in oca (goba) coordinates o ( o ) Outgoing azimutha ange in oca (goba) coordinates () Hemisphere of integration in oca (goba) coordinates x Surface position n Surface norma Surface norma parameterization eevation ange Surface norma parameterization azimutha ange Tangent frame ange D m,m Representation matrices of SO(3) Y,m Spherica harmonic I 1 ACKNOWLEDGMENTS We thank the anonymous reviewers, especiay reviewer 1, for their suggestions, which have ed to a significant improvement in the exposition of the paper. Steve Marschner and Szymon Rusinkiewicz read eary drafts and provided many insightfu comments. This work was supported in part by a Hodgson Reed Stanford graduate feowship. Address correspondence to Ravi Ramamoorthi at the ocation on the tite page. The authors may aso be contacted by e-mai at ravir, hanrahan@graphics.stanford.edu or through the website REFERENCES 1. R. W. Preisendorfer, Hydroogic Optics (U.S. Department of Commerce, Washington, D.C., 1976). 2. S. R. Marschner and D. P. Greenberg, Inverse ighting for photography, in Fifth Coor Imaging Conference: Coor Science, Systems and Appications (Society for Imaging Science and Technoogy, Springfied, Va., 1997), pp I. Sato, Y. Sato, and K. Ikeuchi, Iumination distribution from brightness in shadows: adaptive estimation of iumination distribution with unknown refectance properties in shadow regions, in Seventh IEEE Internationa Conference on Computer Vision (IEEE Computer Society, Los Aamitos, Caif., 1999), pp N. McCormick, Inverse radiative transfer probems: a review, Nuc. Sci. Eng. 112, (1992). 5. G. Mier and C. Hoffman, Iumination and refection maps: simuated objects in simuated and rea environments, in SIGGRAPH 84 Advanced Computer Graphics Animation Seminar Notes (Association for Computing Machinery, New York, 1984). 6. B. Cabra, M. Oano, and P. Nemec, Refection space image based rendering, in SIGGRAPH 99, Computer Graphics Proceedings, Annua Conference Series, A. Rockwood, ed. (Association for Computing Machinery, New York, 1999), pp J. Haddon and D. Forsyth, Shape representations from shading primitives, in Fifth European Conference on Computer Vision, H. Burkhardt and B. Neumann, eds. (Springer-Verag, Berin, 1998), pp D. Jacobs, P. Behumeur, and R. Basri, Comparing images under variabe iumination, in IEEE Conference on Computer Vision and Pattern Recognition (IEEE Computer Society, Los Aamitos, Caif., 1998), pp P. Behumeur and D. Kriegman, What is the set of images of an object under a possibe iumination conditions? Int. J. Comput. Vision 28, (1998). 10. R. Epstein, P. W. Hainan, and A. Yuie, 5 pus or minus 2 eigenimages suffice: an empirica investigation of owdimensiona ighting modes, in IEEE Workshop on Physics-Based Modeing in Computer Vision (IEEE Computer Society, Los Aamitos, Caif., 1995), pp R. Basri and D. Jacobs, Lambertian refectance and inear subspaces, Tech. Rep R (NEC Research Institute, Princeton, N.J., 2000).

12 R. Ramamoorthi and P. Hanrahan Vo. 18, No. 10/October 2001/J. Opt. Soc. Am. A A. Gershun, The ight fied, J. Math. Phys. 18, (1939) (transated by P. Moon and G. Timoshenko). 13. R. McCuney, Introduction to Radiometry and Photometry (Artech House, Norwood, Mass., 1994). 14. S. Gorter, R. Grzeszczuk, R. Szeiski, and M. Cohen, The umigraph, in SIGGRAPH 96, Computer Graphics Proceedings, Annua Conference Series, H. Rushmeier, ed. (Association for Computing Machinery, New York, 1996), pp M. Levoy and P. Hanrahan, Light fied rendering, in SIG- GRAPH 96, Computer Graphics Proceedings, Annua Conference Series, H. Rushmeier, ed. (Association for Computing Machinery, New York, 1996), pp D. Wood, D. Azuma, K. Adinger, B. Curess, T. Duchamp, D. Saesin, and W. Stuetze, Surface ight fieds for 3D photography, in SIGGRAPH 2000, Computer Graphics Proceedings, Annua Conference Series, K. Akeey, ed. (Association for Computing Machinery, New York, 2000), pp R. Ramamoorthi and P. Hanrahan, Anaysis of panar ight fieds from homogeneous convex curved surfaces under distant iumination, panarf. 18. T. Inui, Y. Tanabe, and Y. Onodera, Group Theory and its Appications in Physics (Springer Verag, New York, 1990). 19. J. D. Jackson, Cassica Eectrodynamics (Wiey, New York, 1975). 20. T. M. MacRobert, Spherica Harmonics; An Eementary Treatise on Harmonic Functions, with Appications (Dover, New York, 1948). 21. M. D Zmura, Shading ambiguity: refectance and iumination, in Computationa Modes of Visua Processing, M. S. Landy and J. A. Movshon, eds. (MIT Press, Cambridge, Mass. 1991), pp B. Cabra, N. Max, and R. Springmeyer, Bidirectiona refection functions from surface bump maps, Comput. Graph. 21(4), (1987). 23. S. Westin, J. Arvo, and K. Torrance, Predicting refectance functions from compex surfaces, Comput. Graph. 26(2), (1992). 24. F. Siion, J. Arvo, S. Westin, and D. Greenberg, A goba iumination soution for genera refectance distributions, Comput. Graph. 25(4), (1991). 25. J. Koenderink and A. van Doorn, Phenomenoogica description of bidirectiona surface refection, J. Opt. Soc. Am. A 15, (1998).