Normal Mapping for Precomputed Radiance Transfer

Transcription

1 Norma Mapping for Precomputed Radiance Transfer Peter-Pike Soan Microsoft Corporation Figure : Comparison of PRT, and four variants of norma mapping: God Standard, Separabe, af-life basis, Shifted Associated Legendre Poynomias Abstract Norma mapping is a variant of bump mapping that is commony used in computer games. It modes compex surface variations by expicity storing a surface norma in a texture map. owever, it has not been used with precomputed radiance transfer (PRT), a technique for modeing an object s response to a parameterized mode of ighting, which enabes interactive rendering of compex goba iumination effects such as soft shadows and interrefections. This paper presents severa techniques that effectivey combine norma mapping and precomputed radiance transfer for rigid objects. In particuar, it investigates representing the convoved radiance function in various bases and borrowing concepts from the separabe decomposition of BRDF s. Keywords: Graphics ardware, Iumination, Norma Mapping, Rendering, Shadow Agorithms. Introduction Generating reaistic images of compex scenes at interactive rates is a chaenging probem. Games traditionay have used a combination of static ight maps and mutipe dynamic point or directiona ight sources. The surfaces generay are textured with both refectance properties and norma maps [Binn 978; Peercy et a. 997] that approximate compex surface detais. These textures tend to be samped at high spatia samping rates and are often ayered and tied to create even higher effective samping rates. In contrast, precomputed ighting techniques require unique representations of a signa and are stored at much ower samping rates. It is impractica to sampe these unique detais at the effective composite samping rate. A recent technique, precomputed radiance transfer [Soan et a. ], enabes compex goba iumination effects ike soft shadows and inter-refections to be rendered in rea time for rigid objects under dynamic distant ighting. Whie a technique has been proposed to mode these higher frequencies [Soan et a. 3b], it is too heavyweight for computer games. Norma maps have been successfuy integrated with goba iumination in both off-ine production [Tabeion and Lamorette 4] and interactive games [McTaggart 4] under static goba ighting. Inspired by this work, we propose severa simpe techniques that extend diffuse PRT for rigid objects to hande norma maps. They are ightweight, and they ony mode goba iumination effects at coarse scaes, negecting fine scae shadowing of the surface detais.. Background and Reated Work Static goba iumination and norma mapping have been successfuy integrated both off-ine [Tabeion and Lamorette 4] and in the recent game af-life [McTaggart 4]. Our work is heaviy infuenced by the atter paper, which is effectivey a vector irradiance formuation of radiosity that makes norma mapping effective. We sha investigate severa hemispherica bases for representing parameterized goba iumination, and incude a nove basis based on concepts used for representing BRDF s [Kautz and McCoo 999].. Precomputed Radiance Transfer PRT [Soan et a. ] enabes interactive rendering of compex goba iumination effects for rigid objects. For competeness, we briefy describe PRT and discuss why previous work does not adequatey address norma mapping. In this paper scaar quantities and ow dimensiona vectors are denoted in itaics, matrices with upper case bod etters and vectors with ower case bod etters. The genera mathematica form of PRT is: ( ) ( v) T ev b RM ere e(v) is outgoing radiance, b(v) is a vector that represents the materias response to ighting in view direction v expressed in a oca coordinate system; R is a matrix that rotates ighting into the oca frame; and M is a matrix that maps distant ighting to transferred incident radiance. For diffuse surfaces, a singe transfer vector t has been used that directy modes the reationship between distant ighting and outgoing radiance. These spatiay varying operators are computed using sighty modified off-ine goba iumination techniques; for detais see [Soan et a. ;Soan et a. 3a;Lehtinen 4]. These formuations have fine surface detais burned into the mode of transfer, which precudes their use with norma mapping. One exception to this is the bi-scae radiance transfer technique [Soan et a. 3b], which, ike this work, modes coarse effects using operators that map from distant ighting to smooth incident radiance. owever, in that work the fine scae was modeed using a heavyweight representation that modeed both masking effects and arbitrary materias. We represent convoved incident radiance with a more compact basis which makes our technique much ighter weight, but can ony hande diffuse materias and does not mode shading effects at the fine scaes of the norma map. Loca, Deformabe Precomputed Radiance Transfer (LDPRT) [Soan et a. 5] modes ony fine

2 scae transfer effects for deformabe objects, whie this work modes ony coarse scae effects for rigid objects.. Spherica armonics Spherica harmonics are the spherica anaog of the fourier basis on the unit circe and have been used extensivey in computer graphics. In this paper they are used to represent both the distant ighting environment and the convoved oca ighting environment. The genera form is: K e P m m (, ) m im ϕ m θϕ (cos θ), N, where P m are the associated Legendre poynomias and K m are the normaization constants K m ( + ) ( m)! 4 π ( + m)! The above definition forms a compex basis; a rea-vaued basis is given by the simpe transformation m m m Re( ), m> K cos( mϕ) P (cos θ), m> y Im( ), m< K sin( mϕ) P (cos θ), m< m m m m θ, m K P (cos ), m Low vaues of (caed the band index) represent ow-frequency basis functions over the sphere. The basis functions for band reduce to poynomias of order in x, y, and z. Spherica harmonics can ony efficienty represent smooth ighting environments. Smooth ighting environments induce ow spatia samping rates, making them more practica for appications ike computer games..3 Irradiance Environment Maps An irradiance environment map [Ramamoorthi and anrahan ] enabes interactive rendering of diffuse objects under any iumination environment without shadowing. This is done by representing the ighting environment and the normaized cosine kerne directy in spherica harmonics. Convoution can be performed efficienty in this space, and ony the first 9 terms of the convoved ighting environment are needed to represent it accuratey. Irradiance environment maps are used in this paper to efficienty decoupe norma variation from ighting variation. 3. Norma Mapping for PRT At any point on the surface of an object a transfer matrix represents how distant ighting expressed in some ighting basis (spherica harmonics for this work) maps to transferred incident radiance. Even if the object has compex norma variation, this incident radiance is a smooth function, whie the outgoing or exit radiance has much more variation. Decouping norma variation from incident radiance, makes practica rendering objects with compex surface variation. The straightforward way to address this for diffuse objects is: expoit the observation in other papers [Ramamoorthi and anrahan ; Basri and Jacobs ], and have transfer matrices that project into quadratic spherica harmonics foowed by a convoution with the camped cosine kerne. Then, interpoate irradiance at some coarse scae, and simpy evauate Irradiance needs to be converted to outgoing radiance to be visuaized. This can be done by convoving, using a normaized cosine kerne (divide by Pi) and keeping the abedo in [,]. this using a norma, ooked up in a norma map. This can be expressed mathematicay as foows: e n (3.) ( ) ( ) T y n CRM ere, e(n) is the outgoing radiance as a function of the surface norma n; y(n) is a vector generated by evauating the quadratic spherica harmonic basis functions in the norma direction; C is a diagona matrix that convoves quadratic spherica harmonics with the normaized cosine kerne; R is a spherica harmonic rotation matrix that rotates a quadratic S function into the oca frame; and M is a transfer matrix that maps the varying distant ighting of some order into quadratic oca ighting. Unfortunatey this is sti too heavyweight; in particuar, 7 numbers are required to represent the irradiance environment map. If this is computed at the vertices of the mesh, 7 of the 8 interpoators have to be consumed for this aone. The rest of this paper is focused on deaing with efficient approximations to the above expressions. 3. Projection into emispherica Basis One possibe soution that we wi briefy discuss here is to simpy project the irradiance environment map into a hemispherica basis. Two obvious choices are the basis used in the paper [McTaggart 4], which consist of three camped inear basis functions and the basis used in the paper [Tabeion and Lamorette 4] which is effectivey determined from the oca inear spherica harmonic ighting coefficients but does not incude the DC term. In this paper we investigate using the af-life basis. In the GDC presentation, shader code was presented with and without camping vaues to zero. For PRT, not camping produced better visua resuts and that is what is used in the paper. The three basis functions are: b,, 6 3 b,, 3 3 b,, 6 3 They are orthogona when integrated over the hemisphere, and can be normaized by scaing by 3 ( π ). Another basis choice is the one proposed in Gautron s paper [Gautron 4]. This is an orthogona basis generated by appying the shifting theorem to the spherica harmonic basis. These basis functions can be expressed based on the Eucidean coordinates of a point on the unit hemisphere. The first four basis functions are: π ( z ) z z y π z z z x π π z These functions are ceary not poynomias except for the L basis functions, but can be evauated efficienty using shaders. When z the imit of the functions with z in the denominator The dominant ighting direction that is referred to in this paper can be determined from the inear S coefficients. owever, the cosine term woud have to be negected to mode the same resut (it is ater added in using the actua surface norma).

3 equas, x/sqrt(-z ) cos(phi), y/sqrt(- z ) sin(phi), and so the functions are defined over the compete hemisphere. Projection matrices from spherica harmonics for both bases are in the appendices. 3. Separabe Approximation An aternative approach is to use the concepts from separabe BRDF factorization [Kautz and McCoo 999] to buid a minima speciaized basis. Separabe approximations have been used with PRT to mode gossy surfaces [Liu et a. 4;Wang et a. 4]. The ighting is treated the same way, but in this case norma variation in tangent space for diffuse surfaces is being modeed instead of view variation for arbitrary materias. As in the gossy case, the goa is to reduce the dimensionaity of the matrix, reducing both the size of the dataset and the amount of computation required in the shaders. Moving to a directiona basis, Equation 3. can be re-written as: e n (3.) ( ) ( ) T b n ACRM Where b(n) is a vector of coefficients for bi-inear basis functions on the unit square mapped to the hemisphere [Shirey and Chiu 997], which has at most 4 non-zero vaues for a given norma. Coefficient a ij of the matrix A represents evauating the quadratic spherica harmonic basis function j in norma direction i. In this paper 4 discrete norma directions are used (3x3 sampes on the unit square) so this matrix is 4x9. Computing the singuar vaue decomposition [Press et a. 99] of A, factors the matrix into a product of 3 matrices USV, where U is a 4x9 orthogona matrix, S is a 9x9 diagona matrix and V is a 9x9 orthogona matrix. Using the first M singuar vaues, truncates the matrix U to have M coumns and the matrix V M rows. This eaves the fina approximation: T ( ) (3.3) e( n) b( n) U S V CRM m m m The term in parentheses is an M dimensiona row vector that is a function of the surface norma. This can be efficienty evauated using texturing hardware by packing the coumns of U m into individua coor channes of a texture, and, if M>4, into mutipe textures. In this paper we used M4, and instead of using a texture that computes the hemisphere to unit square mapping (or compex shader code), we resampe the coumns of U m into a texture parameterized by an orthographic projection of the norma vector (ignoring z in tangent space) which is represented at a higher resoution (64x64.) This texture is then samped directy..% 8.% 6.% 4.%.%.% Figure : Accuracy vs. number of singuar vaues The figures in the paper use 4 rows, requiring 3 interpoants to be passed from the vertex shader to the pixe shader. If the convoution matrix C is absorbed into the matrix A we beieve the SVD wi resut in higher accuracy. 4. Rendering Shading is straightforward using these techniques. In every frame, the transfer matrices at the vertices have to be mutipied by the distant ighting environment. Then the coefficients for the correct basis are passed down to the pixe shaders, where the current norma is ooked up in the norma map and used to evauate the particuar basis. The separabe approximation is more invoved, since the norma dependent textures have to be dereferenced as we. The shader code beow is the pixe shader when M4. The vertex shader passes down the texture coordinates and three registers that contain the basis coefficients. There are three textures that are samped: the norma dependent texture, the norma map, and the abedo of the surface. foat4 StandardSVDPS( VS_OUTPUT_SVD In ) { foat4 RGBCoor(,,,); // sampe abedo/norma map foat4 vabedo texd(abedotex, In.uv); foat4 vnorma texd(normtex, In.uv); } // sampe norma dependent texture foat4 vu texd(usamper,vnorma); // compute irradiance RGBCoor.r dot(in.cr,vu); RGBCoor.g dot(in.cg,vu); RGBCoor.b dot(in.cb,vu); // scae by abedo and return return RGBCoor*vAbedo; The transfer matrices can be compressed using CPCA [Soan et a. 3a] shifting more of the workoad to the GPU, decreasing the amount of data and increasing performance. 5. Resuts We have impemented a of the techniques in the paper. Tabe presents a synopsis of the resuts for the two modes shown in the paper. The precomputation of the transfer matrices is essentiay the same as in earier papers [Soan et a. ; Soan et a. 3b] and projection into either the ight speciaized separabe or the anaytic basis is a simpe matrix mutipy. Scene #f #v GS Sep SAL L GPUGS GPUSep Simpe Compex Tabe : Performance resuts (fps), GS: God Standard, Sep: Separabe, SAL: Shifted Associated Legendre, L: af-life basis, GPUGS: God Standard+PCA, GPUSep: Separabe+PCA Visuay a the techniques have reasonabe fideity, the shifted associated egendre poynomias subjectivey ooks the worst. Not much time has been spent optimizing the CPCA parameters for these datasets; in fact, the exampes are a reported with the simpe parameters of a singe custer and 4 PCA basis vectors. A of these exampes shoud require fewer custers compared to what was used in previous work [Soan et a. 3a], because the transfer matrices represent a signa that has been convoved with a normaized cosine obe and have fewer rows. A timings are in

4 frames per second and were recorded on a.ghz AMD Opteron with a nvidia GeForce 68 graphics card. Figure shows a comparison on the compex mode between conventiona PRT and these norma mapping variants. Note the compex detais on the side of the object that are competey encoded in the norma map. Figure 3 shows three norma maps mapped onto the simpe object rendered with PRT and a these techniques. Note the differences on the bumpy object. 6. Concusions Norma mapping is a compeing technique that has been used with anaytic ighting and with static precomputed ighting. We extend these techniques to practica impementations for more genera precomputed ighting techniques, and investigate a famiy of soutions that trade off cost with accuracy. Of the three techniques presented, the anaytic basis used in the paper [McTaggart 4] and the separabe basis with a four term approximation seem the most promising. Whie oca transfer is negected with the presented techniques, ideas from ambient occusion can be used to generate a crude approximation. In particuar, the product of the ighting environment and the DC approximation to the visibiity function, when projected into spherica harmonics, resuts in simpy scaing the ighting environment by this DC term. This turns out to be a mathematica justification for a common ambient occusion technique. In the future it is worth investigating compression of the transfer matrices more carefuy, and possiby integrating a more sophisticated mode of oca transfer whie remaining ightweight enough to be practica for game appications. Aso, foding the convoution into the A matrix coud ead to better resuts. 7. Acknowedgements The Drone mode and textures were created by Shannon Drone, and the norma maps were created by Ben Luna. I woud ike to thank James Soan for editing and the anonymous reviewers for their hepfu comments. References BASRI, R., AND JACOBS, D. Lambertian Refectance and Linear Subspaces, ICCV. BLINN, J. Simuation of Wrinked Surfaces. SIGGRAP 978. GAUTRON, P., KRIVANEK, K. PATTANAIK, S., AND BOUATOUC K. A Nove emispherica Basis for Accurate and Efficient Rendering, Eurographics Symposium on Rendering 4. KAUTZ J., AND MCCOOL, M. Interactive Rendering with Arbitrary BRDFs Using Separabe Approximations. Eurographics Workshop on Rendering 999. LETINEN, J. Foundations of Precomputed Radiance Transfer, Master s Thesis, esinki University of Technoogy, September 4. LIU, X., SLOAN, P., SUM,., AND SNDER, J. A-Frequency Precomputed Radiance Transfer for gossy objects. Eurographics Symposium on Rendering 4. MCTAGGART, G. af-life Source Shading, GDC 4. PEERC, M., AIRE, J., AND CABRAL, B. Efficient Bump Mapping ardware, SIGGRAP 997 PRESS, W., TEUKOLSK, S., VETTERLING, W., AND FLANNER, B. 99. Numerica Recipes in C, Second Edition. Cambridge University Press. RAMAMOORTI, R., AND ANRAAN,. An Efficient Representation for Irradiance Environment Maps, SIGGRAP. SIRLE, P, AND CIU, K, A Low Distortion Map between Disk and Square, Journa of Graphics Toos, vo., no. 3, 997, SLOAN, P., KAUTZ, J., AND SNDER, J. Precomputed Radiance Transfer for Rea-Time Rendering in Dynamic, Low-Frequency Lighting Environ-ments, SIGGRAP. SLOAN, P,. ALL, J., ART, J., AND SNDER, J. Custered Principa Components for Precomputed Radiance Transfer, SIGGRAP 3. SLOAN, P., LIU, X., SUM,., AND SNDER J. Bi-Scae Radiance Transfer, SIGGRAP 3. SLOAN, P, LUNA, B, AND SNDER J, Loca, Deformabe Precomputed Radiance Transfer, SIGGRAP 5. TABELLION, E., AND LAMORLETTE, A. An Approximate Goba Iumination System for Computer-Generated Fims, SIGGRAP 4 WANG, R., TRAN, J., AND LUEBKE, D. A-Frequency Reighting of Non-Diffuse Objects Using Separabe BRDF Approximation, Eurographics Sumposium on Rendering Appendix: Other Basis Functions 7. Shifted Associated Legendre Poynomias The matrix beow projects the spherica harmonics into the first four shifted associated egendre poynomias in cosed form. Where: a 6 4 a b a b 44 9 n ( 3+ ) ( ), b 3 7. af-life Basis 39 n The quadratic spherica harmonics can be projected into this basis using the foowing matrix: π 4 π π 4 π 3 π 6 π 3 π π 4 π π 4 π 3 π 6 π 3 π π π π 6 π 6 π 7.3 Numerica Comparison Beow is a tabe that shows the mean squared error (MSE) integrated over the hemisphere when projecting each of the quadratic spherica harmonic basis functions into these two bases. If the corresponding basis function is in the nu space of the basis, the MSE equas.5, and as shown beow basis functions and are in the nu space of both basis. It is aso worth pointing out that those two basis functions are aso in the nu space of the first four SVD coefficients; the 5 th and 6 th coefficients are exacty those basis functions. SAL 4.6e- 4.6e- 5e- 4.5e- 3.e- 4.5e- 5e- L.3e- 5e-.5e- 3.8e-.5e- 5e- Tabe : Mean squared error integrated over hemisphere. SAL: Shifted Associated Legendre, L: af-life basis.

5 Figure 3: Simpe scene, PRT, God Standard, Separabe, af-life, Shifted Associated Legendre Poynomias