Normal Map Industry Survey

Transcription

1 Normal Map Industry Survey

2 Today s Schedule 10:00am SEGMENT 0: Adam Myhill with introduction 10:10am SEGMENT 1: Zbrush (Scott Spencer ) 11:10am SEGMENT 2: (Gio Nakpil and Rich Diamant) 12:10pm BREAK Lunch (1 hour) 1:10pm SEGMENT 3: Normal Map Implementation Buffet 2:50pm BREAK- 10 minute 3:00pm SEGMENT 4: Texture-based Normal Map Sampling Survey (Ricardo Ariza) 4:00pm SEGMENT 5: Real World Data Sampling with Steve Chapman 4:50pm BREAK- 10 minute 5:00pm SEGMENT 6: Issues with normal maps in the major consoles (Adam Myhill) Ends at 6pm

3 What are Normal Maps? Brief history of normal maps While this idea of taking geometric details from a high resolution model had been introduced in "Fitting Smooth Surfaces to Dense Polygon Meshes" by Krishnamurthy and Levoy, Proc. SIGGRAPH 1996, where this approach was used for creating displacement maps over nurbs, its application to more common triangle meshes came later In 1998 two papers were presented with the idea of transferring details as normal maps from high to low poly meshes: "Appearance Preserving Simplification", by Cohen et al. SIGGRAPH 1998, and "A general method for recovering attribute values on simplified meshes" by Cignoni et al. 10 year anniversary!

4 What are Normal Maps? Overview of how they work Normal mapping is sometimes referred to as "Dot3 bump mapping". While bump mapping perturbs the existing normal (the way the surface is facing) of a model, normal mapping replaces the normal entirely. - Wikipedia Each colour channel of the normal map represents an 8 bit peterb or bending of the pixel normal on an axis, one axis for each R, G, B chnnels

5 What are Normal Maps? Strengths, weaknesses, common misconceptions Positive Relatively inexpensive way to represent highly detailed surfaces Allows for much lighter meshes Easier to weight + rig Faster to animate Moves computation from CPU to GPU (typically)

6 What are Normal Maps? Strengths, weaknesses, common misconceptions Negative Doesn t do anything for silhouettes Only good for high and mid frequency detail Doesn t animate (without a bunch more work...) Often overly used - inefficient asymmetry between geo and texture DXTN textures are much larger than DXT1 Moves computation from CPU to GPU (typically)

7 What are Normal Maps? Overhead: Instruction increase / shader complexity Texture budgets: Strive to balance normal map / geo density Lighting: Normal maps are only as good as the lighting on them Complex ambient: Irradiance / Spherical Harmonics Decent specular contributions! Creating them takes time: High res model Low res model Rendering / Photoshop blending / cleanup

8 What are Normal Maps? Introduce our speakers and organizers Scott Spencer: Art Director at Gentle Giant Studios Gio Nakpil: 3D Modeler at ILM Rich Diamant: Lead Character Artist at Naughty Dog Ricardo Ariza: Senior Character Artist and Naughty Dog Adam Myhill: Sr. Technical Artist at Pandemic Brisbane Steve Chapman: VP of Gentle Giant Studios Alex Velazquez: Character Modeler at Ravensoft

9 Segment 1: Zbrush ZBrush Creature and Character Design: A Traditional Approach In this section I will discuss using ZBrush s digital sculpting tools to create creature and character models. When I sculpt digitally I try to focus on the same foundations laid by traditional sculptors. When working I try and be aware of gesture and form first leaving the details for later. For this demonstration I decided to use a traditional clay sculpture which was scanned into a digital model as a base. This allows me to also cover ways of bringing scan-data into ZBrush as well as illustrate the same techniques applied in two different mediums.

10 Work Samples Segment 1: Zbrush

11 Work Samples Segment 1: Zbrush

12 Digital Sculpting Segment 1: Zbrush

13 Digital Sculpting Segment 1: Zbrush

14 Segment 1: Zbrush An organized approach to sculpting: Form Analysis

15 Light makes form Segment 1: Zbrush

16 Segment 1: Zbrush ZBrush s default materials do not allow for interactive lighting. By changing to the Basic Material You can move the light as you work and get a much clearer idea of the form relationships and transitions Using the flat color shader to check the silhouette. This can be easily be hot keyed allowing you to quickly switch between flat shade and shaded mode. A useful tip is to export the mid subdivision level to Maya where it can be imported as an obj, lit, and checked under a different perspective camera than ZBrush s.

17 Segment 1: Zbrush Sculpting with rakes Rakes are a real world sculpting tools used for the rapid development of form by subtracting from the clay surface

18 Segment 1: Zbrush In ZBrush the rake brush can be used to both add or subtract form building shapes in a rapid and gestural manner

19 Segment 1: Zbrush The Shade: From clay to game mesh

20

21

22 Segment 1: Zbrush The Ogre in ZBrush The Ogre sculpture is scanned and brought into ZBrush to become a ZTool

23 Segment 1: Zbrush Importing the sculpture scan into ZBrush as a Ztool Orient the mesh in Maya to scans are inherently asymmetrical so this is not a perfect process but enough to allow you to use many tools in symmetry mode. In ZBrush use the Transpose tools to center pose in preparation for remeshing. Scan data must be remeshed as it consists of random triangles and not suitable for sculpting or animation Remeshing in ZBrush using topology tools. The drawback to this is the inability to create a mouth bag or eyes there is however a workaround for this we will discuss later

24 Segment 1: Zbrush Customizing the ZBrush brushes Flattening the edit curve Brush mod and Smoothing Curves Isolating details on layers and removing noise

25 Segment 1: Zbrush Major design changes can be made to an existing sculpture through the use of mesh Insert and the clay brushes. Here a horn has been added to the design sculpt and blended into a single mesh. This process will require new topology to be built and details transferred

26 Segment 1: Zbrush Displacement Map Detail Transfer Eye sockets and mouth bags can be added once the sculpture is completed. by using a process called Displacement map detail transfer. This process uses a 16bit displacement map in ZBrush to transfer details from one mesh to another. This technique can be more predictable than using the ZBrush projection tools under Topology And Subtools.

27 Zmapper for Normal Maps Segment 1: Zbrush

28 Segment 1: Zbrush ZMapper can also be used to generate normal maps between arbitrary meshes

29 Segment 1: Zbrush The same process can be applied to this scan of actor Brad Pitt

30 Personal Works Segment 2: Mudbox

31 Segment 2: Mudbox

46 Segment 2: Mudbox A sculptural approach to digital creature creation Hierarchy of forms Primary forms Secondary forms Tertiary forms

47 Segment 2: Mudbox Primary forms The most prominent forms that define the character Forms were all the other sub-forms (secondary and tertiary) will rest on

48 Segment 2: Mudbox Secondary forms Forms that add complexity to primary forms Should always compliment primary forms rather than overpower them From a distance, secondary forms should be the least prominent when compared to primary forms

49 Segment 2: Mudbox Tertiary forms Sub-forms that add complexity to secondary forms (ie. Wrinkles) The least important of the forms The most overly used in digital sculpting When used properly however, these forms define the quality and texture of the surface From a distance, tertiary forms should be the least prominent of forms The cherry on top, The cake can do without it, but sure looks good with it (if used properly)

50 Segment 2: Mudbox Planes (Light and Shadow) Faceted forms that define how light and shadow hits the surface of the sculpture Gives sculpture depth and contrast

51 Segment 2: Mudbox Normal Mapping inside of Mudbox: Limitations Creating your normal map

52 Segment 3: 3DS Max Material Editor Under Maps select the Bump Slot. Select Normal Bump from the Material/Map Browser.

53 Segment 3: 3DS Max Normal Bump Select the Normal Slot Select Bitmap from the Material/Map Browser This will open a file browser for you to select your normal map.

54 Segment 3: 3DS Max Direct X Manager. Toggle on the DX display of Standard Material. Select the Show Map in viewport button. You can now view the normal map in any shaded viewport. Perspective will show the model with specular settings. Toggle of the DX display off as it slows down your scene, especially when using undo. The material is still viewable in the renderer.

55 Segment 3: Maya Normal Mapping inside of Maya: Limitations Using Transfer Maps for normal mapping Preventing errors Viewing normal maps in Real-time and rendering normal maps

56 Segment 3: XSI Overview of the XSI workflow Reference photography Specular extraction Z-Brush XSI workflow Photoshop cleanup / tips and tricks

57 Segment 3: XSI Lens compression: Get back as far as you can and zoom in. At least 70mm (35mm equiv) Lighting: Try to get it as flat and even as possible. Overcast days with a bounce card is great Head position: Tilt back and close eyes for a shot, so you get the under nose and chin information Get a shot of eyes closed, model and texture head with eyelids closed then animate open (otherwise your blinks will look lame)

58 Segment 3: XSI Reference Photography Not ideal, but the best under the circumstances... Pretty decent flat lighting

59 Segment 3: XSI Reference photography Specular extraction through differential cross polorization: How to create perfect specmaps and high frequency normal map detail

60 Segment 3: XSI Reference photography Specular extraction through differential cross polorization: How to create perfect specmaps and high frequency normal map detail

61 Segment 3: XSI Specular extraction through differential cross polarization

63 Segment 3: XSI Specular extraction through differential cross polorization 1. Get a polarizing filter + tripod 2. Sit your character down and face them into the afternoon sun 3. Tripod your camera at 90* to sun angle 4. Rotate polarizer until you see max sheen and max specular block 5. Mark angles on filter, they ll be 90* apart 6. Shoot one at open, quickly rotate 90* to closed shoot again 7. Bring into Photoshop, stabilize images over top of each other 8. Do a difference multiply function 9. Desaturate and add contrast 10.Voila! Perfect specular capture!!

65 Segment 3: XSI Zbrush Amazing 5 minutes from Z-Brush to XSI video

66 Segment 3: XSI XSI Amazing 7 minute XSI workflow video

67 Segment 3: XSI Goals: Clean rendertree interface: Only display useful nodes Properly display normal maps in realtime viewport Display requirements: bones all helper objects / nulls wireframes camera safe guides DXT9/10 can t do this! Need to use OGL XSI doesn t have a shader which can do this...

68 Segment 3: XSI

69

70 Nvidia FX composer CgFX shader

71 Segment 4: Texture sampling

79 Segment 4: Texture sampling Adobe Photoshop NVIDIA Normal Map Filter Plug-in Map Preview Normalize Feature Furthering Depth and Detail Information Authoring DDS Mip Maps

80 The Interface

81 Nvidia Plugin Demo

82 Segment 4: Texture sampling Adobe Photoshop NVIDIA Normal Map Filter Pros Pulls shallow and moderate details efficiently from diffuse maps Normalize feature fixes any inverted normal information Cons Unintuitive interface Limited range of information can be extracted from maps Limited preview capabilities Quick preview of maps to check depth information

83 Segment 4: Texture sampling Crazybump Loading & Exporting Multiple Maps Updating Pre-Existing Maps Refining Map Details Creating Various Maps from a Single Texture

84 The Interface

85 Crazybump Demo

86 Segment 4: Texture sampling Crazybump Pros Speed, Ease of use and efficiency Ability to load and view multiple maps on an object Pre-existing maps can be loaded and modified Layering of normal maps Real time updating on map changes on primitives or imported geometry Removes shadows and highlights and does color correction on diffuse maps Real time, tileable texture preview Creates multiple maps from single source Normal maps, Displacement, Occlusion, Specular Cons Inability to Paint Information Artifacting can be accentuated Clipping of image levels can occur if sliders are pushed too far

87 Segment 4: Texture sampling Illuminate Labs Turtle 3d normal mapping Texture baking Surface transfer Radiosity normal maps Edge Dilation

88 The Interface

89 Segment 4: Texture sampling Illuminate Labs Turtle Pros Ability to work with 3d objects for normal mapping Cons Leaves dirty nodes in scene Surface transfer of normal maps and diffuse maps Radiosity (occlusion) normal maps Ignores inconsistent normal's

90 Segment 4: Texture sampling Zbrush creating Tileable Textures Importing Height Map Reworking Details into Height Map Using Layers Pushing & Pulling Information from Layers Painting Textures Exporting Normal, Diffuse and Height Information

91 Zbrush Demo

92 Segment 4: Texture sampling Zbrush Tileable Textures Pros Most versatile All information contained in single file Ability to consistently edit file High depth limit in terms of Normals Works seamlessly with Photoshop (or any other software that supports.psd s) Ability to import and use 3d objects Cons High learning curve Slowest process Inability to link and offset multiple layers

93 Segment 5: Data Capture Star Wars Pirates of The Caribbean The Matrix Harry Potter Chronicles of Narnia The Mummy Spider-Man Jumper Indy 4 Ratatouille The Kingdom Avatar The Dark Knight Chronicles of Narnia Beowulf Deja Vu The Hulk Ghost Rider

94 Segment 5: Data Capture Rapid asset acquisition with 3D scans

95 Segment 5: Data Capture Rapid asset acquisition with 3D scans Gear and technology survey

96 Segment 5: Data Capture Gear and technology survey Human body scanning - Cyberware WB4

97 Segment 5: Data Capture Gear and technology survey Human body scanning - Inspeck Capturer

98 Segment 5: Data Capture Gear and technology survey Human body scanning - Homebrew body scanner

99 Segment 5: Data Capture Human body scanning - Processing data Remove carbon deposits

100 Segment 5: Data Capture Human body scanning - Processing data process, dewarp, & stack waterlines

101 Segment 5: Data Capture Human body scanning - Processing data process, dewarp, & stack waterlines

102 Segment 5: Data Capture Human body scanning - Processing data Preliminary polygonization

103 Segment 5: Data Capture Human body scanning - Processing data cleaned polygonization

104 Segment 5: Data Capture Gear and technology survey Human head scanning - Cyberware PS

105 Segment 5: Data Capture Gear and technology survey Human head scanning - Cyberware PS

106 Segment 5: Data Capture Gear and technology survey Keep accurate records

107 Segment 5: Data Capture Capture multiple expressions

108 Segment 5: Data Capture Capture multiple expressions

109 Segment 5: Data Capture Spy on the ILM crew during downtime

110 Cleanup raw data Segment 5: Data Capture

115 Merge head and body Segment 5: Data Capture

116 Plan animation layout Segment 5: Data Capture

117 Plan animation layout Segment 5: Data Capture

118 Hang with the talent Segment 5: Data Capture

119 Segment 5: Data Capture Why the need to take separate texture photos

120 Segment 5: Data Capture Various object scanners - Polhemus Cobra

121 Segment 5: Data Capture Various object scanners - Polhemus Cobra

122 Segment 5: Data Capture Various object scanners - Gom ATOS

123 Segment 5: Data Capture Various object scanners - Gom ATOS

124 Segment 5: Data Capture Various object scanners - Cyberware 3030

125 Segment 5: Data Capture Various object scanners - Mensi Soisic

126 Segment 5: Data Capture Various object scanners - Mensi Soisic

127 Segment 5: Data Capture Various object scanners - Trimble Lidar

135 Segment 5: Data Capture Processing to low res proxy for normal map extraction

143 Segment 5: Data Capture

147 Segment 6: Issues + Tricks Overview Issues with normal maps on the PS3 and Xbox 360 Tips tricks and optimizationsto get the best results Do you even need normal maps? Derivative generation

148 Segment 6: Issues + Tricks Efficient balance of normal maps VS geometry The costs associated: Optimizing your resources - find the balance between geo and texture Inefficient layouts and model design sap quality A single 512 normal map takes approx the same memory as polygons!

149 Segment 6: Issues + Tricks Memory breakdown: 1 x 512 normal map = over 2400 polygons (memory) 1 dual UV channel Vertex = 36 bytes (and that s horrible efficiency) So would you rather have polys or a 512 normal map? You need to optimize your CPU / GPU loading: Weighted models, static models, poly density, normal maps, everything else... Only by tuning your art into this performance sweet spot, will you maximize your visuals

150 Segment 6: Issues + Tricks Tangent decoupling: Normal map model render discontinuity needs to be the same as the model export discontinuity angle, otherwise render artifacts can occur. IE Rip normal map at 100*, but export model at 60* Exporting models: XSI dotxsi v3.6 exporter messes with your model s normals

151 Segment 6: Issues + Tricks Improper DXT compression on normal maps: The difference between DXT1 and DXTN: DXT1 uses a 565 colour scheme: 5 bits to red and blue, 6 to green DXTN uses equal RGB bias Humans are green sensitive, it s the middle frequency of our visible spectrum. Most digital cameras have more green sensors How to notice improper DXT compression on normal maps: Strange banding and pixelization on top and lateral angles Mushyness Extra memory!

152 Segment 6: Issues + Tricks Other common problems: World VS object tangent space issues Typically it s world tangent space on a dynamic object Mirroring normal maps Seams Running out of memory!

153 Segment 6: Issues + Tricks Tips tricks and optimizations: Realtime tangent preview You can find and determine bugs easier You can tell if seams are a model or texture issue (or both)

154 Segment 6: Issues + Tricks

157

158

159

160

161 Segment 6: Issues + Tricks Dealing with seams Typical problems are: Texture seams: Paint 127 red or clone seam boundaries Mirroring seams: How do you get seamless results on mirrored normal maps? Break your model down the seam Allow for your tangents to be 180* apart Align your verts back together and ensure matched weighting Paint 127 red down your normal map seam BONUS: Process your geo normals to be totally perpendicular down the X axis for all boundaries

162 Segment 6: Issues + Tricks Tips tricks and optimizations Texel density consistency: Mip map displays Do tests where you paint your mips bold colours: High Red, medium orange, low blue, etc. Check to see how much you see the high. You ll be surprised. Expose mip map bias controls: There is a lot of performance and visual quality tuning here! Push out till it sparkles and then pull back in 4X anisotropic filtering really helps here. Pay the 2-3ms it is so incredibly worth it, and not just for your normal maps

163 Segment 6: Issues + Tricks Lighting! So incredibly important Spherical Harmonics / Irradiance lighting Realtime specular Hybrid lighting scemes MRT rendering Normal support within framebuffer

164 Segment 6: Issues + Tricks Lighting! So incredibly important

165 Segment 6: Issues + Tricks Lighting! So incredibly important

166 Segment 6: Issues + Tricks Tips tricks and optimizations: Fading off normal maps in an LOD scheme, often costs more than just leaving them on all the time Nvidia photoshop tools work best on collapsed PSD (does not work great on layered info) Collapse all nm layers and run a normalize pass on it to ensure all colors fall within normal map values.

167 Segment 6: Issues + Tricks After creating Nvidia detail pass texture to overlay on your baked normals, neutralize the blue channel of your nvidia detail pass (levels on your blue channel, put it to 127). Then you can overlay it on top of your baked layer. Ensure as much as possible, that your seams in UVs are dead straight vertical or horizontal. This prevents little to no anti-aliasing and bogus nm values on seams which will increase as you resize your textures and they get resampled. It is actually more important to have a higher rez normal map than diffuse texture. Often a 2:1 ratio works great

168 Segment 6: Issues + Tricks Tips tricks and optimizations: Texture size optimizations. Normal maps are king. 1/1 normal map ½ - ¼ diffuse map 1/1 - ½ specular map 1/1 - ½ eccentricity / roughness map

170 Segment 6: Issues + Tricks Do you even need them? Examples of game renders using ZERO normal maps!

172

174

175

176

177 Questions Derivative shader overview: all the look, 1/3 the memory Persistent system: Every variable realtime updateable The importance of tuning on target It s not what you can fit in memory, it s what you can get off disk in time

178 Derive them?

179 Derive them?

180 Derive them?

181 Special Thanks Mike Yeomans: XSI video workflow Tom Hsu: Z-Brush video Martin Farren, Simon Stevenson, Dave Pervil, Tony Albrecht: Engine team Robin Maddock: Persistent real-time update system

182 Special Thanks

183 Questions