The Graphics Pipeline

Transcription

1 Announcements The Graphics Pipeline Homework 3 due 02/27 Project kickoff 02/27 Patrick Cozzi University of Pennsylvania CIS Spring 2012 Course Contents Agenda Mobile Brief Graphics Review Graphics Pipeline Mapping the Graphics Pipeline to Hardware Parallel Algorithms CUDA Real-Time Rendering OpenGL / WebGL Graphics Pipeline GPU Architecture 1

2 Graphics Review Graphics Review: Modeling Graphics Modeling Polygons vs. triangles How do we store a triangle mesh? Modeling Rendering Animation Implicit Surfaces Real-Time Rendering Triangles Triangles 2

3 Implicit Surfaces Graphics Review: Rendering Images from Image from Graphics Review: Rendering Rendering Goal: Assign color to pixels Two Parts Visible surfaces What is in front of what for a given view Shading Simulate the interaction of material and light to produce a pixel color Graphics Review: Animation Move the camera and/or agents, and rerender the scene In less than 16.6 ms (60 fps) 3

4 Graphics Pipeline Walkthrough Images from Pull together a vertex from one or more buffers Also called primitive processing (GL ES) or input assembler (D3D) Image from OpenGL provides lots of flexibility for pulling vertex attributes from different buffers For example (no indices): OpenGL provides lots of flexibility for pulling vertex attributes from different buffers For example (no indices): positions texture coordinates positions texture coordinates normal, binormal, bitagent... normal, binormal, bitagent... Vertex 0 Vertex 0 4

5 OpenGL provides lots of flexibility for pulling vertex attributes from different buffers For example (no indices): positions texture coordinates Transform incoming vertex position from model to clip coordinates Perform additional per-vertex computations; modify, add, or remove attributes passed down the pipeline Formerly called the Transform and Lighting (T&L) stage. Why? vertex P model normal, binormal, bitagent... Vertex 1 modified vertex P clip P clip = (M model-view-projection )(P model ) Model to clip coordinates requires three transforms: model to world world to eye eye to clip Use 4x4 matrices passed to the vertex shader as uniforms Model to world: P world = (M model )(P model ) P world = (M model )(P model ) P eye = (M view ) (P world ) P clip = (M projection ) (P eye ) vertex modified vertex uniforms, e.g., matrices, etc. Image from 5

6 World to eye: Eye to clip coordinates: P eye = (M view ) (P world ) P clip = (M projection ) (P eye ) Image from Image from In practice, the model, view, and projection matrices are commonly burnt into one matrix? Why? Model to clip coordinate transformation is just one use for the vertex shader. Another use: animation. P clip = (M projection )(M view )(M model )(P model ) P clip = (M model-view-projection )(P model ) How would you implement pulsing? Image from 6

7 How would you implement pulsing? How do we compute the displacement? Displace position along surface normal over time Consider: float displacement = 0.5 * (sin(u_time) + 1.0); What are the shortcomings? How do we compute the displacement? Image from Image from How do we compute the displacement? How do we get the varying bulge? Consider: float displacement = u_scalefactor * 0.5 * (sin(u_frequency * u_time) + 1.0); What are the other shortcomings? Image from Image from 7

8 How do we get the varying bulge? What varies per-vertex and what does not? Consider float displacement = u_scalefactor * 0.5 * (sin(position.y * u_frequency * u_time) + 1.0); float displacement = u_scalefactor * 0.5 * (sin(position.y * u_frequency * u_time) + 1.0); Image from On all modern GPUs, vertex shaders can read from textures as well as uniform variables. Example: Textures can provide height maps for displacement mapping What is this useful for? vertex uniforms textures modified vertex Images from 8

9 Technology preview: vertex shaders are becoming available to CSS on the web as CSS shaders RenderMonkey Demos Bounce Demo Morph Particle System More info on CSS shaders: RenderMonkey: A vertex shader processes one vertex. Primitive assembly groups vertices forming one primitive, e.g., a triangle, line, etc. A vertex shader processes one vertex. Primitive assembly groups vertices forming one primitive, e.g., a triangle, line, etc. VS VS 9

10 A vertex shader processes one vertex. Primitive assembly groups vertices forming one primitive, e.g., a triangle, line, etc. A vertex shader processes one vertex. Primitive assembly groups vertices forming one primitive, e.g., a triangle, line, etc. VS VS A vertex shader processes one vertex. Primitive assembly groups vertices forming one primitive, e.g., a triangle, line, etc. A vertex shader processes one vertex. Primitive assembly groups vertices forming one primitive, e.g., a triangle, line, etc. VS VS 10

11 A vertex shader processes one vertex. Primitive assembly groups vertices forming one primitive, e.g., a triangle, line, etc. A vertex shader processes one vertex. Primitive assembly groups vertices forming one primitive, e.g., a triangle, line, etc. VS VS Perspective Division and Viewport Transform Clipping There are a series of stages between primitive assembly and rasterization. Perspective division P ndc = (P clip ).xyz / (P clip ).w There are a series of stages between primitive assembly and rasterization. Clipping Viewport transform P window = (M viewport-transform )(P ndc ) Image from Image from 11

12 Rasterization Determine what pixels a primitive overlaps Shades the fragment by simulating the interaction of light and material Loosely, the combination of a fragment shader and its uniform inputs is a material Also called a Pixel Shader (D3D) How would you implement this? What about aliasing? fragment fragment s color uniforms textures What happens to non-position vertex attributes during rasterization? What is the triangle-to-fragment ratio? What exactly is the fragment input? What are examples of useful uniforms? Useful textures? Example: Blinn-Phong Lighting Example: Blinn-Phong Lighting float diffuse = max(dot(n, L), 0.0); float specular = max(pow(dot(h, N), u_shininess), 0.0); Why not evaluate per-vertex and interpolate during rasterization? Image from Image from 12

13 Per-fragment vs. per-vertex lighting Effects of tessellation on per-vertex lighting Which is which? Image from Image from Another example: Texture Mapping Lighting and texture mapping Image from Image from 13

14 Another example: Bump mapping Another example: Bump mapping Fragment shaders can be computationally intense A fragment shader can output color, but what else would be useful? Image from 14

15 A fragment shader can output color, but what else would be useful? Discard the fragment. Why? Depth. Why? Multiple colors. Why? RenderMonkey Demos NPR Glass RenderMonkey: Scissor Test A fragment must go through a series of tests to make to the framebuffer What tests are useful? Discard a fragment if it is within a rectangle defined in window coordinates Why is this useful? Does this need to happen after fragment shading? Scissor Test Stencil Test Depth Test 15

16 Scissor Test Stencil Test The scissor test is useful for scissoring out parts of the window that do not need to be shaded For performance. Why? Post-processing effects Multipass rendering The stencil test can discard arbitrary areas of the window, and count per-fragment A stencil is written to the stencil buffer, and later fragments can be tested against this buffer What is this useful for? Scissor Test Scissor Test Stencil Test Stencil Test Depth Test Depth Test Stencil Test Stencil Test Example: Stencil shadow volumes Example: Stencil shadow volumes Scissor Test Scissor Test Stencil Test Stencil Test Depth Test Depth Test Image from Image from 16

17 Depth Test Depth Test Finds visible surfaces Once called ridiculously expensive Also called the z-test Does it need to be after fragment shading? Scissor Test Stencil Test Depth Test Image from Depth Test ing Combine fragment color with framebuffer color Can weight each color Can use different operations: +, -, etc. Why is this useful? Image from 17

18 ing Graphics Pipeline Walkthrough Example: Translucency After all that, write to the framebuffer! Additive ing C dest = (C source.rgb)(c source.a) + (C dest.rgb); Alpha ing C dest = (C source.rgb)(c source.a) + (C dest.rgb) (1 - C source.a); Image from Images from Evolution of the Programmable Graphics Pipeline Early 90s Pre GPU Pre GPU Fixed function GPU Programmable GPU Unified Shader Processors Slide from 18

19 Why GPUs? 3dfx Voodoo (1996) Exploit Parallelism Pipeline parallel Data-parallel CPU and GPU executing in parallel Still Software In hardware: Fixed-function rasterization, texture mapping, depth testing, etc. 4-6 MB memory PCI bus $299 Hardware: texture filtering, rasterization, MAD, sqrt, etc. Hardware Image from Aside: Mario Kart 64 High fragment load / low vertex load Aside: Mario Kart Wii High fragment load / low vertex load? Image from Image from 19

20 NVIDIA GeForce 256 (1999) NVIDIA GeForce 3 (2001) Hardware In hardware: Fixed-function vertex shading (T&L) Multi-texturing: bump maps, light maps, etc. 10 million polygons per second Direct3D 7 AGP bus Hardware Optionally bypass fixed-function T&L with a programmable vertex shader Optionally bypass fixed-function fragment shading with a programmable fragment shader Many programming limits Direct3D 8 Pentium IV 20 stages GeForce stages Fixed-function stage Programmable stage Image from NVIDIA GeForce 6 (2004) NVIDIA GeForce 6 (2004) textures Hardware Much better programmable fragment shaders Vertex shader can read textures Dynamic branches Multiple render targets PCIe bus OpenGL 2 / Direct3D 9 Vertex shader can read textures Dynamic branches Multiple render targets PCIe bus OpenGL 2 / Direct3D 9 Image from ftp://download.nvidia.com/developer/presentations/2004/gpu_jackpot/shader_model_3.pdf 20

21 NVIDIA GeForce 6 (2004) Dynamic Branches Vertex shader can read textures Dynamic branches Multiple render targets PCIe bus OpenGL 2 / Direct3D 9 Image from ftp://download.nvidia.com/developer/presentations/2004/gpu_jackpot/shader_model_3.pdf Image from Dynamic Branches For best performance, fragment shader dynamic branches should be coherent in screen-space How does this relate to warp partitioning in CUDA? NVIDIA GeForce 6 (2004) 6 vertex shader processors 16 fragment shader processors Image from 21

22 NVIDIA GeForce 8 (2006) Geometry Shader Ground-up GPU redesign Geometry Shaders Transform-feedback OpenGL 3 / Direct3D 10 Unified shader processors Support for GPU Compute Geometry Shaders Image from David Blythe : NVIDIA G80 Architecture Why Unify Shader Processors? Slide from Slide from 22

23 Why Unify Shader Processors? Terminology Shader Model Direct3D OpenGL Video card x Example NVIDIA GeForce 6800 ATI Radeon X x 3.x NVIDIA GeForce 8800 ATI Radeon HD x 4.x NVIDIA GeForce GTX 480 ATI Radeon HD 5870 Slide from Shader Capabilities Shader Capabilities Table courtesy of A K Peters, Ltd. Table courtesy of A K Peters, Ltd. 23

24 Evolution of the Programmable Graphics Pipeline Evolution of the Programmable Graphics Pipeline Slide from Mike Houston: Slide from Mike Houston: 24