GPUs Under the Hood. Prof. Aaron Lanterman School of Electrical and Computer Engineering Georgia Institute of Technology

Transcription

1 GPUs Under the Hood Prof. Aaron Lanterman School of Electrical and Computer Engineering Georgia Institute of Technology

2 Bandwidth Gravity of modern computer systems The bandwidth between key components ultimately dictates system performance Especially true for massively parallel systems processing massive amount of data Tricks like buffering, reordering, caching can temporarily defy the rules in some cases Ultimately, the performance falls back to what the feeds and speeds dictate PCIe replaced AGP (Advanced Graphics Port) Slide by David Kirk/NVIDIA and Wen-mei. W. Hwu, 2007, from UIUC ECE498 Lecture 6, Fall 2007; used with permission See 2

3 3D buzzwords Fill Rate how fast the GPU can generate pixels, often a strong predictor for application frame rate Performance Metrics Mtris/sec - Triangle Rate Mverts/sec - Vertex Rate Mpixels/sec - Pixel Fill (Write) Rate Mtexels/sec - Texture Fill (Read) Rate Msamples/sec - Antialiasing Fill (Write) Rate Slide by David Kirk/NVIDIA and Wen-mei. W. Hwu, 2007, from UIUC ECE498 Lecture 5, Fall 2007; used with permission See 3

4 Adding programmability to the pipeline 3D Application or Game 3D API Commands 3D API: OpenGL or Direct3D CPU GPU Boundary GPU Command & Data Stream GPU Front End Vertex Index Stream Primitive Assembly Assembled Polygons, Lines, and Points Rasterization & Interpolation Pixel Location Stream Raster Operations Pixel Updates Framebuffer Pre-transformed Vertices Programmable Vertex Processor Transformed Vertices Rasterized Pre-transformed Fragments Programmable Fragment Processor Transformed Fragments Slide by David Kirk/NVIDIA and Wen-mei. W. Hwu, 2007, from UIUC ECE498 Lecture 5, Fall 2007; used with permission See 4

5 Shader data Typically floats, and vectors/matrices of floats Fixed size arrays Three main types: Per-instance data, e.g., per-vertex position Per-pixel interpolated data, e.g., texture coordinates Per-batch data, e.g., light position Data are tightly bound to the GPU

6 Shader flow control Very simple No recursion Fixed size loops for Shader Model 2.0 or earlier Simple if-then-else statements allowed in the latest APIs Texkill (asm) or clip (HLSL) or discard (GLSL) allows you to abort a write to a pixel (form of flow control)

7 Specialized instructions (GeForce 6) Dot products Exponential instructions: EXP, EXPP, LOG, LOGP LIT (Blinn specular lighting model calculation!) Reciprocal instructions: RCP (reciprocal) RSQ (reciprocal square root!) Trignometric functions SIN, COS Swizzling (swapping xyzw), write masking (only some xyzw get assigned), and negation is free From GPU Gems 2, p

8 Vertex shader Transform to clip-space (i.e., screen space) Inputs: Common inputs: Vertex position (x, y, z, w) Texture coordinate Constant inputs Can also have fog, color as input, but usually passes them untouched to the pixel shader Output to a pixel (fragment) shader Vertex shader is executed once per vertex, could be less expensive than pixel shader

9 Vertex shader data flow (3.0) Vertex stream 32 Temporary registers al Loop Register r0 r1 r2 r31 a0 Address Register v0 v1 v2 16 Vertex data registers Vertex Shader v15 C0 C1 C2 Cn 12 output registers Constant float registers (at least 256) 16 Constant Integer Registers opos otn ofog od0 od1 opts position texture fog Diff. color Spec. color Output Pt size Each register is a 4-component vector register except al

10 Vertex shader: logical view Per-vertex Input Data Register File r0 r1 r2 r3... Shader Start Addr Bound Textures Bound Samplers Bound Consants Vertex Processing Unit Swizzle / Mask Unit.rgba.xyzw.zzzz.xxyz... cosine log Math/Logic sine sub Unit add... Per-vertex Output Data Transformed and Lit vertices Shader Resources (bound by application) Sampler Unit Texture Memory Shader Constants Input Data Output Data State Information Memory Architectural State Control Logic

11 Some uses of vertex shaders Transform vertices to clip-space Pass normal, texture coordinates to PS Transform vectors to other spaces (e.g., texture space) Calculate per-vertex lighting (e.g., Gouraud shading) Distort geometry (waves, fish-eye camera) Adapted from Mart Slot s presentation

12 Easy cross products and normalization From Stanford CS448A: Real-Time Graphics Architectures See graphics.stanford.edu/courses/cs448a-01-fall 12

13 Blinn lighting in one instruction From Stanford CS448A: Real-Time Graphics Architectures See graphics.stanford.edu/courses/cs448a-01-fall 13

14 Simple graphics pipeline From Stanford CS448A: Real-Time Graphics Architectures See graphics.stanford.edu/courses/cs448a-01-fall 14

15 Pixel (or fragment) shader (1) Determine each fragment s color Custom (sophisticated) pixel operations Texture sampling Inputs Interpolated output from vertex shader Typically vertex position, vertex normals, texture coordinates, etc. These registers could be reused for other purpose Output Color (including alpha) Depth value (optional)

16 Pixel (or fragment) shader (2) Executed once per pixel, hence typically executed many more times than a vertex shader It is advantageous to compute stuff on a per-vertex basis to improve performance

17 Pixel shader data flow (3.0) Temporary registers r0 r1 r31 v0 Pixel stream v1 Color (diff/spec) and texture coord. registers oc0 Pixel Shader v9 odepth C0 C1 Cn s0 s1 s15 Constant registers (16 INT, 224 Float) Sampler Registers (Up to 16 texture surfaces can be read in a single pass) color Depth

18 Pixel shader: logical view Interpolator Per-pixel Input Data Register File r0 r1 r2 r3... Shader Start Addr Bound Textures Bound Samplers Bound Consants Pixel Processing Unit Swizzle / Mask Unit.rgba.xyzw.zzzz.xxyz... cosine log Math/Logic sine sub Unit add... Per-pixel Output Data Pixel Color Depth Info Stencil Info Shader Resources (bound by application) Sampler Unit Texture Memory Color buffer Depth Buffer Stencil Buffer Shader Constants Input Data Output Data State Information Memory Architectural State Control Logic

19 Some uses of pixel shaders Texturing objects Per-pixel lighting (e.g., Phong shading) Normal mapping (each pixel has its own normal) Shadows (determine whether a pixel is shadowed or not) Environment mapping Adapted from Mart Slot s presentation

20 Old GeForce graphics pipeline Host Vertex Control VS/T&L Vertex Cache Triangle Setup Raster Shader ROP FBI Texture Cache Frame Buffer Memory Slide by David Kirk/NVIDIA and Wen-mei. W. Hwu, 2007, from UIUC ECE498 Lecture 5, Fall 2007; used with permission See 20

21 Vertex cache Host Vertex Control VS/T&L Vertex Cache Re-using vertices between primitives saves PCIe bus bandwidth and GPU computational resources Triangle Setup Raster Shader ROP FBI Texture Cache Frame Buffer Memory A vertex cache attempts to exploit commonality between triangles to generate vertex reuse Unfortunately, many applications do not use efficient triangular ordering Slide by David Kirk/NVIDIA and Wen-mei. W. Hwu, 2007, from UIUC ECE498 Lecture 5, Fall 2007; used with permission See 21

22 Texture cache Host Vertex Control T&L Vertex Cache Stores temporally local texel values to reduce bandwidth requirements Triangle Setup Raster Shader ROP FBI Texture Cache Frame Buffer Memory Due to nature of texture filtering high degrees of efficiency are possible (75% or better hit rates) Reduces texture (memory) bandwidth by a factor of four for bilinear filtering Slide by David Kirk/NVIDIA and Wen-mei. W. Hwu, 2007, from UIUC ECE498 Lecture 5, Fall 2007; used with permission See 22

23 Built-in texture filtering (GeForce 6) Pixel texturing Hardware supports 2D, 3D, and cube map Non power-of-2 textures OK Hardware handles addressing and interpolation Bilinear, trilinear (3D or mipmap), anisotropic Vertex texturing Vertex processors can access texture memory too Only nearest-neighbor filtering supported in G60 hardware 23

24 Host ROP (from Raster Operations) Vertex Control T&L Vertex Cache Triangle Setup Raster C-ROP performs frame buffer blending Combinations of colors and transparency Shader ROP FBI Texture Cache Frame Buffer Memory Antialiasing Read/Modify/Write the Color Buffer Z-ROP performs the Z operations Determine the visible pixels Discard the occluded pixels Read/Modify/Write the Z-Buffer ROP on GeForce also performs Coalescing of transactions Z-Buffer compression/decompression Slide by David Kirk/NVIDIA and Wen-mei. W. Hwu, 2007, from UIUC ECE498 Lecture 5, Fall 2007; used with permission See 24

25 The frame buffer Host Vertex Control T&L Vertex Cache The primary determinant of graphics performance other than the GPU The most expensive component of a graphics product other than the GPU Memory bandwidth is the key Frame buffer size also determines Local texture storage Maximum resolutions Anitaliasing resolution limits Triangle Setup Raster Shader ROP FBI Texture Cache Frame Buffer Memory Slide by David Kirk/NVIDIA and Wen-mei. W. Hwu, 2007, from UIUC ECE498 Lecture 5, Fall 2007; used with permission See 25

26 Frame Buffer Interface (FBI) Host Vertex Control Surface Engine T&L Vertex Cache Manages reading from and writing to frame buffer Perhaps the most performance-critical component of a GPU GeForce s FBI is a crossbar Independent memory controllers for 4+ independent memory banks for more efficient access to frame buffer Triangle Setup Raster Shader ROP FBI Texture Cache Frame Buffer Memory Slide by David Kirk/NVIDIA and Wen-mei. W. Hwu, 2007, from UIUC ECE498 Lecture 5, Fall 2007; used with permission See 26

27 GeForce 7800 GTX board details SLI Connector Single slot cooling svideo TV Out DVI x 2 16x PCI-Express Slide by David Kirk/NVIDIA and Wen-mei. W. Hwu, 2007, from UIUC ECE498 Lecture 6, Fall 2007; used with permission See 256MB/256-bit DDR3 600 MHz 8 pieces of 8Mx32 27

28 From G70 Architecture NVIDIA 7800 GTX Vertex Processors Pixel Processors ROPs (Raster Op. Units) 28

29 NVIDIA 7800 GTX NVIDIA 7800 GTX Vertex processors 7800 GTX has G70 Architecture 8 of these Vertex Processors G70 Architecture 29

30 NVIDIA 7800 GTX NVIDIA 7800 GTX Pixel processors 8 MADD (multiply/add) instructions in a single cycle G70 Architecture 7800 GTX has 24 of these From 30

31 NVIDIA 7800 GTX Modern GPUs: unified design G70 Architecture Vertex Processors Slide by David Luebke from G70 Architecture 31

32 GeForce 8 architecture Slide by David Luebke from 32

33 Why unify? (1) Slide by David Luebke from 33

34 Why unify? (2) Slide by David Luebke from 34

35 Dynamic load balancing Company of Heroes Slide by David Luebke from 35

36 Motivation for shader languages Programming powerful hardware with assembly code is hard Programmers need the benefits of a high-level language: Easier programming Easier code reuse Easier debugging Portability Assembly DP3 R0, c[11].xyzx, c[11].xyzx; RSQ R0, R0.x; MUL R0, R0.x, c[11].xyzx; MOV R1, c[3]; MUL R1, R1.x, c[0].xyzx; DP3 R2, R1.xyzx, R1.xyzx; RSQ R2, R2.x; MUL R1, R2.x, R1.xyzx; ADD R2, R0.xyzx, R1.xyzx; DP3 R3, R2.xyzx, R2.xyzx; RSQ R3, R3.x; MUL R2, R3.x, R2.xyzx; DP3 R2, R1.xyzx, R2.xyzx; MAX R2, c[3].z, R2.x; MOV R2.z, c[3].y; MOV R2.w, c[3].y; LIT R2, R2;... float3 cspecular = pow(max(0, dot(nf, H)), phongexp).xxx; float3 cplastic = Cd * (cambient + cdiffuse) + Cs * cspecular; From The Cg Tutorial

37 Shader languages HLSL/Cg most common Both are more-or-less compatible Other alternatives: GLSL (for OpenGL) Assembly?