Siddharth Deokar Graduate Research Student Department of Computer Science University of Minnesota Duluth Advised by Pete Willemsen May 22, 2009
Outline Problem Overview Background System Overview Sorting on the GPU & Blending Aggregate Snow LDR, HDR Rendering Modeling of snow Results Conclusion 2
Problem Overview (The Big Picture) Snowing conditions Real Life Problem Following behind a snowplow or other vehicles can be dangerous Difficult to see other vehicles or judge their speed Due to poor visibility and snow, it becomes difficult to detect speed changes of other vehicles resulting in the potential for rear end collisions Why snow rendering? Create a virtual interactive environment where a person can drive around in blowing snow and foggy conditions. Understand how we drive in snowing conditions Try to make winter driving safe 3
Problem Overview Continued Goal of my research Effective rendering of snowing conditions Improve visual rendering of snow Interaction between light and snow Create more realistic scenes Experiment Framework: Simulated Highway [Pete Willemsen] & Snow Plow [Michele Olsen] Video (SnowRenderingSimulation.mov) 4
Background GPU Why GPU? Many Stream Processors SIMD Architecture Massive Parallelization Dedicated for rendering high end graphics like video games, 3D games Textures Main memory structure of a GPU Each colored square is an individual texel, which can be a vector values Shaders? One shader for updating positions Another Shader for sorting positions 12 13 14 15 8 9 10 11 4 5 6 7 0 1 2 3 5
Background continued Particle System [Andrew Norgren] Point Sprites Render millions of particles in real time Video (ParticlesAsPointSprites.mov) 6
System Overview Sorting & Blending Why sorting or blending? Quick Sort GPU Sort Additive Blend Aggregate Snow Effect High Dynamic Range Rendering Model snow particles Transparency Scattering of Light 7
Sorting & Blending on the GPU Why is sorting required? Objects are drawn in a sequence which is provided by the user in his code Problem with transparent objects blocking objects behind them Video (unsorted particles.mov) 8
Sorting of Snow Particles Sort the particles based on each individual particle s distance from the eye Euclidean Distance Quick Sort Sorting done by CPU Maps with GPU memory every frame to sort Map the particle positions from the vertex buffer to CPU memory 9
Sorting of Snow Particles Continued GPU Sort Inherent Parallelism Gets rid of CPU communication Faster than using the best sorting algorithms on the CPU Algorithms like Quick Sort cannot be implemented on the GPU as it cannot write to arbitrary memory locations The GPU does not support this functionality to avoid write after read operations by different stream processors when accessing the same memory location 10
GPU Sort Implemented by Govindaraju, N., Raghuvanshi, N., Henson, M., Tuft, D., Manocha, D. [A Cache Efficient Sorting Algorithm for Database and Data Mining Computations using Graphics Processors, 2005] They have used texture mapping to implement a bitonic sort algorithm I have modified the algorithm to suit the sorting of particles in our system 11
GPU Sort Continued Bitonic Sequence A sequence which has at most one local minimum or maximum When you break a bitonic sequence at the minimum or maximum you get two sorted sequences, one ascending and the other descending Examples: 1,2,3,4,5,6,7,8 2,5,8,10,7,4,3,1 9,8,5,3,6,10,11,12 splitting the second sequence at 10 gives us 2,5,8,10 and 7,4,3,1 which are sequences in ascending and descending order respectively In the third example, the minimum is 3 and splitting the sequence at 3 gives us 9, 8, 5, 3 and 6, 10, 11, 12 which are sequences in descending and ascending order respectively 12
GPU Sort Continued (Bitonic Sort) Stage 1: BitonicSequence Created Stage 2: Bitonic Sort Pass 1 Pass 2 Pass 3 Pass 4 Pass 5 Pass 6 13
Example of Bitonic Sort Unsorted Sequence 12 8 14 6 15 10 9 13 Pass 1 12 8 14 6 15 10 9 13 Result 8 12 14 6 10 15 13 9 Pass 2 8 12 14 6 10 15 13 9 Result 8 6 14 12 13 15 10 9 Pass 3 8 6 14 12 13 15 10 9 Bitonic Sequence 6 8 12 14 15 13 10 9 Pass 4 6 8 12 14 15 13 10 9 Result 6 8 10 9 15 13 12 14 Pass 5 6 8 10 9 15 13 12 14 Result 6 8 10 9 12 13 15 14 Pass 6 6 8 10 9 12 13 15 14 Sorted Sequence 6 8 9 10 12 13 14 15 14
GPU Sort Continued Applying GPU Sort to our system The particles are stored in a 2-D texture where each cell holds the position of a particle as a 4-D vector containing values <x,y,z,w> Calculate Euclidean distance of each particle from eye Indices for particles? Indices Distances GPU Sort Sorted Indices 1 2 3 4 25 38 33 28 1 14 9 4 5 6 7 8 + 40 32 27 36 = 16 8 3 12 9 10 11 12 37 34 26 31 13 10 2 7 13 14 15 16 29 35 39 30 5 11 15 6 15
GPU Sort Continued Drawback GPU Sort is very fast for a small number of particles but becomes slow as the number of particles increases For example, the time taken to render a frame with one million particles is more than 0.01 seconds which is necessary for an interactive environment. Solution GPU Sort done over a number of passes It is Fast but, This approach is restricted as the particles are moving and hence we cannot have a large number of passes to sort the particles in a particular position On the other hand, by using a few passes gives us artifacts which are easily noticeable to the human eye 16
Additive Blending Alternative to sorting Alpha value is used to combine the color value of a fragment with the color of a pixel already stored in the framebuffer 17
GPU Sort and Blending Continued Video (Sorting.mov) 18
Aggregate Snow Effects Snow particles are confined in a certain domain To show snow particles everywhere is not computationally feasible In real world it is hard to see individual snow particles after a certain distance Aggregation of snow becomes more prominent as you look into the horizon with objects becoming less and less visible with increasing distance 19
Aggregate Snow Effects Continued Visibility As the amount of snow increases, visibility decreases Derive a relation between the snow density in our simulation system and visibility in the simulated scene Video (Real World Aggregation.mov) 20
Aggregation: Snow Density 21
Aggregate Snow Effect Video (Visibility.mov) 22
High Dynamic Range Rendering Our visual system supports a very high contrast ratio which is not reflected in graphical scenes Low Dynamic Range Rendering 8 bits per color (RGB) Contrast ratio of 256:1 per color High Dynamic Range Rendering Better contrast ratio Create more realistic scenes Example: Light Source (yellow) LDR RGB = {1,1,0} HDR RGB = {4,4,0} 4 times brighter than a light source with intensity {1,1,0} 23
HDR Rendering Process Render Scene Extract High Luminance Downsample & Blur Additive Blending Final Image Bloom Mask 24
HDR Rendering Continued Render using HDR Extract High Luminance Downsampling & Blur Bloom Mask Image rendered using HDR Final Image 25
HDR Rendering Continued HDR Scene is tone mapped in order to be displayed on the monitor HDR Image Tone Mapping LDR Image Tone Mapping 26
HDR Rendering Continued Final Image displayed on screen 27
Modeling of Snow Snow Translucent particles Interact with light to produce scattering effect Model shape of a snow particle, appearance and scattering of light due to snow 28
Transparency Gaussian Function Gives snow opacity at the centre and becomes transparent as we move away from the centre 29
Shapes for snow particles Particles are modeled after actual snow flakes These images give an outer boundary to our snow particles so that they resemble actual snowflakes 30
Shapes for snow particles continued Particles as snow flakes Gaussian Transparency Snow flake Final Image with Gaussian transparency and snow flake shape 31
Dynamic Lights We can have many lights active in the system at any given moment of time Light source information is available through use of textures Data 1 Position 1 Intensity 1 Direction 1 Spotlight Parameters 1 Data 2 Position 2 Intensity 2 Direction 2 Spotlight Parameters 2 Data 3 Position 3 Intensity 3 Direction 3 Spotlight Parameters 3 Data 4 Position 4 Intensity 4 Direction 4 Spotlight Use active lights for scattering calculations Parameters 4 32
Scattering Ray Tracing Tracing path of light through every pixel Computationally expensive Phase /Scattering function Henyey Greenstein Phase Function Widely used in atmospheric sciences because of its simplicity eye θ 33
Scattering Continued Single parameter g (asymmetry parameter) controls the distribution of scattered light Forward Scattering Back Scattering 34
Scattering Video (scattering.mov) 35
Results Sorting 4 3.5 3 2.5 2 1.5 Time In 1 seconds 0.5 0 15K 65K 250K 500K 1M 2M Number of Particles Quick Sort GPU Sort with Passes GPU Sort w/o Passes 36
Results Continued GPU Sort (With Passes Vs Without Passes) 0.02 0.018 0.016 0.014 0.012 0.01 0.008 0.006 Time In seconds 0.004 0.002 0 15K 65K 250K500K 1M 2M Number of Particles GPU Sort with Passes GPU Sort w/o Passes 37
Results Continued Visibility Image 1 = > Visibility 0.76 (Less snow 6000 particles/s) Image 2 = > Visibility 0.57 (Heavy Snow 600000 particles/s) Image 3 = > Visibility 0.16 (Very Heavy Snow 900000 particles/s) 38
Results Continued Snow Model OSG Snow (Video osgsnowrendering.mov) Our snow model Snow system with all the effects like Gaussian transparency, scattering, HDR and fog 39
Summary 1. Sort the particles based on distance from eye OR use additive blend 2. Model snow particle 3. Get the number of lights in the scene 4. Calculate color for each particle 5. Calculate effect of every light on the particle 6. Apply Gaussian Transformation 7. Apply Scattering Function 8. Apply snow aggregation to other objects in the scene 9. Render scene to a frame buffer using HDR for lighting calculations 10. Render HDR image 11. Down sample and apply Gaussian blur 12. Apply resulting bloom mask to original image 13. Render final image 40
Snow Model Video (snow.mov) 41
Implementation Implemented on top of particle system created by Andrew Norgren, Pete Willemsen Implemented in C++ with OpenGL and OpenGL Shading Language Tested on 2.4 GHz AMD Dual Core Processor with a NVIDIA GeForce 8800 Ultra graphics card 128 Stream Processors Memory 728 MB Matlab for snow density calculations 42
Conclusion A snow domain which is more realistic to outside world Individual & Aggregate Snow modeled effectively Interactive Environment (Real Time!) Future Tie visibility with real world data Sharp snow particles in the near plane and blurry snow (gaussian) at a distance Sorting particles near the viewer Shadowing 43
Acknowledgements Pete Willemsen L. Zimmerman Andrew Norgren, Michele Olsen 44
Thank You Questions?