Virtual Reality Modeling

Transcription

1 Virtual Reality Modeling Modeling & VR Toolkits System architecture 1

2 The VR modeling cycle The VR geometric modeling: ü Object surface shape: polygonal meshes (vast majority); splines (for curved surfaces); ü Object appearance: Lighting (shading) texture mapping 2

3 The surface polygonal (triangle) mesh (X 1,Y 1,Z 1 ) (X 3,Y 3,Z 3 ) (X 5,Y 5,Z 5 ) Non-shared vertex (X 0,Y 0,Z 0 ) Shared vertex (X 2,Y 2,Z 2 ) (X 4,Y 4,Z 4 ) Triangle meshes are preferred since they are memory and computationally efficient (shared vertices) Object polygonal shape: ü Can be programmed from scratch using OpenGL or other toolkit editor; it is tedious and requires skill; ü Can be obtained from CAD files; ü Can be created using a 3D digitizer (stylus), or a 3D scanner (tracker, cameras and laser); ü Can be purchased from existing online databases; files have vertex location and connectivity information, but are static. 3

4 Geometric Modeling CAD-file based models: ü Done using AutoCAD; ü Each moving part a separate file; ü Files need to be converted to formats compatible with VR toolkits; ü Advantage use of preexisting models in manufacturing applications. Geometric Modeling CAD-file based models: ü Done using AutoCAD; ü Each moving part a separate file; ü Files need to be converted to formats compatible with VR toolkits; ü Advantage use of preexisting models in manufacturing applications. 4

5 Geometric Modeling Venus de Milo created using the HyperSpace 3D digitizer, 4200 textured polygons using NuGraph toolkit Polhemus 3D scanners: ü Eliminate direct contact with object. ü uses two cameras, a laser, and magnetic trackers (if movable objects are scanned) ü Scanning resolution 0.5 mm at 200 mm range; ü Scanning speed is 50 lines/sec; ü Range is mm scanner-object range. 5

6 Geometric Modeling Polhemus FastScan 3D scanner (can scan objects up to 3 m long). Polhemus scanner DeltaSphere D scanner Feature Polhemus scanner DeltaSphere scanner Range 0.56 m 14.6 m Resolution m 0.25 mm Control manual automatic Speed 50 lines/sec 25,000 samples/sec 6

7 Conversion of scanner data: ü Scanners produce a dense cloud of vertices (x,y,z). ü Using such packages as Wrap ( the point data is transformed into surface data (including editing and decimation) Point cloud from scanner Polygonal mesh after decimation Geometric Modeling using online databases Low res. Model 600 polygons Higher resolution model > 20,000 polygons. 7

8 Geometric Modeling Object Visual Appearance ü Scene illumination (local or global); ü Texture mapping; ü Multi-textures ü Use of textures to do illumination in the rasterizing stage of the pipeline 8

9 Flat shaded Teapot Shaded teapot Global scene illumination ü The inter-reflections and shadows cast by objects on each other. 9

10 Radiosity illumination ü Results in a more realistic looking scene Without radiosity With radiosity Texture mapping ü It is done in the rasterizer phase of the graphics pipeline, by mapping assigning texture space coordinates to polygon vertices (or splines), then mapping these to pixel coordinates; ü Texture increase scene realism; ü Texture provide better 3D spatial cues (they are perspective transformed); ü They reduce the number of polygons in the scene increased frame rate. 10

11 Textured room image for increased realism from VR Modeling The Texturing Functional Sub-Stages Application Geometry Rasterizer Compute Object space Location Use projector Function for Parameter space Corresponder Function to Find pixel Apply value Transform function Modify Illumination equation 11

12 How to create textures: ü Models are available on line in texture libraries of cars, people, construction materials, etc. ü Custom textures from scanned photographs or ü Using an interactive paint program to create bitmaps VR Geometric Modeling Tree, higher resolution model 45,992 polygons. Tree represented as a texture 1 polygon, 1246x1280 pixels ( 12

13 Normal texture Allow more complex textures Reflectivity texture Background texture Transparency texture Bump maps Multi-texturing for bump mapping: ü Lighting effects caused by irregularities on object surface are simulated through bump mapping ; ü This encodes surface irregularities as textures; ü No change in model geometry. No added computations at the geometry stage; ü Done as part of the per-pixel shading operations of the NSR 13

14 Bump mapping per-pixel shading Normal texture Multitexture Bump texture Multi-texturing for lighting: ü Several texels can be overlaid on one pixel; ü One application in more realistic lighting; ü Polygonal lighting is real-time but requires lots of polygons (triangles) for realistic appearance Vertex lighting of low polygon count surface lights are diffuse tessellated. Vertex lighting of high polygon count surface lights have realistic appearance. High computation load 14

15 Multi-texturing (texture blending): ü Realistic-looking lighting can be done with 2D textures called light maps ; ü Not applicable to real-time (need to be recomputed when object moves) Standard lighting map 2-D texture Light map texture overlaid on top of wall texture. Realistic and low polygon count. Not real-time! KINEMATICS MODELING: ü Homogeneous transformation matrices; ü Object position; ü Transformation invariants; ü Object hierarchies; ü Viewing the 3D world. 15

16 Camera system of coordinates y T global fingertip (t) z x T 3 fingertip T 2 3 (t) T 1 2 (t), T palm 1 (t) T global W (t) T W source T W palm (t) Receiver system of coordinates Source system of coordinates World system of coordinates VR Modeling The Graphics Rendering Pipeline (revisited) Application Geometry Rasterizer Model & View Transformation The Geometry Functional Sub-Stages Lighting Projection Clipping Screen Mapping 16

17 VR Modeling The Rendering Pipeline Application Geometry Rasterizer Model & View Transformation The Geometry Functional Sub-Stages Lighting Projection Clipping Screen Mapping Rutgers University - Virtual Reality Technology Model and Viewing Transformations: ü Model transforms link object coordinates to world coordinates. By changing the model transform, the same object can appear several times in the scene. We call these instances p 1x (t) p 1y (t) p 1z (t) [ ] Virtual ball Instance 2 of Virtual ball Instance 5 World system or coordinate p 5x (t) p 5y (t) p 5z (t) [ ] 17

18 Camera system of coordinates z x y T camera fingertip 2 (t) T camera fingertip 1 (t) ü The View Transform matrix captures the position and orientation of the virtual camera in the virtual world; ü It maps world coordinates to camera space (also called eye space ); ü The camera is located at the origin of the camera coordinate system, looking in the negative Z axis, with Y pointing upwards, and X to the right. VR Modeling The Rendering Pipeline Application Geometry Rasterizer The Geometry Functional Sub-Stages Model & View Transformation Lighting Projection Clipping Screen Mapping 18

19 Projection Transformations: ü Models what portion (volume) of the virtual world the camera actually sees. There are two kinds of projections, parallel projection and perspective projection VR uses perspective. projection plane projection reference point Parallel projection Perspective projection VR Modeling The Rendering Pipeline Application Geometry Rasterizer Model & View Transformation The Geometry Functional Sub-Stages Lighting Projection Clipping Screen Mapping 19

20 Clipping Transformation: ü Since the fulcrum maps to the unit cube, only objects inside it will be rendered. Some objects are partly inside the unit cube (ex. the line and the rectangle). Then they need to be clipped. The vertex V 1 is replaced by new one at the intersection between the line and the viewing cone, etc. Scene clipping V 1 Unit cube V 1 X X V 3 V 2 V 3 V 2 Z Z VR Modeling The Rendering Pipeline Application Geometry Rasterizer Model & View Transformation The Geometry Functional Sub-Stages Lighting Projection Clipping Screen Mapping 20

21 Screen Mapping (Viewport Transformation): ü The scene is rendered into a window with corners (x 1,y 1 ), (x 2,y 2 ) ü Screen mapping is a translation followed by a scaling that affects the x and y coordinates of the primitives (objects), but not their z coordinates. Screen coordinates plus z [-1,1] are passed to the rasterizer stage of the pipeline. Screen mapping V 1 X (x2,y2) V 3 V 2 (x1,y1) Z The VR physical modeling: 21

22 The Haptics Rendering Pipeline Traversal Tactile Display Collision Detection Calculation Smoothing Mapping Haptic Texturing Application Geometry Rasterizer Display Scene Traversal View Transform Lighting Projection Texturing adapted from (Popescu, 2001) Rutgers University - Virtual Reality Technology The Haptics Rendering Pipeline Traversal Tactile Display Collision Detection Calculation Smoothing Mapping Haptic Texturing 22

23 Collision detection: ü Uses bounding box collision detection for fast response; ü Two types of bounding boxes, with fixed size or variable size (depending on enclosed object orientation). ü Fixed size is computationally faster, but less precise Variable size Bounding Box Fixed size Bounding Box Undetected collision Ball at time t Longest distance Ball at time t + 1/fps If longest distance d < v 1/fps then collision is undetected (v is the ball velocity along its trajectory) 23

24 Two-stage collision detection: ü For more precise detection, we use a two-stage collision detection: an approximate (bounding box ) stage, followed by a slower exact collision detection stage. Bounding Box Pruning multi-body pairs yes Simulation no Pair-wise exact Collision Detection Collision response yes Surface cutting: ü An extreme case of surface deformation is surface cutting. This happens when the contact force exceed a given threshold; ü When cutting, one vertex gets a co-located twin. Subsequently the twin vertices separate based on spring/damper laws and the cut enlarges. Mesh before cut Cutting instrument Mesh after cut V1 V2 V1 V2 24

25 Collision response surface deformation The Haptics Rendering Pipeline Traversal Tactile Display Collision Detection Calculation Smoothing Mapping Haptic Texturing 25

26 Haptic interface Haptic Interface Point I -Haptic Interface Point Haptic Interface Point Object polygon Penetration distance output for homogeneous elastic objects F = { F max K d, for 0 d d max for d max < d where F max is that haptic interface maximum output force saturation F max Hard object Soft object d max 1 d max 2 Penetration distance d 26

27 The Haptics Rendering Pipeline Traversal Tactile Display Collision Detection Calculation Smoothing Mapping Haptic Texturing The haptic mesh: ü A single HIP is not sufficient to capture the geometry of fingertip-object contact; ü The curvature of the fingertip, and the object deformation need to be realistically modeled. Screen sequence for squeezing an elastic virtual ball 27

28 Haptic mesh Mesh point i Penetration distance for mesh point i Haptic mesh force calculation Penetration distance for mesh point i Haptic Interface Point i For each haptic interface point of the mesh: F haptic-mesh i = K object d mesh i N surface where d mesh i are the interpenetrating distances at the mesh points, N surface is the weighted surface normal of the contact polygon 28

29 The Haptics Rendering Pipeline Traversal Tactile Display Collision Detection Calculation Smoothing Mapping Haptic Texturing mapping displayed by the Rutgers Master interface: F displayed = (Σ F haptic-mesh ) cosθ where θ it the angle between the mesh force resultant and the piston 29

30 The Haptics Rendering Pipeline Traversal Tactile Display Collision Detection Calculation Smoothing Mapping Haptic Texturing Haptic interface Haptic Interface Point Y Z Object polygon X F texture = A sin(m x) sin(n y), where A, m, n are constants: A gives magnitude of vibrations; m and n modulate the frequency of vibrations in the x and y directions 30

31 Level of detail segmentation: ü Level of detail (LOD) relates to the number of polygons on the object s surface. Even if the object has high complexity, its detail may not be visible if the object is too far from the virtual camera (observer). Tree with 27,000 polygons (details are not perceived) Tree with 27,000 polygons Discrete Geometry LOD: ü Uses several discrete models of the same virtual object; ü Models are switched based on their distance from the camera (r < r 0 ; r 0 < r < r 1 ; r 1 < r < r 2 ; r 2 < r) r 1 r 2 r 0 LOD 0 LOD 1 LOD 2 31

32 Single-object Adaptive Level of Detail Sphere with 8192 triangles Uniform high density, sec to render Sphere with 537 triangles adaptive LOD, sec to render (SGI RE2, single R10000 workstation) Single-object Adaptive Level of Detail Bunny with 69,451 triangles Uniform high density, sec to render Bunny with 3615 triangles adaptive LOD, sec to render (SGI RE2, single R10000 workstation) 32

33 Model management Only the current universe needs to be rendered Solid modeling: Voxels The object is seen as part of volume We use the è il voxel (volume element) 33

34 Solid modeling: Voxels Uniform 3D grid Easy management Large use of memory It is possible to efficiently compress the data Polygonal Mesh Approximate description of the surface Mesh: a set of polygons The polygons do not intersect and they share a vertex or an edge Each polygon approximates a portion of the surface They are used mostly mesh of triangles Accurate representation large number of triangles the model could be heavy Selecting the number of triangles as a trade-off 34

35 35