Virtual Reality Features of NVIDIA GPUs
Download as PPT, PDF6 likes1,682 views
This document summarizes virtual reality rendering features of NVIDIA Pascal GPUs. It discusses how Pascal supports efficient single-pass stereo rendering by generating left and right eye views in one rendering pass. It also describes how Pascal implements lens matched shading to better match the rendered images to an HMD lens, reducing pixel resampling. Finally, it notes Pascal's use of window rectangle testing to discard pixels that fall outside the lens region, improving performance.
Virtual Reality Features of NVIDIA GPUs
- 1. Mark Kilgard, July 27, 2016 Virtual Reality Features of NVIDIA GPUs
- 2. 2 Mark Kilgard Principal System Software Engineer OpenGL driver implementation & API evolution Other interests Programmable shading languages, original Cg GPU shading language Web, vector graphics & path rendering, GPU rendering for 2D graphics Also known for creating OpenGL Utility Toolkit (GLUT) My Background
- 3. 3 GPU Challenges for Virtual Reality Rendering 1. Must render unique view for each eye Two scene renders per frame 2. Wide field-of-view rendered More pixels, distributed towards periphery 3. HMD lens magnifies rendered scene Non-linear inverse lens warp Allows the HMD lens magnification to provide convincing sense of presence
- 4. 4 Magnification Effect of HMD Lens Graph paper viewed & magnified through HMD lens
- 5. 5 Head Mounted Display Lens Effect Barrel Distorted Images on HMD Screen Magnified Overlapping Wide Field-of-view Binocular View
- 6. 6 Focus on Virtual Reality Features of Pascal GPUs Virtual Reality rendering features in Pascal GPUs GeForce 1080/1070/1060 NEW Quadro P6000 & P5000 As exposed in standard 3D APIs
- 7. 7 Background: NVIDA GPU Architecture Road Map Our interest NVIDIA GPU architectures of interest: Pascal What is Pascal mentioned on last slide?
- 8. 8 Conventional 3D Single-eye Scene
- 9. 9 Conventional 3D Single-eye Scene
- 10. 10 Wide Stereo Views of Same Scene Left Right
- 11. 11 Left Right
- 12. 12 Right Left
- 13. 13 Left & Right Views Are Similar but Different Left and Right eye views have different occlusion While left & right eye views look similar, must be draw independently Really only different in a small translation in X NVIDIA Single Pass Stereo supports generating left & right eye positions GPU automatically makes left & right eye versions of each primitive Then renders each view to a different framebuffer slice Exposed in OpenGL by NV_stereo_view_rendering extension Exposed for Direct3D 11 by NVAPI SinglePassStereoMode feature
- 14. 14 Image Difference of Two Views − + 0.5 = Left eye view Right eye view Clamped difference image
- 15. 15 Left & Right Views Rendered Efficiently in One Pass
- 16. 16 Lens Matched Shading Left Right
- 17. 17 Visualizing Lens Matched Shading
- 18. 18 Lens Matched Shading Left Right
- 19. 19 Visualization of Lens Matched Shading Rendering
- 20. 20 Still Needs Non-linear Warp Lens Matched Shading rendering Warped image suitable for HMD display Non-linear image
- 21. 21 Still Needs Non-linear Warp Lens Matched Shading rendering Warped image suitable for HMD display Non-linear image BUT substantially less pixel motion and distortion than conventional VR re-warping
- 22. 22 Example HMD Post-rendering Warp
- 23. 23 Lens Matched Shading with Window Rectangle Testing
- 24. 24 Lens Matched Shading with Window Rectangle Testing Nothing in black corners is shaded or even rasterized Yellow lines show overlaid 8 inclusive window rectangles Same 8 window rectangles “shared” by each view’s texture array layer
- 25. 25 Lens Matched Shading with Window Rectangle Testing
- 26. 26 What Didn’t Have to be Rendered 17% of left view pixels 17% of right view pixels
- 27. 27 Window Rectangle Testing During Rasterization New multi-vendor OpenGL extension EXT_window_rectangles Provides a set of window-space rectangles, 8 for NVIDIA GPUs Rasterizer either includes or excludes rasterization from rectangles Discards at speed-of-light As if free to rasterize to irregular rectangular regions Window Rectangles Test NEW stage OpenGL’s fragment processing pipeline
- 28. 28 Warped Lens Matched Scene Warped version of Lens Matched Shading to match HMD lens
- 29. 29 Warped Lens Matched Shading
- 30. 30 Visualizing Warp Window Rectangles
- 31. 31 What Didn’t Have to be Warped 17% of left view pixels 17% of right view pixels
- 32. 32 VR Rendering Pipeline in Images LMS Right Eye View Warped Right Eye View LMS Left Eye View Warped Left Eye View Scene LMS = Lens Matched Shading
- 33. 33 VR Rendering Pipeline in Images LMS Right Eye View Warped Right Eye View LMS Left Eye View Warped Left Eye View Scene Displayed within HMD Single Rendering Pass Single Pass Stereo + Lens Matched Shading + Window Rectangle Testing Drawn with Single Triangle Fragment Shader Warping Window Rectangle Testing Perception to user is linear rendering HMD lens “undoes” warping to provide a perceived wide field-of-view Pascal does all this efficiently in a single rendering pass! 8 viewports, 1 pass
- 34. 34 Traditional 3D Rendering Pipeline Transform 3D View Downsample Display Frame App Processing if multisampled uses pinhole camera projection, rasterizing to uniform rectilinear pixel grid monitor directly displays uniform rectilinear pixel grid Rasterize 3D View
- 35. 35 Virtual Reality Rendering Pipeline Rendering Left & Right Eye Views Transform Left View Downsample Display Frame to HMD App Processing Rasterize Left View Transform Right View Downsample Rasterize Right View HMD Warp HMD Warp warps uniform rectilinear grid based on HMD’s device profile Transform Right View Rasterize Right View Transform Right View Downsample Rasterize Right View Transform Right View HMD Warp Downsample Rasterize Right View Transform Right View second view rendering
- 36. 36 Virtual Reality Rendering Pipeline With Lens Matched Shading Transform Left View Downsample Display Frame to HMD App Processing Rasterize Left View Transform Right View Downsample Rasterize Right View HMD Warp HMD Warp Lens Matched Shading Lens Matched Shading replicates primitive to 4 projected quadrants less pixel resampling effort & error operating on LMS rasterized images
- 37. 37 Virtual Reality Rendering Pipeline With Single Pass Stereo Transform Left & Right View Downsample Display Frame to HMD App Processing Rasterize Left View Downsample Rasterize Right View HMD Warp HMD Warp one transform instance, outputs left & right position
- 38. 38 Virtual Reality Rendering Pipeline Single Pass Stereo + Lens Matched Shading Downsample Display Frame to HMD Rasterize Left View Downsample Rasterize Right View HMD Warp HMD Warp Lens Matched Shading Lens Matched Shading Transform Left & Right View App Processing + Advantages 1. Single transform for both views 2. Closer match between rendering & lens images 3. Fewer overall pixels for similar quality use window rectangle testing to bound lens region, during both rasterization & warp
- 39. 39 Pascal GPU Features for VR Unique to NVIDIA Pascal renders left & right eye In single rendering pass Halves CPU load for rendering Better concurrent texture locality OpenGL: NV_stereo_view_rendering Direct3D: NVAPI SinglePassStereoMode Better matches rendering to HMD lens Single pass Four separately projected quadrants Less rendering, less resampling error OpenGL: NV_clip_space_w_scaling Direct3D: NVAPI ModifiedWSupport
- 40. 40 Other NVIDIA Virtual Reality Initiatives
- 41. 41 More Information VRWORKS 2.0 Software Development Kit Available NOW Pascal OpenGL extensions for Virtual Reality Specification now public In official OpenGL.org extension registry OpenGL Extension Wrangler 2.0 (GLEW) support Example code in VRWORKS 2.0 SDK VRWORKS also documents NVAPI usage For Direct3D 11 Includes Examples code