SlideShare a Scribd company logo

NVIDIA OpenGL in 2016

Download as PPT, PDF
Mark Kilgard, profile picture
Mark Kilgard

Mark Kilgard discusses advancements and features of OpenGL at SIGGRAPH 2016, highlighting the evolution of the API and its support for modern graphics programming. Key topics include the introduction of new extensions, debugging tools, and features aimed at enhancing virtual reality graphics. The presentation covers recent developments in NVIDIA's GPU architecture, alongside specific examples of using OpenGL for efficient rendering across different platforms.

Technology
Related topics:
Virtual Reality
1 of 123
Downloaded 130 times
1
2
Most read
3
4
5
6
Most read
7
8
9
10
Most read
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
Mark Kilgard, July 24 SIGGRAPH 2016, Anaheim NVIDIA OpenGL in 2016
2 Mark Kilgard • Principal System Software Engineer OpenGL driver and API evolution Cg (“C for graphics”) shading language GPU-accelerated path rendering & web browser rendering • OpenGL Utility Toolkit (GLUT) implementer • Specified and implemented much of OpenGL • Author of OpenGL for the X Window System • Co-author of Cg Tutorial • Worked on OpenGL for 25 years My Background
3 NVIDIA’s OpenGL Leverage Debugging with Nsight Programmable Graphics Tegra Quadro OptiX GeForce Adobe Creative Cloud
Jeff Kiel - Manager, Graphics Tools NSIGHT VSE AND OPENGL VR
5 AGENDA Intro to Nsight & Developer Tools VR debugging GPU Range Profiling Roadmap
6 C/C++ JetPack NVTX NVIDIA Tools eXtension Compile Debug Profile Trace Hardware Support IDE Integration Standalone and CLI Getting Started…
7 NSIGHT VISUAL STUDIO EDITION 5.2 •New Range Profiler, including OpenGL and DirectX12 •Vulkan Support •New Geometry View •Oculus VR SDK support, OpenGL and DX11 •CUDA 8.0 support •VR, Vulkan, and Advanced Graphics Profiling
8 UE4’S VR ENGINE •Render pass per eye View 0 Left View 1 Right Time Depth Pass Lighting Pass . . . View 0 Left View 1 Right
9 DEMO TIME!DEMO TIME!
10 ROADMAP When you get back from SIGGRAPH: 5.2 RC1 September, 2016: 5.2 Final
11 ROADMAP Q4 2016: 5.3 The Future Tell Me What You Need!?!?
12 NVIDIA’s OpenGL Leverage Debugging with Nsight Programmable Graphics Tegra Quadro OptiX GeForce Adobe Creative Cloud
13 OpenGL Codebase Leverage Same driver code base supports multiple APIs OpenGL for Embedded, Mobile, and Web Multi-vendor, explicit, low-level graphics from Khronos
14 Still the One Truly Common & Open 3D API OS X Linux FreeBSD Solaris Android Windows
15 NVIDIA OpenGL in 2016 Provides OpenGL’s Maximally Available Superset Pascal Extensions 2015 ARB extensions OpenGL 4.5 Core Maxwell Extensions Legacy EXT & Other Compatibility Extensions OpenGL Complete Compatibility Path Rendering Multi-GPU. SLI Approaching Zero Driver Overhead NVIDIA Multi-generation GPU Initiatives DirectX inter-op Vulkan inter-op ES Enhancements Full OpenGL ES 3.2 Khronos Standard Expected Compatibility NVIDIA Initiatives GPU Generation Features
16 Background: NVIDA GPU Architecture Road Map Our interest NVIDIA GPU architectures of interest: Maxwell & Pascal What are Maxwell and Pascal mentioned on last slide?
17 OpenGL’s Recent Advancements 2014 2015 2016 New ARB Extensions 3 standard extensions, beyond 4.5 • ARB_sparse_buffer • ARB_pipeline_statistics_query • ARB_transform_feedback_overflow_query Maxwell Extensions • Novel graphics features • 14 new extensions • Global Illumination & Vector Graphics focus
18 OpenGL’s Recent Advancements 2014 2015 2016 New ARB Extensions 3 standard extensions, beyond 4.5 • ARB_sparse_buffer • ARB_pipeline_statistics_query • ARB_transform_feedback_overflow_query New ARB 2015 Extension Pack • Shader functionality • ARB_ES3_2_compatibility (shading language support) • ARB_parallel_shader_compile • ARB_gpu_shader_int64 • ARB_shader_atomic_counter_ops • ARB_shader_clock • ARB_shader_ballot • Graphics pipeline operation • ARB_fragment_shader_interlock • ARB_sample_locations • ARB_post_depth_coverage • ARB_ES3_2_compatibility (tessellation bounding box + multisample line width query) • ARB_shader_viewport_layer_array • Texture mapping functionality • ARB_texture_filter_minmax • ARB_sparse_texture2 • ARB_sparse_texture_clamp Maxwell Extensions • Novel graphics features • 14 new extensions • Global Illumination & Vector Graphics focus
19 OpenGL’s Recent Advancements 2014 2015 2016 New ARB Extensions 3 standard extensions, beyond 4.5 • ARB_sparse_buffer • ARB_pipeline_statistics_query • ARB_transform_feedback_overflow_query Maxwell Extensions • Novel graphics features • 14 new extensions • Global Illumination & Vector Graphics focus New ARB 2015 Extension Pack • Shader functionality • ARB_ES3_2_compatibility (shading language support) • ARB_parallel_shader_compile • ARB_gpu_shader_int64 • ARB_shader_atomic_counter_ops • ARB_shader_clock • ARB_shader_ballot • Graphics pipeline operation • ARB_fragment_shader_interlock • ARB_sample_locations • ARB_post_depth_coverage • ARB_ES3_2_compatibility (tessellation bounding box + multisample line width query) • ARB_shader_viewport_layer_array • Texture mapping functionality • ARB_texture_filter_minmax • ARB_sparse_texture2 • ARB_sparse_texture_clamp Pascal Extensions • Novel graphics features • 5 new extensions • Virtual Reality focus OpenGL SPIR-V Support • Standard Shader Intermediate Representation • ARB_gl_spirv • Vulkan interoperability
20 Maxwell OpenGL Extensions • Voxelization, Global Illumination, and Virtual Reality NV_viewport_array2 NV_viewport_swizzle AMD_vertex_shader_viewport_index AMD_vertex_shader_layer • Vector Graphics extensions NV_framebuffer_mixed_samples EXT_raster_multisample NV_path_rendering_shared_edge • Advanced Rasterization NV_conservative_raster NV_conservative_raster_dilate NV_sample_mask_override_coverage NV_sample_locations, now ARB_sample_locations NV_fill_rectangle • Shader Improvements NV_geometry_shader_passthrough NV_shader_atomic_fp16_vector NV_fragment_shader_interlock, now ARB_fragment_shader_interlock EXT_post_depth_coverage, now ARB_post_depth_coverage Requires GeForce 950, Quadro M series, Tegra X1, or better New Graphics Features of NVIDIA’s Maxwell GPU Architecture
21 Background: Viewport Arrays Several Maxwell (and Pascal) extensions build on Viewport Arrays Viewport arrays introduced to OpenGL standard by OpenGL 4.1 Feature of Direct3D 11 First introduced to OpenGL by NV_viewport_array extension Each viewport array element contains Viewport transform Scissor box and enable Depth range Provides N mappings of clip-space to scissored window-space Original conception Geometry shader could “steer” primitives into any of 16 viewport array elements Geometry shader would set the viewport index of a primitive Result: primitive is rasterized based on the indexed viewport array state 0 1 2 ... 15 xv yv wv hv n,f xs ys ws hs es 0 0 640 480 0,1 0,0,640,480,0 640 0 640 480 0,1 0,0,640,480,0 640 480 640 480 0,1 0,0,640,480,0 ... Viewport array state Indexed Array of Viewport & Scissor State
22 Viewport Arrays Visualized vertex shader geometry shader vertex shader vertex shader view frustum clipping viewport & depth range transform scissored rasterizer Viewport array state 0 1 2 ... 15 xv yv wv hv n,f xs ys ws hs es viewport index = 0 viewport index = 1 viewport index = 2 assembled triangle geometry shader primitive output stream (3 triangles) 0 0 640 480 0,1 0,0,640,480,0 640 0 640 480 0,1 0,0,640,480,0 640 480 640 480 0,1 0,0,640,480,0 ... resulting framebuffer
23 Viewport Index Generalized to Viewport Mask • Geometry shaders & viewport index approach proved limiting... • Common use of geometry shaders: view replication One stream of OpenGL commands  draws N views But inherently expensive for geometry shader to replicate N primitives Underlying issue: one thread of execution has to output N primitives • First fix Replace scalar viewport index per primitive with a viewport bitmask • Viewport mask does the primitive replication Viewport mask lets geometry shader output primitive to all, some, or none of viewport indices Examples 0xFFFF would replicate primitive 16 times, one primitive for each respective viewport index 0x0301 would output a primitive to viewport indices 9, 8, and 0 Maxwell’s NV_viewport_array2 extension Analogy: forcing too much water through a hose geometry shader
24 Geometry Shader Allowed to “Pass-through” of Vertex Attributes Geometry shaders are very general! 1 primitive input  N primitives output, where N is capped but still dynamic input vertex attributes can be arbitrarily recomputed Not conducive to executing efficiently Applications often just want 1 primitive in  constant N primitives out with NO change of vertex attributes though allowing for computing & output of per- primitive attributes NV_geometry_shader_passthrough supports a simpler geometry shader approach Hence more efficient Particularly useful when viewport mask allows primitive replication Restrictions 1 primitive in, 1 primitive out BUT writing the per-primitive viewport mask can force replication of 0 to 16 primitives, one for each viewport array index No modification of per-vertex attributes Allowances Still get to compute per-primitive outputs Examples: viewport mask and texture array layer Maxwell’s NV_geometry_shader_passthrough Extension
25 Analogy for Geometry Shader “Pass-through” of Vertex Attributes Geometry shader just computes per-primitive attributes and passes along primitive “Pass-through” of vertex attributes means geometry shader cannot modify them Full service geometry shader Efficient, low touch Slower, high touch Requires good behavior, many restrictions apply Fully general, anyone can use this line
26 Example Pass-through Geometry Shader layout(triangles) in; layout(triangle_strip) out; layout(max_vertices=3) out; in Inputs { vec2 texcoord; vec4 baseColor; } v_in[]; out Outputs { vec2 texcoord; vec4 baseColor; }; void main() { int layer = compute_layer(); // function not shown for (int i = 0; i < 3; i++) { gl_Position = gl_in[i].gl_Position; texcoord = v_in[i].texcoord; baseColor = v_in[i].baseColor; gl_Layer = layer; EmitVertex(); } } #extension GL_NV_geometry_shader_passthrough : require layout(triangles) in; // No output primitive layout qualifiers required. // Redeclare gl_PerVertex to pass through "gl_Position". layout(passthrough) in gl_PerVertex { vec4 gl_Position; }; // Declare "Inputs" with "passthrough" to copy members attributes layout(passthrough) in Inputs { vec2 texcoord; vec4 baseColor; }; // No output block declaration required void main() { // The shader simply computes and writes gl_Layer. We don't // loop over three vertices or call EmitVertex(). gl_Layer = compute_layer(); } Simple Example: Sends Single Triangle To Computed Layer BEFORE: Conventional geometry shader (slow) AFTER: Passthrough geometry shader (fast)
27 Outputting Layer Allows Layered Rendering • Example: Bind to particular level of 2D texture array with glFramebufferTexture Then gl_Layer output of geometry shader renders primitive to designated layer (slice) Allows Rendering to 3D Textures and Texture Arrays Texture array index for texturing, or gl_Layer for layered rendering 0 1 2 3 4 0 1 2 3 4 Mipmaplevelindex Example 2D texture array with 5 layers
28 Aside: Write Layer and Viewport Index from a Vertex Shader • Originally only geometry shaders could write the gl_ViewportIndex and gl_Layer outputs • Disadvantages Limited use of layered rendering and viewport arrays to geometry shader Often awkward to introduce a geometry shader for just to write these outputs GPU efficiency is reduced by needing to configure a geometry shader • AMD_vertex_shader_viewport_index allows gl_ViewportIndex to be written from a vertex shader • AMD_vertex_shader_layer allows gl_Layer to be written from a vertex shader • Good example where NVIDIA adopts vendor extensions for obvious API additions Generally makes OpenGL code more portable and life easier for developers in the process Maxwell’s AMD_vertex_shader_viewport_index & AMD_vertex_shader_layer Extensions
29 Further Extending Viewport Array State with Position Component Swizzling • Original viewport array state viewport transform depth range transform scissor box and enable • Maxwell extension adds new state four position component swizzle modes one for clip-space X, Y, Z, and W • Eight allowed modes GL_VIEWPORT_SWIZZLE_POSITIVE_X_NV GL_VIEWPORT_SWIZZLE_NEGATIVE_X_NV GL_VIEWPORT_SWIZZLE_POSITIVE_Y_NV GL_VIEWPORT_SWIZZLE_NEGATIVE_Y_NV GL_VIEWPORT_SWIZZLE_POSITIVE_Z_NV GL_VIEWPORT_SWIZZLE_NEGATIVE_Z_NV GL_VIEWPORT_SWIZZLE_POSITIVE_W_NV GL_VIEWPORT_SWIZZLE_NEGATIVE_W_NV Maxwell’s NV_viewport_swizzle extension Viewport array state 0 1 2 ... 15 xv yv wv hv n,f xs ys ws hs es xswyswzswwws 0 0 128 128 0,1 0,0,128,128,0 x+,y+,z+,w+ 0 0 128 128 0,1 0,0,128,128,0 y+,z+,x+,w+ 0 0 128 128 0,0 0,0,128,128,0 z+,x+,y+,w+ ... standard viewport array state NEW swizzle state
30 Reminder of Cube Map Structure • Cube map is essentially 6 images Six 2D images arranged like the faces of a cube +X, -X, +Y, -Y, +Z, -Z • Logically accessed by 3D (s,t,r) un- normalized vector Instead of 2D (s,t) Where on the cube images does the vector “poke through”? That’s the texture result • Interesting question Can OpenGL efficiently render a cube map in a single rendering pass? Cube Map Images are Position Swizzles Projected to 2D
31 Example of Cube Map Rendering
32 Example of Cube Map Rendering +X −X +Z −Z +Y−Y 0 123 4 5 Faces Labeled and Numbered by Viewport Index
33 Layer to Render Can Be Relative to Viewport Index • Geometry shader can “redeclare” the layer to be relative to the viewport index GLSL usage layout(viewport_relative) out highp int gl_Layer; • After viewport mask replication, primitive’s gl_Layer value is biased by its viewport index Allows each viewport index to render to its “own” layer • Good for single-pass cube map rendering usage Use passthrough geometry shader to write 0x3F (6 bits set, views 0 to 5) to the viewport mask Usage: gl_ViewportMask[0] = 0x3F; // Replicate primitive 6 times Set swizzle state of each viewport index to refer to proper +X, -X, +Z,-Y, +Z, -Z cube map faces Requires NV_viewport_swizzle extension Caveat: Force the window-space Z to be an eye-space planar distance for proper depth testing Requires inverse W buffering for depth testing Swizzle each view’s “Z” into output W Make sure input clip-space W is 1.0 and swizzled to output Z Means window-space Z will be one over W or a planar eye-space distance from eye, appropriate for depth testing Requires to have floating-point depth buffer for W buffering Bonus Feature of Maxwell’s NV_viewport_array2 extension
34 (Naïve) Fast Single-pass Cube Map Rendering #define pX GL_VIEWPORT_SWIZZLE_POSITIVE_X_NV #define nX GL_VIEWPORT_SWIZZLE_NEGATIVE_X_NV #define pY GL_VIEWPORT_SWIZZLE_POSITIVE_Y_NV #define nY GL_VIEWPORT_SWIZZLE_NEGATIVE_Y_NV #define pZ GL_VIEWPORT_SWIZZLE_POSITIVE_Z_NV #define nZ GL_VIEWPORT_SWIZZLE_NEGATIVE_Z_NV #define pW GL_VIEWPORT_SWIZZLE_POSITIVE_W_NV glDisable(GL_SCISSOR_TEST); glViewport(0, 0, 1024, 1024); glViewportSwizzleNV(0, nZ, nY, pW, pX); // positive X face glViewportSwizzleNV(1, pZ, nY, pW, nX); // negative X face glViewportSwizzleNV(2, pX, pZ, pW, pY); // positive Y face glViewportSwizzleNV(3, pX, nZ, pW, nX); // negative Y face glViewportSwizzleNV(4, pX, nY, pW, pZ); // positive Z face glViewportSwizzleNV(5, nX, nY, pW, nZ); // negative Z face #extension GL_NV_geometry_shader_passthrough : require #extension GL_NV_viewport_array2 : require layout(triangles) in; // No output primitive layout qualifiers required. layout(viewport_relative) out highp int gl_Layer; // Redeclare gl_PerVertex to pass through "gl_Position". layout(passthrough) in gl_PerVertex { vec4 gl_Position; }; // Declare "Inputs" with "passthrough" to copy members attributes layout(passthrough) in Inputs { vec2 texcoord; vec4 baseColor; }; void main() { gl_ViewportMask[0] = 0x3F; // Replicate primitive 6 times gl_Layer = 0; } With Maxwell’s NV_viewport_array2 & NV_viewport_swizzle Viewport array state configuration Passthrough geometry shader non-naïve version would perform per-face culling in shader Getting swizzles from this table from the OpenGL 4.5 specification ensures your swizzles matches OpenGL’s cube map layout conventions
35 GPU Voxelization, typically for Global Illumination • Concept: desire to sample the volumetric coverage within a scene Ideally sampling the emittance color & directionality from the scene too Input: polygonal meshes Output: 3D grid (texture image) where voxels hold attribute values + coverage The Other Main Justification for Viewport Swizzle Voxelization pipeline Passthrough geometry shader + viewport swizzle makes this fast
36 What’s Tricky About Voxelization • Not your regular rasterization into a 2D image! • Instead voxelization needs rasterizing into a 3D grid Represented on the GPU as a 3D texture or other 3D array of voxels • BUT our GPU and OpenGL only know how to rasterize in 2D So exploit that by rasterizing into a “fake” 2D framebuffer ARB_framebuffer_no_attachments extension allows rasterizing to framebuffer lacking any attachments for color or depth-stencil The logical framebuffer has a width & height, but no pixel storage • Approach: Rasterize a given triangle within the voxelization region on an orthogonal axis direction where triangle has the largest area (X, Y, or Z axis) Then fragment shader does (atomic) image stores to store coverage & attributes at the appropriate (x,y,z) location in 3D grid Caveat: Use conservative rasterization to avoid missing features Skip rendering a 2D image with pixels... because we need a 3D result Exact details are involved, but a fast geometry shader & viewport swizzling make Dominant Axis Selection efficient
37 What’s the Point of Voxelization? Direct lighting feels over dark Feeds a GPU Global Illumination Algorithm
38 What’s the Point of Voxelization? Feeds a GPU Global Illumination Algorithm Global illumination with ambient occlusion avoids the over-dark feel
39 Direct lighting feels over dark What’s the Point of Voxelization? Feeds a GPU Global Illumination Algorithm
40 Global Illumination with specular effects capture subtle reflections in floor too What’s the Point of Voxelization? Feeds a GPU Global Illumination Algorithm
41 What’s the Point of Voxelization? Improving the Ambient Contribution on Surfaces Flat ambient (no diffuse or specular directional lighting shown)
42 What’s the Point of Voxelization? Improving the Ambient Contribution on Surfaces Screen-space ambient occlusion improves the sense of depth a little
43 What’s the Point of Voxelization? Improving the Ambient Contribution on Surfaces True global illumination for ambient makes the volumetric structure obvious
44 Example Voxelization Sample scene
45 Example Voxelization Voxelized directional coverage
46 Example Voxelization Voxelized opacity
47 Example Voxelization Voxelized opacity, downsampled
48 Example Voxelization Voxelized opacity, downsampled twice
49 Complete Global Illumination is Complex • Complete implementation included in NVIDIA VXGI Implements Voxel Cone Tracing Part of Visual FX solutions • Implemented for DirectX 11 But all the underlying GPU technology is available as OpenGL extensions NV_viewport_array2 NV_viewport_swizzle NV_geometry_shader_passthrough NV_conservative_raster NVIDIA Provides Implementations
50 Conservative Rasterization • Mentioned on last slide as an extension used for global illumination Easy to enable: glEnable(GL_CONSERVATIVE_RASTERIZATION_NV); Additional functionality: Also provides ability to provide addition bits of sub-pixel precision • Conventional rasterization is based on point-sampling Pixel is covered if the pixel’s exact center is within the triangle Multisample antialiasing = multiple pixel locations per pixels Means rasterization can “miss” coverage if sample points for pixels or multisample locations are missed Point sampling can under-estimate ideal coverage • Conservative rasterization Guarantees coverage if any portion of triangle intersects (overlaps) the pixel square Caveat: after sub-pixel snapping to the sub-pixel grid However may rasterize “extra” pixels not overlapping pixel squares intersected by the triangle Conservative rasterization typically over-estimates ideal coverage Intended for algorithms such as GPU voxelization where missing coverage results in rendering artifacts—and be tolerant of over-estimated coverage Maxwell’s NV_conservative_raster extension
51 Conservative Rasterization Visualized • Green pixel squares have their pixel center covered by the triangle • Pink pixel squares intersect the triangle but do NOT have their pixel centered covered Consider Conventional Rasterization of a Triangle Pink pixel square indicate some degree of under-estimated coverage
52 Conservative Rasterization Visualized • Push triangle edges away from the triangle center (centroid) by half-pixel width • Constructs a new, larger (dilated) triangle covering more samples Consider Conventional Rasterization of a Dilated Triangle Notice all the pink pixel squares are within the dilated triangle
53 Conservative Rasterization Visualized • Yellow pixel square indicate pixels within dilated triangle but not intersected by the original triangle Overestimated Rasterization of a Dilated Triangle Notice all the yellow pixel squares are within the dilated triangle
54 Caveats Using Conservative Rasterization • Shared edges of non-overlapping rasterized triangles are guaranteed not to have either Double-hit pixels Pixel gaps • Rule is known as “watertight rasterization” Very useful property in practice Example: avoids double blending at edges Coverage can be under-estimated; long, skinny triangles might cover zero samples • Interpolation at a covered pixel center (or sample locations when multisampling) are guaranteed to return values within bounds of primitives vertex attributes • Conservative rasterization makes no such guarantee against double-hit pixels • Indeed double-hit pixels are effective guaranteed along shared triangle edges • Algorithms using conservative rasterization must be tolerant of over-estimated coverage Long, skinny triangles have more dilation over-estimated coverage error • Interpolation can become extrapolation when interpolation location is not within the original primitive! You have been warned shared edge
55 Conservative Rasterization Dilate Control Provides control to increase the amount of conservative dilation when GL_CONSERVATIVE_RASTERIZATION_NV is enabled Straightforward usage glConservativeRasterParameterfNV (GL_CONSERVATIVE_RASTER_DILATE_NV, 0.5f); 0.5 implies an additional half-pixel offset to the dilation, so extra conservative Actual value range is [0, 0.75] in increments of 0.25 Initial value is 0.0 Maxwell’s NV_conservative_raster_dilate extension
56 Conservative Rasterization versus Polygon Smooth • OpenGL supports polygon smooth rasterization mode since OpenGL 1.0 Example usage: glEnable(GL_POLYGON_SMOOTH) • glEnable(GL_CONSERVATIVE_RASTERIZATION_NV) is different from glEnable(GL_POLYGON_SMOOTH)? Subtle semantic difference • NVIDIA implements GL_POLYGON_SMOOTH by computing point-inside-primitive tests at multiple sample locations within each pixel square So computes fractional coverage used to modulate alpha component post-shading Typically recommended for use with glBlendFunc(GL_SRC_ALPHA_SATURATE, GL_ONE) blending enabled Polygon smooth should not over-estimate fractional coverage • Conservative rasterization works by dilation, as explained Conservative rasterization does not compute a fractional coverage So there is no modulation of alpha by the fractional coverage What’s the difference?
57 Maxwell Vector Graphics Improvements • Simple idea: mixed sample counts Improve antialiasing quality & performance of vector graphics rendering Every color samples gets N stencil/depth samples • Notion of stencil-depth test changes OLD notion: stencil & depth tests must either fail or pass, Boolean result NEW notion: multiple stencil & depth values per color sample mean the stencil & depth test can “fractionally pass” • GPU automatically modulates post-shader RGBA color by fractional test result Assumes blending configured Similar to fractional coverage blending in CPU-based vector graphics • Advantages Works very cleanly with NV_path_rendering Much reduced memory footprint ¼ at same coverage quality Much less memory bandwidth Superior path rendering anti-aliasing quality, up to 16x Minimal CPU overhead Maxwell provides super- efficient “cover” operation Maxwell’s NV_framebuffer_mixed_samples Extension glCoverageModulationNV(GL_RGBA);
58 16:1 Fractional Stencil Test Example 87.5% fractional stencil test (14 of 16) 1 color sample, 16 stencil samples 100% fractional stencil test (16 of 16) 0% fractional stencil test (0 of 16) 37.5% fractional stencil test (6 of 16) Examine Fractional Stencil Test Results
59 4 color samples, 16 stencil samples Each color sample separately modulated and blended! 0%, 100%, 0%, 50% fractional stencil test (1 of 4, 4 of 4, 0 of 4, 1 of 4) 0%, 0%, 0%, 0% fractional stencil test (0 of 4, 0 of 4, 0 of 4, 0 of 4) 100%, 100%, 100%, 100% fractional stencil test (4 of 4, 4 of 4, 4 of 4, 4 of 4) 100%, 100%, 100%, 50% fractional stencil test (4 of 4, 4 of 4, 4 of 4, 2 of 4) 16:4 Fractional Stencil Test Example Examine Fractional Stencil Test Results
60 Mixed Sample Configurations 8:8 8:4 8:2 8:1 8x1x 2x 4x 16x 1x 1:1 2:1 4:1 16:1 2x 2:2 4:2 16:2 4x 4:4 16:4 8x 16:8 Coverage/stencil samples per pixel Colorsamplesperpixel Maxwell’s NV_framebuffer_mixed_samples Extension
61 N = 1 2 4 8 16 M = 1 2 4 8 = pixel region LEGEND = color sample = sample location Mixed Samples Visualized Application determines the quality/performance/memory; many choices
62 Better Vector Graphics Performance Tiger SVG Scene GK204 (Kepler) vs. GM204 (Maxwell2) vs. GM204 with NV_framebuffer_mixed_samples 0.00 0.50 1.00 1.50 2.00 2.50 3.00 100x100 200x200 300x300 400x400 500x500 500x500 600x600 700x700 800x800 900x900 1000x1000 1100x1100 Window Resolution Millisecondsperframe GK104 16:16 GM20416:16 GM204 16:4 GM204 16:1 Kepler conventional 16x Maxwell 2 conventional 16x Maxwell 2, 16:4 & 16:1 Faster & ¼ memory footprint Smaller is better (faster!) While Using Much Less Framebuffer Memory
63 Fast, Flexible Vector Graphics Results NV_framebuffer_mixed_samples + NV_path_rendering combined Web pages Flash type games Text, even in with perspective Emojis!  Illustrations Mapping All rendering shown at 16:1 quality
64 NVIDIA OpenGL Features Integrated in Google’s Skia 2D Graphics Library • Skia is Google’s 2D graphics library • Primarily for web rendering • Used by Chromium, Firefox, and Google’s Chrome browser • Skia has support today for GPU-acceleration with OpenGL exploiting • NV_path_rendering for vector graphics filling & stroking • NV_framebuffer_mixed_samples for efficient framebuffer representation • EXT_blend_func_extended for extended Porter-Duff blending model • KHR_blend_equation_advanced for advanced Blend Modes
65 Naïve Mixed Sample Rendering Causes Artifacts • Easy to render paths with NV_path_rendering + NV_framebuffer_mixed_samples • Reason: two-step “Stencil, then Cover” approach guarantees proper coverage is fully resolved in first “stencil” pass, then color is updated in “cover” pass • Just works by design • But what if you want to render a simple convex shape like a rectangle with conventional rasterization & mixed samples? • Draw rectangle as two triangles • Into 16:1 mixed sample configuration • But fractional coverage modulation causes seam along internal edge! Requires Careful use of NV_framebuffer_mixed_samples 4x pixel magnification double blending crack   great 16x antialiasing on external edges
66 Examine the Situation Carefully • Two triangles A and B • Where A is 100% fine • Where B is 100% fine • External edge of A is properly antialiased • External edge of B is properly antialiased • PROBLEM is shared edge • Both triangles claim fractional coverage along this edge • Causes Double Blending • Can we “fix” rasterization so either A or B, but never both claim the shared edge? • YES, Maxwell GPUs can • Using NV_sample_mask_override_coverage extension Maxwell’s NV_sample_mask_override_coverage Extension Helps 100% A 100% B A’s antialiased edge B’s antialiased edge Problematic double-blended shared edge
67 Solution: Triangle A Claims Coverage or B Claims, But not Both void main() { gl_FragColor = gl_Color; } #version 400 compatibility #extension GL_NV_sample_mask_override_coverage : require layout(override_coverage) out int gl_SampleMask[]; const int num_samples = 16; const int all_sample_mask = 0xffff; void main() { gl_FragColor = gl_Color; if (gl_SampleMaskIn[0] == all_sample_mask) { gl_SampleMask[0] = all_sample_mask; } else { int mask = 0; for (int i=0; i<num_samples; i++) { vec2 st; st = interpolateAtSample(gl_TexCoord[0].xy, i); if (all(lessThan(abs(st),vec2(1)))) mask |= (1 << i); } int otherMask = mask & ~gl_SampleMaskIn[0]; if (otherMask > gl_SampleMaskIn[0]) gl_SampleMask[0] = 0; else gl_SampleMask[0] = mask; } } Handle in fragment shader: by overriding the sample mask coverage BEFORE: Simply output interpolated color AFTER: Interpolate color + resolve overlapping coverage claims trivial fragment shader
68 Solution: Triangle A Claims Coverage or B Claims, But not Both void main() { gl_FragColor = gl_Color; } #version 400 compatibility #extension GL_NV_sample_mask_override_coverage : require layout(override_coverage) out int gl_SampleMask[]; const int num_samples = 16; const int all_sample_mask = 0xffff; void main() { gl_FragColor = gl_Color; if (gl_SampleMaskIn[0] == all_sample_mask) { gl_SampleMask[0] = all_sample_mask; } else { int mask = 0; for (int i=0; i<num_samples; i++) { vec2 st; st = interpolateAtSample(gl_TexCoord[0].xy, i); if (all(lessThan(abs(st),vec2(1)))) mask |= (1 << i); } int otherMask = mask & ~gl_SampleMaskIn[0]; if (otherMask > gl_SampleMaskIn[0]) gl_SampleMask[0] = 0; else gl_SampleMask[0] = mask; } } Handle in fragment shader: by overriding the sample mask coverage BEFORE: Simply output interpolated color AFTER: Interpolate color + resolve overlapping coverage claims additional re-rasterization epilogue early accept optimization sample mask override coverage support
69 NV_sample_mask_override_coverage • BEFORE: Fragment shaders can access sample mask for multisample rasterization • Indicates which individual coverage samples with a pixel are covered by the fragment • Fragment shader can also “clear” bits in the sample mask to discard samples • But in standard OpenGL, no way to “set” bits to augment coverage • Fragment’s output sample mask is always bitwise AND’ed with original sample mask • NOW: Maxwell’s NV_sample_mask_override_coverage allows overriding coverage! • The fragment shader can completely rewrite the sample mask • Clearing bits still discards coverage • BUT setting bits not previously set augments coverage • Powerful capability enables programmable rasterization algorithms • Like example in previous slide to fix double blending artifacts What does it allow?
70 Other Sample Mask Coverage Override Uses • Handles per-sample stencil test for high-quality sub-pixel clipping • These techniques integrated today into Skia Works for general quadrilaterals, even in drawn in perspective Adapts well to drawing circles and ellipses And even rounded rectangles Example: 16x quality blended ellipses
71 Maxwell OpenGL Extensions • Voxelization, Global Illumination, and Virtual Reality NV_viewport_array2 NV_viewport_swizzle AMD_vertex_shader_viewport_index AMD_vertex_shader_layer • Vector Graphics extensions NV_framebuffer_mixed_samples EXT_raster_multisample NV_path_rendering_shared_edge • Advanced Rasterization NV_conservative_raster NV_conservative_raster_dilate NV_sample_mask_override_coverage NV_sample_locations, now ARB_sample_locations NV_fill_rectangle • Shader Improvements NV_geometry_shader_passthrough NV_shader_atomic_fp16_vector NV_fragment_shader_interlock, now ARB_fragment_shader_interlock EXT_post_depth_coverage, now ARB_post_depth_coverage Requires GeForce 950, Quadro M series, Tegra X1, or better New Graphics Features of NVIDIA’s Maxwell GPU Architecture Lacked time to talk about these extensions
72 • Graphics pipeline operation •ARB_fragment_shader_interlock •ARB_sample_locations •ARB_post_depth_coverage •ARB_ES3_2_compatibility •Tessellation bounding box •Multisample line width query •ARB_shader_viewport_layer_array • Texture mapping functionality •ARB_texture_filter_minmax •ARB_sparse_texture2 •ARB_sparse_texture_clamp • Shader functionality •ARB_ES3_2_compatibility •ES 3.2 shading language support •ARB_parallel_shader_compile •ARB_gpu_shader_int64 •ARB_shader_atomic_counter_ops •ARB_shader_clock •ARB_shader_ballot 2015: In Review OpenGL in 2015 ratified 13 new standard extensions
73 Need a Full Refresher on 2014 and 2015 OpenGL? • Honestly, lots of functionality in 2014 & 2015 if you’ve not followed carefully Available @ http://www.slideshare.net/Mark_Kilgard
74 Pascal GPU OpenGL Extensions • Pascal has 5 new OpenGL extensions • Major goal: improving Virtual Reality support • Several extensions used in combination • NV_stereo_view_rendering • efficiently render left & right eye views in single rendering pass • NV_viewport_array2 + NV_geometry_shader_passthrough—discussed already • NV_clip_space_w_scaling • extends viewport array state with per-viewport re-projection • EXT_window_rectangles • fast inclusive/exclusive rectangle testing during rasterization • Multi-vendor extension supported on all modern NVIDIA GPUs • High-end Virtual Reality with two GPUs • New explicit NV_gpu_multicast extension • Render left & right eyes with distinct GPUs New for 2016
75 Basic question Why should the Virtual Reality (VR) image shown in a Head Mounted Display (HMD) feel real? Ignoring head tracking and the realism of the image itself... just focused on the image generation
76 Why HMD’s Image ≈ Perception of Reality ≈ lens(lens-1(rendered image)) ≈ lens(screen) HMD image ≈ lens image screen ≈ lens-1(rendered image) ≈ rendered image rendered image ≈ pin hole image pin hole image ≈ eye view eye view ≈ perception of reality by warping by rendering model by anatomy by psychology by optics lens image = lens(screen) ≈ pin hole image ≈ eye view ≈ perception of reality image ≈ lens(lens-1(image)) by composition Portion of transformation involving GPU rendering & resampling Twin goals 1. Minimize HMD resampling error 2. Increase rendering efficiency
77 Goal of Head Mounted Display (HMD) Rendering •Goal: perceived HMD image ≈ visual perception of reality • Each image pair on HMD screen, as seen through its HMD lens, should be perceived as images of the real world •Assume pin hole camera image ≈ real world • Traditional computer graphics assumes this • Perspective 3D rasterization idealizes a pin hole camera • Human eye ball also approximately a pin hole camera •perceived HMD image = lens(screen image) • Function lens() warps image as optics of HMD lens does •screen image = lens-1 (pin hole camera image) • Function lens-1 () is inverse of the lens image warp •perceived image ≈ lens(lens-1 (pin hole camera image)) •pin hole camera image ≈ eye view
78 Pin Hole Camera Ideal Albrecht Dürer: Artist Drawing with Perspective Device Normal computer graphics generally good at rendering “pin hole” camera images And people are good at interpreting such images as 3D scenes But HMDs have a non-linear image warping due to lens distortion
79 Lens Distortion in HMD •Head-mounted Display (HMD) magnifies its screen with a lens •Why is a lens needed? • To feel immersive • Immersion necessitates a wide field- of-view • So HMD lens “widens” the HMD screen’s otherwise far too narrow field-of-view •Assume a radial symmetric magnify • Could be a fancier lens & optics • BUT consumer lens should be inexpensive & lightweight Graph paper viewed & magnified through HMD lens
80 Example HMD Post-rendering Warp
81 Lens Performs a Radial Symmetric Warp Adding circles to image shows distortion increases as the radius increases Original Image Overlaid with circles
82 Pin-hole Camera Image Assumptions •Assume a conventionally rendered perspective image • In other words a pin-hole camera image •r is the distance of a pixel (x,y) relative to the center of the image at (0,0) so •Theta is the angle of the pixel relative to the origin •Assume pin hole camera image has maximum radius of 1 • So the X & Y extent of the images is [-1..1] 22 yxr += θ θ sin cos ry rx = =
83 Radius Remapping for an HMD Magnifying Lens • A lens in an HMD magnifies the image • What is magnification really? • Magnifying takes a pixel at a given radius and “moves it out” to a larger radius in the magnified image • In the HMD len’s image, each pin-hole camera pixel radius r is mapped to alternate radius rlensImage • This maps each pixel (x,y) in the pin-hole camera image to an alternate location (xlensImage,ylensImage) • Without changing theta agedisplaylensImage rrkrkr Im 4 2 2 1 ...)1( +++= ...1 4 2 2 1 +++ = rkrk r r lensImage gedisplayIma Essentially a Taylor series approximating actual optics of lens
84 Lens Function Coefficients for Google Cardboard Lens coefficients k1 & k2 are values that can be measured Additional coefficients (k3, etc.) are negligible Coefficients for typical lens in Google Cardboard k1 = 0.22 k2 = 0.26 Big question Can we render so the amount of resampling necessary to invert a particular lens’s distortion is minimized?
85 Radius Remapping for Lens Matched Shading (LMS) • Assume a conventionally rendered perspective image • In other words a pin-hole camera image • r is the distance of a pixel (x,y) relative to the center of the image at (0,0) so • Theta is the angle of the pixel relative to the origin • Lens Matched Shading provides an alternate radius rLMS for the same pixel (xLMS,yLMS) • This maps each pixel (x,y) to an alternate location • Without changing theta θ θ sin cos LMSLMS LMSLMS ry rx = = θ θ sin cos ry rx = = θθ sincos1 rprp r rLMS ++ = 22 yxr += OLD: Conventional “pin hold” camera rendering NEW: Lens Matched Shading rendering
86 Concentric circles in pin hole camera view gets “squished” by inverse lens transform HMD’s Inverse Lens Warp pin hole camera view (conventionally rendered image) inverse lens warp view (HMD screen) k1 = 0.22 k2 = 0.26 4 2 2 1 21 rkrk r r lensImage gedisplayIma ++ =
87 Lens Matched Shading p = 0.26007 θθ sincos1 rprp r rLMS ++ = pin hole camera view Lens Matched Shading (rendered framebuffer image) Concentric circles in pin hole camera view gets “projected” towards origin
88 Complete Process of Lens Matched Shading ideal pin hole camera view rendered image with lens matched shading lens warped image image as perceived viewed through HMD lens while different, these two images are “well matched” so warp between them minimizes pixel movement and resampling
89 What is Optimal Value for p? A reasonable measure of optimality is root mean square error of difference between LMS and inverse lens warp radii over entire lens So what p minimizes this integral for a particular lens’s coefficients When k1 = 0.22 & k2 = 0.26, optimal p ≈ 0.26007 θ θθ π ddrr rprp r rkrk r 2 2 0 1 0 4 2 2 1 sincos121∫∫         ++ − ++ * Analysis assumes a Google Cardboard-type device; Oculus has asymmetric visible screen region
90 Matched Overlap of Lens Matched Shading and Lens Warped Image k1 = 0.22 k2 = 0.26 p = 0.26007 Root Mean Square (RMS) error = 0.0598
91 Much Worse Overlap of Conventional Projection and Lens Warped Image Root Mean Square (RMS) error = 0.273 k1 = 0.22 k2 = 0.26 p = 0
92 Advantages of Lens Matched Shading •What is rendered by GPU is closer (less error) to what the HMD needs to display than conventional “pin hole” camera rendering •Means less resampling error • There’s still a non-linear re-warping necessary • However the “pixel movement” for the warp is greatly reduced •Another advantage: fewer pixels need be rendered for same wide field of view •Also want application to render left & right views with LMS in a single efficient rendering pass
93 Single-eye Scene Simple 3D scene
94 Stereo Views of Same Scene Left and Right eye view of same simple scene Two scenes are slightly different if compared
95 Swapped Stereo Views Right and Left (swapped) eye view of same simple scene Two scenes are slightly different if compared
96 Image Difference of Two Views − + 0.5 = Left eye view Right eye view Clamped difference image
97 Lens Matched Shading Same left & right eye view but rendered with w scaling
98 Lens Matched Shading Quadrants Same left & right eye view but rendered with w scaling Each quadrant gets different projection to “tilt to center”
99 Visualization of Lens Matched Shading Rendering
100 Warped Lens Matched Shaped Warped version of lens shading to match HMD lens
101 Lens Matched Shading with Window Rectangle Testing Same Lens Matched Shading but with EXT_window_rectangles Nothing in black corners is shaded or even rasterized
102 Lens Matched Shading with Window Rectangle Testing Nothing in black corners is shaded or even rasterized Yellow lines show overlaid 8 inclusive window rectangles Same 8 window rectangles “shared” by each view’s texture array layer
103 Standard OpenGL Per-fragment Operations
104 NEW Window Rectangles Test in Per-fragment Operations Window Rectangles Test NEW stage
105 Straightforward API • glWindowRectanglesEXT(GLenum mode, GLsizei count, const GLint rects[]); • mode can be either GL_INCLUSIVE_EXT or GL_EXCLUSIVE_EXT • count can be from 0 to maximum number of supported window rectangles • Must be at least 4 (for AMD hardware) • NVIDIA hardware supports 8 • Rectangles allowed to overlap and/or disjoint • Each rectangle is (x,y,width,height) • width & height must be non-negative • Initial state • GL_EXCLUSIVE_NV with zero rectangles • Excluding rendering from zero rectangles means nothing is discarded by window rectangles test Multi-vendor EXT_window_rectangles Extension
106 Lens Matched Shading with Window Rectangle Testing Nothing in black corners is shaded or even rasterized Yellow lines show overlaid 8 inclusive window rectangles Same 8 window rectangles “shared” by each view’s texture array layer
107 Warped Lens Matched Shading with Window Rectangle Testing during Rendering Identical as “Lens Matched Shading” despite corners not being rasterized because corners don’t contribute to warped version
108 Warped Lens Matched Shading with Win. Rect. Testing during Rendering & Warping Same prior image, but warp now uses window rectangles Avoids wasting time warping corners not visible through lens
109 Visualizing Warp Window Rectangles Point: Window rectangle testing used TWICE #1 during Lens Matched Shading rendering pass #2 during warping pass
110 VR Rendering Pipeline LMS Right Eye View Warped Right Eye View LMS Left Eye View Warped Left Eye View Scene Displayed within HMD Single Rendering Pass Single Pass Stereo + Lens Matched Shading + Window Rectangle Testing Drawn with Single Triangle Fragment Shader Warping Window Rectangle Testing Perception to user is linear rendering HMD lens “undoes” warping to provide a perceived wide field-of-view Pascal does all this efficiently in a single rendering pass! 8 viewports, 1 pass
111 OpenGL Extensions Used in LMS VR Pipeline • Allows vertex shader to output two clip-space positions • (x1,y,z,w) and (x2,y,z,w) • Results in TWO primitives one for left eye & one for right eye • New GLSL built-ins • gl_SecondaryPositionNV • Like gl_Position but for “second eye’s view” • gl_SecondaryViewportMaskNV[] • Like gl_ViewportMaskNV[] but for “second eye’s view” • Also can steer primitives to different texture array slices • layout(secondary_view_offset = 1) int gl_Layer; Pascal’s NV_stereo_view_rendering Extension
112 OpenGL Extensions Used in LMS VR Pipeline Adds a new set of state to viewport array elements Each viewport index can recompute clip space as w = w + A x + B y Pascal’s NV_clip_space_w_scaling Extension Viewport array state 0 1 2 3 ... 15 xv yv wv hv n,f xs ys ws hs es xswyswzswwws A,B 0 0 1024 1024 0,1 0,0, 512,512,1 x+,y+,z+,w+ −0.26,−0.26 0 0 1024 1024 0,1 512,0, 512,512,1 y+,z+,x+,w+ +0.26,−02.6 0 0 1024 1024 0,1 512,0, 512,512,1 z+,x+,y+,w+ −0.26,−0.26 ... standard viewport array state swizzle state NEW w scaling 0 0 1024 1024 0,1 512,512, 512,512,1 z+,x+,y+,w+ +0.26,+0.26 Four quadrants for Lens Matched Shading
113 Example Lens Matched Shading Rendered Image Example image A=+0.2, B=+0.2A=−0.2, B=+0.2 A=−0.2, B=−0.2 A=+0.2, B=−0.2
114 More Information on NVIDIA Virtual Reality GPU Support Growing Software Development Kit (SDK) for Virtual Reality Focus on GPU efficiency Whitepapers and sample code Both OpenGL and Direct3D supported https://developer.nvidia.com/vrworks Get the VRWORKS 2.0 SDK
115 Still More Pascal OpenGL Extensions NVX_blend_equation_advanced_multi_draw_buffers • No API, simply relaxes error restriction so advanced blend modes from KHR_blend_equation_advanced & NV_blend_equation_advanced work with more than 1 color attachment • Important for CMYK rendering NV_conservative_raster_pre_snap_triangles • More Conservative Rasterization control • Allows conservative rendering dilation prior to sub-pixel snapping NV_shader_atomic_float64 • Atomic shader operations on double-precision values CYMK color space rendering with multiple color attachments Pascal’s non-Virtual Reality Enhancements
116 OpenGL extension exposing Khronos intermediate language for parallel compute and graphics New standard Khronos extension for OpenGL Just announced! July 22, 2016 Allows compiled SPIR-V code to be passed directly to OpenGL driver Accepts SPIR-V output from open source Glslang Khronos Reference compiler https://github.com/KhronosGroup/glslang Other compilers can target SPIR-V too Khronos standard extension ARB_gl_spirv +
117 SPIR-V Ecosystem LLVM Third party kernel and shader Languages •SPIR-V •Khronos defined and controlled cross-API intermediate language •Native support for graphics and parallel constructs •32-bit Word Stream •Extensible and easily parsed •Retains data object and control flow information for effective code generation and translation OpenCL C++OpenCL C GLSL Khronos has open sourced these tools and translators IHV Driver Runtimes Other Intermediate Forms SPIR-V Validator SPIR-V (Dis)Assembler LLVM to SPIR-V Bi-directional Translator Khronos plans to open source these tools soon https://github.com/KhronosGroup/SPIR/tree/spirv-1.1 Open source C++ front-end released HLSL Khronos has open sourced these tools and translators Khronos plans to open source these tools soon Khronos has open sourced these tools and translators HLSL Khronos plans to open source these tools soon Khronos has open sourced these tools and translators GLSLHLSL Khronos plans to open source these tools soon Khronos has open sourced these tools and translators OpenCL C GLSLHLSL Khronos plans to open source these tools soon Khronos has open sourced these tools and translators OpenCL C++OpenCL C GLSLHLSL Khronos plans to open source these tools soon Khronos has open sourced these tools and translators LLVM to SPIR-V Bi-directional Translator OpenCL C++OpenCL C GLSLHLSL Khronos plans to open source these tools soon Khronos has open sourced these tools and translators SPIR-V Validator LLVM to SPIR-V Bi-directional Translator OpenCL C++OpenCL C GLSLHLSL Khronos plans to open source these tools soon Khronos has open sourced these tools and translators SPIR-V (Dis)Assembler SPIR-V Validator LLVM to SPIR-V Bi-directional Translator OpenCL C++OpenCL C GLSLHLSL Khronos plans to open source these tools soon Khronos has open sourced these tools and translators New with ARB_gl_spirv
118 NVIDIA’s SIGGRAPH Driver Update •NVIDIA historically releases a “developer” driver at SIGGRAPH with support for all Khronos standard extensions announced at SIGGRAPH • This year too  •Monday (July 25, 2016) NVIDIA will put out a new SIGGRAPH driver • ARB_gl_spirv • Major extension in terms of compiler infrastructure & shader support • EXT_window_rectangles • Updates to Pascal OpenGL extensions • For Windows and Linux operating systems Developed driver with ARB_gl_spirv extension https://developer.nvidia.com/opengl-driver
119 GLEW Support Available NOW GLEW = The OpenGL Extension Wrangler Library Open source library Pre-built distribution: http://glew.sourceforge.net/ Source code: https://github.com/nigels-com/glew Your one-stop-shop for API support for all OpenGL extension APIs Just released GLEW 2.0 (July 2016) provides API support for ARB_gl_spirv EXT_window_rectangles All of NVIDIA’s Maxwell extensions All of NVIDIA’s Pascal extensions All other NVIDIA multi-GPU generation initiatives Examples: NV_path_rendering, NV_command_list, NV_gpu_multicast Thanks to Nigel Stewart, GLEW maintainer, for this
120 NVIDIA OpenGL in 2016 Provides OpenGL’s Maximally Available Superset Pascal Extensions 2015 ARB extensions OpenGL 4.5 Core Maxwell Extensions Legacy EXT & Other Compatibility Extensions OpenGL Complete Compatibility Path Rendering Multi-GPU. SLI Approaching Zero Driver Overhead NVIDIA Multi-generation GPU Initiatives DirectX inter-op Vulkan inter-op ES Enhancements Full OpenGL ES 3.2 Khronos Standard Expected Compatibility NVIDIA Initiatives GPU Generation Features
121 Last Words •Lots of new OpenGL features in NVIDIA’s 2016 Driver •Highlights • OpenGL 2015 Khronos standard extensions all supported by NVIDIA • Maxwell’s features for • GPU Voxelization & Global Illumination • Vector Graphics • And Pascal supports all these features too • Pascal’s features for efficient Virtual Reality rendering • NVIDIA supports new ARB_gl_spirv extension • Provides shader compilation inter-operability for Vulkan and OpenGL
122 SIGGRAPH Paper Using OpenGL to Check Out • Harnesses OpenGL-based GPU tessellation • Avoids the complex patch splitting in current OpenSubdiv approach • Wednesday, July 27 • Ballroom C/D/E • 3:45 to 5:55pm session
NVIDIA OpenGL in 2016

More Related Content

PDF
OpenGL 4.4 - Scene Rendering Techniques
Narann29
 
PDF
Advanced Scenegraph Rendering Pipeline
Narann29
 
PPTX
Optimizing the Graphics Pipeline with Compute, GDC 2016
Graham Wihlidal
 
PPT
NVIDIA OpenGL and Vulkan Support for 2017
Mark Kilgard
 
PPTX
FrameGraph: Extensible Rendering Architecture in Frostbite
Electronic Arts / DICE
 
PPSX
Advancements in-tiled-rendering
mistercteam
 
PPTX
Beyond porting
Cass Everitt
 
PPTX
Progressive Lightmapper: An Introduction to Lightmapping in Unity
Unity Technologies
 
OpenGL 4.4 - Scene Rendering Techniques
Narann29
 
Advanced Scenegraph Rendering Pipeline
Narann29
 
Optimizing the Graphics Pipeline with Compute, GDC 2016
Graham Wihlidal
 
NVIDIA OpenGL and Vulkan Support for 2017
Mark Kilgard
 
FrameGraph: Extensible Rendering Architecture in Frostbite
Electronic Arts / DICE
 
Advancements in-tiled-rendering
mistercteam
 
Beyond porting
Cass Everitt
 
Progressive Lightmapper: An Introduction to Lightmapping in Unity
Unity Technologies
 

What's hot (20)

PDF
Graphics Gems from CryENGINE 3 (Siggraph 2013)
Tiago Sousa
 
PPTX
Triangle Visibility buffer
Wolfgang Engel
 
PPT
Destruction Masking in Frostbite 2 using Volume Distance Fields
Electronic Arts / DICE
 
PPTX
What is OpenGL ?
Mohammad Hosein Nemati
 
PPTX
Five Rendering Ideas from Battlefield 3 & Need For Speed: The Run
Electronic Arts / DICE
 
PPTX
Low-level Shader Optimization for Next-Gen and DX11 by Emil Persson
AMD Developer Central
 
PPTX
OpenGL basics
Mohammad Hosein Nemati
 
PDF
Screen Space Decals in Warhammer 40,000: Space Marine
Pope Kim
 
PPT
Star Ocean 4 - Flexible Shader Managment and Post-processing
umsl snfrzb
 
PDF
Taking Killzone Shadow Fall Image Quality Into The Next Generation
Guerrilla
 
PPTX
A Scalable Real-Time Many-Shadowed-Light Rendering System
Bo Li
 
PPTX
Stochastic Screen-Space Reflections
Electronic Arts / DICE
 
PPT
NVIDIA OpenGL 4.6 in 2017
Mark Kilgard
 
PPTX
GS-4106 The AMD GCN Architecture - A Crash Course, by Layla Mah
AMD Developer Central
 
PPT
OpenGL 3.2 and More
Mark Kilgard
 
PDF
More explosions, more chaos, and definitely more blowing stuff up
Intel® Software
 
PPSX
Holy smoke! Faster Particle Rendering using Direct Compute by Gareth Thomas
AMD Developer Central
 
PPT
NVIDIA's OpenGL Functionality
Mark Kilgard
 
PPT
Secrets of CryENGINE 3 Graphics Technology
Tiago Sousa
 
PDF
Rendering Tech of Space Marine
Pope Kim
 
Graphics Gems from CryENGINE 3 (Siggraph 2013)
Tiago Sousa
 
Triangle Visibility buffer
Wolfgang Engel
 
Destruction Masking in Frostbite 2 using Volume Distance Fields
Electronic Arts / DICE
 
What is OpenGL ?
Mohammad Hosein Nemati
 
Five Rendering Ideas from Battlefield 3 & Need For Speed: The Run
Electronic Arts / DICE
 
Low-level Shader Optimization for Next-Gen and DX11 by Emil Persson
AMD Developer Central
 
OpenGL basics
Mohammad Hosein Nemati
 
Screen Space Decals in Warhammer 40,000: Space Marine
Pope Kim
 
Star Ocean 4 - Flexible Shader Managment and Post-processing
umsl snfrzb
 
Taking Killzone Shadow Fall Image Quality Into The Next Generation
Guerrilla
 
A Scalable Real-Time Many-Shadowed-Light Rendering System
Bo Li
 
Stochastic Screen-Space Reflections
Electronic Arts / DICE
 
NVIDIA OpenGL 4.6 in 2017
Mark Kilgard
 
GS-4106 The AMD GCN Architecture - A Crash Course, by Layla Mah
AMD Developer Central
 
OpenGL 3.2 and More
Mark Kilgard
 
More explosions, more chaos, and definitely more blowing stuff up
Intel® Software
 
Holy smoke! Faster Particle Rendering using Direct Compute by Gareth Thomas
AMD Developer Central
 
NVIDIA's OpenGL Functionality
Mark Kilgard
 
Secrets of CryENGINE 3 Graphics Technology
Tiago Sousa
 
Rendering Tech of Space Marine
Pope Kim
 
Ad

Viewers also liked (20)

PPTX
Approaching zero driver overhead
Cass Everitt
 
PPTX
OpenGL 4.5 Update for NVIDIA GPUs
Mark Kilgard
 
PDF
OpenGL NVIDIA Command-List: Approaching Zero Driver Overhead
Tristan Lorach
 
PPTX
Migrating from OpenGL to Vulkan
Mark Kilgard
 
PDF
Modern OpenGL Usage: Using Vertex Buffer Objects Well
Mark Kilgard
 
PPT
CS 354 More Graphics Pipeline
Mark Kilgard
 
PPT
GTC 2012: NVIDIA OpenGL in 2012
Mark Kilgard
 
PPT
CS 354 Vector Graphics & Path Rendering
Mark Kilgard
 
PPT
CS 354 Acceleration Structures
Mark Kilgard
 
PPTX
Siggraph 2016 - Vulkan and nvidia : the essentials
Tristan Lorach
 
PPT
CS 354 Project 1 Discussion
Mark Kilgard
 
PPT
CS 354 Introduction
Mark Kilgard
 
PPT
CS 354 Ray Casting & Tracing
Mark Kilgard
 
PDF
Buttefly
Renaldas Zioma
 
PDF
NV_path_rendering Frequently Asked Questions
Mark Kilgard
 
PPT
NV_path rendering Functional Improvements
Mark Kilgard
 
PPTX
Exposición Scanline
jdtorrespal
 
PPT
GTC 2009 OpenGL Gold
Mark Kilgard
 
PPT
SIGGRAPH Asia 2012 Exhibitor Talk: OpenGL 4.3 and Beyond
Mark Kilgard
 
PDF
Programming with NV_path_rendering: An Annex to the SIGGRAPH Asia 2012 paper...
Mark Kilgard
 
Approaching zero driver overhead
Cass Everitt
 
OpenGL 4.5 Update for NVIDIA GPUs
Mark Kilgard
 
OpenGL NVIDIA Command-List: Approaching Zero Driver Overhead
Tristan Lorach
 
Migrating from OpenGL to Vulkan
Mark Kilgard
 
Modern OpenGL Usage: Using Vertex Buffer Objects Well
Mark Kilgard
 
CS 354 More Graphics Pipeline
Mark Kilgard
 
GTC 2012: NVIDIA OpenGL in 2012
Mark Kilgard
 
CS 354 Vector Graphics & Path Rendering
Mark Kilgard
 
CS 354 Acceleration Structures
Mark Kilgard
 
Siggraph 2016 - Vulkan and nvidia : the essentials
Tristan Lorach
 
CS 354 Project 1 Discussion
Mark Kilgard
 
CS 354 Introduction
Mark Kilgard
 
CS 354 Ray Casting & Tracing
Mark Kilgard
 
Buttefly
Renaldas Zioma
 
NV_path_rendering Frequently Asked Questions
Mark Kilgard
 
NV_path rendering Functional Improvements
Mark Kilgard
 
Exposición Scanline
jdtorrespal
 
GTC 2009 OpenGL Gold
Mark Kilgard
 
SIGGRAPH Asia 2012 Exhibitor Talk: OpenGL 4.3 and Beyond
Mark Kilgard
 
Programming with NV_path_rendering: An Annex to the SIGGRAPH Asia 2012 paper...
Mark Kilgard
 
Ad

Similar to NVIDIA OpenGL in 2016 (20)

PPT
Introduction To Geometry Shaders
pjcozzi
 
PPTX
OpenGL Shading Language
Jungsoo Nam
 
PDF
Introduction of openGL
Gary Yeh
 
PPT
An Introduction to NV_path_rendering
Mark Kilgard
 
PPTX
Python Raster Function - Esri Developer Conference - 2015
akferoz07
 
PPT
Hardware Shaders
gueste52f1b
 
PPT
GTC 2009 OpenGL Barthold
Mark Kilgard
 
PDF
Alex_Vlachos_Advanced_VR_Rendering_Performance_GDC2016
Alex Vlachos
 
PDF
Dynamic sorting algorithm vizualizer.pdf
AgneshShetty
 
PPT
Creating Custom Charts With Ruby Vector Graphics
David Keener
 
PDF
Hpg2011 papers kazakov
mistercteam
 
PDF
Opengl basics
pushpa latha
 
PPT
OpenGL for 2015
Mark Kilgard
 
PPT
OpenGL 4 for 2010
Mark Kilgard
 
PDF
Webrender 1.0
Daosheng Mu
 
PDF
What's new in Hexagon-Geospatial Power Portfolio 2016
Planetek Italia Srl
 
PPT
D3 D10 Unleashed New Features And Effects
Thomas Goddard
 
DOCX
Task 2
Thomas_Giblin_16
 
PDF
"Combining Flexibility and Low-Power in Embedded Vision Subsystems: An Applic...
Edge AI and Vision Alliance
 
PDF
Chapter-3.pdf
mekelle university(EiT-M)
 
Introduction To Geometry Shaders
pjcozzi
 
OpenGL Shading Language
Jungsoo Nam
 
Introduction of openGL
Gary Yeh
 
An Introduction to NV_path_rendering
Mark Kilgard
 
Python Raster Function - Esri Developer Conference - 2015
akferoz07
 
Hardware Shaders
gueste52f1b
 
GTC 2009 OpenGL Barthold
Mark Kilgard
 
Alex_Vlachos_Advanced_VR_Rendering_Performance_GDC2016
Alex Vlachos
 
Dynamic sorting algorithm vizualizer.pdf
AgneshShetty
 
Creating Custom Charts With Ruby Vector Graphics
David Keener
 
Hpg2011 papers kazakov
mistercteam
 
Opengl basics
pushpa latha
 
OpenGL for 2015
Mark Kilgard
 
OpenGL 4 for 2010
Mark Kilgard
 
Webrender 1.0
Daosheng Mu
 
What's new in Hexagon-Geospatial Power Portfolio 2016
Planetek Italia Srl
 
D3 D10 Unleashed New Features And Effects
Thomas Goddard
 
Task 2
Thomas_Giblin_16
 
"Combining Flexibility and Low-Power in Embedded Vision Subsystems: An Applic...
Edge AI and Vision Alliance
 
Chapter-3.pdf
mekelle university(EiT-M)
 

More from Mark Kilgard (18)

PDF
D11: a high-performance, protocol-optional, transport-optional, window system...
Mark Kilgard
 
PPT
Computers, Graphics, Engineering, Math, and Video Games for High School Students
Mark Kilgard
 
PPT
Virtual Reality Features of NVIDIA GPUs
Mark Kilgard
 
PPT
EXT_window_rectangles
Mark Kilgard
 
PPT
Slides: Accelerating Vector Graphics Rendering using the Graphics Hardware Pi...
Mark Kilgard
 
PDF
Accelerating Vector Graphics Rendering using the Graphics Hardware Pipeline
Mark Kilgard
 
PPT
SIGGRAPH Asia 2012: GPU-accelerated Path Rendering
Mark Kilgard
 
PPT
GPU accelerated path rendering fastforward
Mark Kilgard
 
PDF
GPU-accelerated Path Rendering
Mark Kilgard
 
PPT
SIGGRAPH 2012: GPU-Accelerated 2D and Web Rendering
Mark Kilgard
 
PPT
SIGGRAPH 2012: NVIDIA OpenGL for 2012
Mark Kilgard
 
PPT
GTC 2012: GPU-Accelerated Path Rendering
Mark Kilgard
 
PPT
CS 354 Final Exam Review
Mark Kilgard
 
PPT
CS 354 Surfaces, Programmable Tessellation, and NPR Graphics
Mark Kilgard
 
PPT
CS 354 Performance Analysis
Mark Kilgard
 
PPT
CS 354 Global Illumination
Mark Kilgard
 
PPT
CS 354 Typography
Mark Kilgard
 
PPT
CS 354 Bezier Curves
Mark Kilgard
 
D11: a high-performance, protocol-optional, transport-optional, window system...
Mark Kilgard
 
Computers, Graphics, Engineering, Math, and Video Games for High School Students
Mark Kilgard
 
Virtual Reality Features of NVIDIA GPUs
Mark Kilgard
 
EXT_window_rectangles
Mark Kilgard
 
Slides: Accelerating Vector Graphics Rendering using the Graphics Hardware Pi...
Mark Kilgard
 
Accelerating Vector Graphics Rendering using the Graphics Hardware Pipeline
Mark Kilgard
 
SIGGRAPH Asia 2012: GPU-accelerated Path Rendering
Mark Kilgard
 
GPU accelerated path rendering fastforward
Mark Kilgard
 
GPU-accelerated Path Rendering
Mark Kilgard
 
SIGGRAPH 2012: GPU-Accelerated 2D and Web Rendering
Mark Kilgard
 
SIGGRAPH 2012: NVIDIA OpenGL for 2012
Mark Kilgard
 
GTC 2012: GPU-Accelerated Path Rendering
Mark Kilgard
 
CS 354 Final Exam Review
Mark Kilgard
 
CS 354 Surfaces, Programmable Tessellation, and NPR Graphics
Mark Kilgard
 
CS 354 Performance Analysis
Mark Kilgard
 
CS 354 Global Illumination
Mark Kilgard
 
CS 354 Typography
Mark Kilgard
 
CS 354 Bezier Curves
Mark Kilgard
 

Recently uploaded (20)

PDF
Bridging biosciences and deep learning for revolutionary discoveries: a compr...
IAESIJAI
 
PDF
KodekX | Application Modernization Development
KodekX
 
PDF
Architecting across the Boundaries of two Complex Domains - Healthcare & Tech...
Peter Tan
 
PPT
“AI and Expert System Decision Support & Business Intelligence Systems”
ZubinRadhakrishnan
 
PPT
Teaching material agriculture food technology
LiaRayya
 
PDF
Encapsulation_ Review paper, used for researhc scholars
gurumoop
 
PDF
Machine learning based COVID-19 study performance prediction
IAESIJAI
 
PDF
Shreyas Phanse Resume: Experienced Backend Engineer | Java • Spring Boot • Ka...
SHREYAS PHANSE
 
PDF
The Rise and Fall of 3GPP – Time for a Sabbatical?
3G4G
 
PPTX
Understanding_Digital_Forensics_Presentation.pptx
ImranKhan423233
 
PPTX
VMware vSphere Foundation How to Sell Presentation-Ver1.4-2-14-2024.pptx
blackmambaettijean
 
PDF
Blue Purple Modern Animated Computer Science Presentation.pdf.pdf
Akar16
 
PPTX
A Presentation on Artificial Intelligence
memembruh336
 
PDF
Empathic Computing: Creating Shared Understanding
Mark Billinghurst
 
PDF
Approach and Philosophy of On baking technology
30anthuong17
 
PPTX
Detection-First SIEM: Rule Types, Dashboards, and Threat-Informed Strategy
ReZa AdineH
 
PDF
Unlocking AI with Model Context Protocol (MCP)
Brian McKeiver
 
PDF
Advanced methodologies resolving dimensionality complications for autism neur...
IAESIJAI
 
PDF
Dropbox Q2 2025 Financial Results & Investor Presentation
Dropbox
 
PDF
NewMind AI Monthly Chronicles - July 2025
NewMind AI
 
Bridging biosciences and deep learning for revolutionary discoveries: a compr...
IAESIJAI
 
KodekX | Application Modernization Development
KodekX
 
Architecting across the Boundaries of two Complex Domains - Healthcare & Tech...
Peter Tan
 
“AI and Expert System Decision Support & Business Intelligence Systems”
ZubinRadhakrishnan
 
Teaching material agriculture food technology
LiaRayya
 
Encapsulation_ Review paper, used for researhc scholars
gurumoop
 
Machine learning based COVID-19 study performance prediction
IAESIJAI
 
Shreyas Phanse Resume: Experienced Backend Engineer | Java • Spring Boot • Ka...
SHREYAS PHANSE
 
The Rise and Fall of 3GPP – Time for a Sabbatical?
3G4G
 
Understanding_Digital_Forensics_Presentation.pptx
ImranKhan423233
 
VMware vSphere Foundation How to Sell Presentation-Ver1.4-2-14-2024.pptx
blackmambaettijean
 
Blue Purple Modern Animated Computer Science Presentation.pdf.pdf
Akar16
 
A Presentation on Artificial Intelligence
memembruh336
 
Empathic Computing: Creating Shared Understanding
Mark Billinghurst
 
Approach and Philosophy of On baking technology
30anthuong17
 
Detection-First SIEM: Rule Types, Dashboards, and Threat-Informed Strategy
ReZa AdineH
 
Unlocking AI with Model Context Protocol (MCP)
Brian McKeiver
 
Advanced methodologies resolving dimensionality complications for autism neur...
IAESIJAI
 
Dropbox Q2 2025 Financial Results & Investor Presentation
Dropbox
 
NewMind AI Monthly Chronicles - July 2025
NewMind AI
 

NVIDIA OpenGL in 2016

  • 1. Mark Kilgard, July 24 SIGGRAPH 2016, Anaheim NVIDIA OpenGL in 2016
  • 2. 2 Mark Kilgard • Principal System Software Engineer OpenGL driver and API evolution Cg (“C for graphics”) shading language GPU-accelerated path rendering & web browser rendering • OpenGL Utility Toolkit (GLUT) implementer • Specified and implemented much of OpenGL • Author of OpenGL for the X Window System • Co-author of Cg Tutorial • Worked on OpenGL for 25 years My Background
  • 3. 3 NVIDIA’s OpenGL Leverage Debugging with Nsight Programmable Graphics Tegra Quadro OptiX GeForce Adobe Creative Cloud
  • 4. Jeff Kiel - Manager, Graphics Tools NSIGHT VSE AND OPENGL VR
  • 5. 5 AGENDA Intro to Nsight & Developer Tools VR debugging GPU Range Profiling Roadmap
  • 6. 6 C/C++ JetPack NVTX NVIDIA Tools eXtension Compile Debug Profile Trace Hardware Support IDE Integration Standalone and CLI Getting Started…
  • 7. 7 NSIGHT VISUAL STUDIO EDITION 5.2 •New Range Profiler, including OpenGL and DirectX12 •Vulkan Support •New Geometry View •Oculus VR SDK support, OpenGL and DX11 •CUDA 8.0 support •VR, Vulkan, and Advanced Graphics Profiling
  • 8. 8 UE4’S VR ENGINE •Render pass per eye View 0 Left View 1 Right Time Depth Pass Lighting Pass . . . View 0 Left View 1 Right
  • 10. 10 ROADMAP When you get back from SIGGRAPH: 5.2 RC1 September, 2016: 5.2 Final
  • 11. 11 ROADMAP Q4 2016: 5.3 The Future Tell Me What You Need!?!?
  • 12. 12 NVIDIA’s OpenGL Leverage Debugging with Nsight Programmable Graphics Tegra Quadro OptiX GeForce Adobe Creative Cloud
  • 13. 13 OpenGL Codebase Leverage Same driver code base supports multiple APIs OpenGL for Embedded, Mobile, and Web Multi-vendor, explicit, low-level graphics from Khronos
  • 14. 14 Still the One Truly Common & Open 3D API OS X Linux FreeBSD Solaris Android Windows
  • 15. 15 NVIDIA OpenGL in 2016 Provides OpenGL’s Maximally Available Superset Pascal Extensions 2015 ARB extensions OpenGL 4.5 Core Maxwell Extensions Legacy EXT & Other Compatibility Extensions OpenGL Complete Compatibility Path Rendering Multi-GPU. SLI Approaching Zero Driver Overhead NVIDIA Multi-generation GPU Initiatives DirectX inter-op Vulkan inter-op ES Enhancements Full OpenGL ES 3.2 Khronos Standard Expected Compatibility NVIDIA Initiatives GPU Generation Features
  • 16. 16 Background: NVIDA GPU Architecture Road Map Our interest NVIDIA GPU architectures of interest: Maxwell & Pascal What are Maxwell and Pascal mentioned on last slide?
  • 17. 17 OpenGL’s Recent Advancements 2014 2015 2016 New ARB Extensions 3 standard extensions, beyond 4.5 • ARB_sparse_buffer • ARB_pipeline_statistics_query • ARB_transform_feedback_overflow_query Maxwell Extensions • Novel graphics features • 14 new extensions • Global Illumination & Vector Graphics focus
  • 18. 18 OpenGL’s Recent Advancements 2014 2015 2016 New ARB Extensions 3 standard extensions, beyond 4.5 • ARB_sparse_buffer • ARB_pipeline_statistics_query • ARB_transform_feedback_overflow_query New ARB 2015 Extension Pack • Shader functionality • ARB_ES3_2_compatibility (shading language support) • ARB_parallel_shader_compile • ARB_gpu_shader_int64 • ARB_shader_atomic_counter_ops • ARB_shader_clock • ARB_shader_ballot • Graphics pipeline operation • ARB_fragment_shader_interlock • ARB_sample_locations • ARB_post_depth_coverage • ARB_ES3_2_compatibility (tessellation bounding box + multisample line width query) • ARB_shader_viewport_layer_array • Texture mapping functionality • ARB_texture_filter_minmax • ARB_sparse_texture2 • ARB_sparse_texture_clamp Maxwell Extensions • Novel graphics features • 14 new extensions • Global Illumination & Vector Graphics focus
  • 19. 19 OpenGL’s Recent Advancements 2014 2015 2016 New ARB Extensions 3 standard extensions, beyond 4.5 • ARB_sparse_buffer • ARB_pipeline_statistics_query • ARB_transform_feedback_overflow_query Maxwell Extensions • Novel graphics features • 14 new extensions • Global Illumination & Vector Graphics focus New ARB 2015 Extension Pack • Shader functionality • ARB_ES3_2_compatibility (shading language support) • ARB_parallel_shader_compile • ARB_gpu_shader_int64 • ARB_shader_atomic_counter_ops • ARB_shader_clock • ARB_shader_ballot • Graphics pipeline operation • ARB_fragment_shader_interlock • ARB_sample_locations • ARB_post_depth_coverage • ARB_ES3_2_compatibility (tessellation bounding box + multisample line width query) • ARB_shader_viewport_layer_array • Texture mapping functionality • ARB_texture_filter_minmax • ARB_sparse_texture2 • ARB_sparse_texture_clamp Pascal Extensions • Novel graphics features • 5 new extensions • Virtual Reality focus OpenGL SPIR-V Support • Standard Shader Intermediate Representation • ARB_gl_spirv • Vulkan interoperability
  • 20. 20 Maxwell OpenGL Extensions • Voxelization, Global Illumination, and Virtual Reality NV_viewport_array2 NV_viewport_swizzle AMD_vertex_shader_viewport_index AMD_vertex_shader_layer • Vector Graphics extensions NV_framebuffer_mixed_samples EXT_raster_multisample NV_path_rendering_shared_edge • Advanced Rasterization NV_conservative_raster NV_conservative_raster_dilate NV_sample_mask_override_coverage NV_sample_locations, now ARB_sample_locations NV_fill_rectangle • Shader Improvements NV_geometry_shader_passthrough NV_shader_atomic_fp16_vector NV_fragment_shader_interlock, now ARB_fragment_shader_interlock EXT_post_depth_coverage, now ARB_post_depth_coverage Requires GeForce 950, Quadro M series, Tegra X1, or better New Graphics Features of NVIDIA’s Maxwell GPU Architecture
  • 21. 21 Background: Viewport Arrays Several Maxwell (and Pascal) extensions build on Viewport Arrays Viewport arrays introduced to OpenGL standard by OpenGL 4.1 Feature of Direct3D 11 First introduced to OpenGL by NV_viewport_array extension Each viewport array element contains Viewport transform Scissor box and enable Depth range Provides N mappings of clip-space to scissored window-space Original conception Geometry shader could “steer” primitives into any of 16 viewport array elements Geometry shader would set the viewport index of a primitive Result: primitive is rasterized based on the indexed viewport array state 0 1 2 ... 15 xv yv wv hv n,f xs ys ws hs es 0 0 640 480 0,1 0,0,640,480,0 640 0 640 480 0,1 0,0,640,480,0 640 480 640 480 0,1 0,0,640,480,0 ... Viewport array state Indexed Array of Viewport & Scissor State
  • 22. 22 Viewport Arrays Visualized vertex shader geometry shader vertex shader vertex shader view frustum clipping viewport & depth range transform scissored rasterizer Viewport array state 0 1 2 ... 15 xv yv wv hv n,f xs ys ws hs es viewport index = 0 viewport index = 1 viewport index = 2 assembled triangle geometry shader primitive output stream (3 triangles) 0 0 640 480 0,1 0,0,640,480,0 640 0 640 480 0,1 0,0,640,480,0 640 480 640 480 0,1 0,0,640,480,0 ... resulting framebuffer
  • 23. 23 Viewport Index Generalized to Viewport Mask • Geometry shaders & viewport index approach proved limiting... • Common use of geometry shaders: view replication One stream of OpenGL commands  draws N views But inherently expensive for geometry shader to replicate N primitives Underlying issue: one thread of execution has to output N primitives • First fix Replace scalar viewport index per primitive with a viewport bitmask • Viewport mask does the primitive replication Viewport mask lets geometry shader output primitive to all, some, or none of viewport indices Examples 0xFFFF would replicate primitive 16 times, one primitive for each respective viewport index 0x0301 would output a primitive to viewport indices 9, 8, and 0 Maxwell’s NV_viewport_array2 extension Analogy: forcing too much water through a hose geometry shader
  • 24. 24 Geometry Shader Allowed to “Pass-through” of Vertex Attributes Geometry shaders are very general! 1 primitive input  N primitives output, where N is capped but still dynamic input vertex attributes can be arbitrarily recomputed Not conducive to executing efficiently Applications often just want 1 primitive in  constant N primitives out with NO change of vertex attributes though allowing for computing & output of per- primitive attributes NV_geometry_shader_passthrough supports a simpler geometry shader approach Hence more efficient Particularly useful when viewport mask allows primitive replication Restrictions 1 primitive in, 1 primitive out BUT writing the per-primitive viewport mask can force replication of 0 to 16 primitives, one for each viewport array index No modification of per-vertex attributes Allowances Still get to compute per-primitive outputs Examples: viewport mask and texture array layer Maxwell’s NV_geometry_shader_passthrough Extension
  • 25. 25 Analogy for Geometry Shader “Pass-through” of Vertex Attributes Geometry shader just computes per-primitive attributes and passes along primitive “Pass-through” of vertex attributes means geometry shader cannot modify them Full service geometry shader Efficient, low touch Slower, high touch Requires good behavior, many restrictions apply Fully general, anyone can use this line
  • 26. 26 Example Pass-through Geometry Shader layout(triangles) in; layout(triangle_strip) out; layout(max_vertices=3) out; in Inputs { vec2 texcoord; vec4 baseColor; } v_in[]; out Outputs { vec2 texcoord; vec4 baseColor; }; void main() { int layer = compute_layer(); // function not shown for (int i = 0; i < 3; i++) { gl_Position = gl_in[i].gl_Position; texcoord = v_in[i].texcoord; baseColor = v_in[i].baseColor; gl_Layer = layer; EmitVertex(); } } #extension GL_NV_geometry_shader_passthrough : require layout(triangles) in; // No output primitive layout qualifiers required. // Redeclare gl_PerVertex to pass through "gl_Position". layout(passthrough) in gl_PerVertex { vec4 gl_Position; }; // Declare "Inputs" with "passthrough" to copy members attributes layout(passthrough) in Inputs { vec2 texcoord; vec4 baseColor; }; // No output block declaration required void main() { // The shader simply computes and writes gl_Layer. We don't // loop over three vertices or call EmitVertex(). gl_Layer = compute_layer(); } Simple Example: Sends Single Triangle To Computed Layer BEFORE: Conventional geometry shader (slow) AFTER: Passthrough geometry shader (fast)
  • 27. 27 Outputting Layer Allows Layered Rendering • Example: Bind to particular level of 2D texture array with glFramebufferTexture Then gl_Layer output of geometry shader renders primitive to designated layer (slice) Allows Rendering to 3D Textures and Texture Arrays Texture array index for texturing, or gl_Layer for layered rendering 0 1 2 3 4 0 1 2 3 4 Mipmaplevelindex Example 2D texture array with 5 layers
  • 28. 28 Aside: Write Layer and Viewport Index from a Vertex Shader • Originally only geometry shaders could write the gl_ViewportIndex and gl_Layer outputs • Disadvantages Limited use of layered rendering and viewport arrays to geometry shader Often awkward to introduce a geometry shader for just to write these outputs GPU efficiency is reduced by needing to configure a geometry shader • AMD_vertex_shader_viewport_index allows gl_ViewportIndex to be written from a vertex shader • AMD_vertex_shader_layer allows gl_Layer to be written from a vertex shader • Good example where NVIDIA adopts vendor extensions for obvious API additions Generally makes OpenGL code more portable and life easier for developers in the process Maxwell’s AMD_vertex_shader_viewport_index & AMD_vertex_shader_layer Extensions
  • 29. 29 Further Extending Viewport Array State with Position Component Swizzling • Original viewport array state viewport transform depth range transform scissor box and enable • Maxwell extension adds new state four position component swizzle modes one for clip-space X, Y, Z, and W • Eight allowed modes GL_VIEWPORT_SWIZZLE_POSITIVE_X_NV GL_VIEWPORT_SWIZZLE_NEGATIVE_X_NV GL_VIEWPORT_SWIZZLE_POSITIVE_Y_NV GL_VIEWPORT_SWIZZLE_NEGATIVE_Y_NV GL_VIEWPORT_SWIZZLE_POSITIVE_Z_NV GL_VIEWPORT_SWIZZLE_NEGATIVE_Z_NV GL_VIEWPORT_SWIZZLE_POSITIVE_W_NV GL_VIEWPORT_SWIZZLE_NEGATIVE_W_NV Maxwell’s NV_viewport_swizzle extension Viewport array state 0 1 2 ... 15 xv yv wv hv n,f xs ys ws hs es xswyswzswwws 0 0 128 128 0,1 0,0,128,128,0 x+,y+,z+,w+ 0 0 128 128 0,1 0,0,128,128,0 y+,z+,x+,w+ 0 0 128 128 0,0 0,0,128,128,0 z+,x+,y+,w+ ... standard viewport array state NEW swizzle state
  • 30. 30 Reminder of Cube Map Structure • Cube map is essentially 6 images Six 2D images arranged like the faces of a cube +X, -X, +Y, -Y, +Z, -Z • Logically accessed by 3D (s,t,r) un- normalized vector Instead of 2D (s,t) Where on the cube images does the vector “poke through”? That’s the texture result • Interesting question Can OpenGL efficiently render a cube map in a single rendering pass? Cube Map Images are Position Swizzles Projected to 2D
  • 31. 31 Example of Cube Map Rendering
  • 32. 32 Example of Cube Map Rendering +X −X +Z −Z +Y−Y 0 123 4 5 Faces Labeled and Numbered by Viewport Index
  • 33. 33 Layer to Render Can Be Relative to Viewport Index • Geometry shader can “redeclare” the layer to be relative to the viewport index GLSL usage layout(viewport_relative) out highp int gl_Layer; • After viewport mask replication, primitive’s gl_Layer value is biased by its viewport index Allows each viewport index to render to its “own” layer • Good for single-pass cube map rendering usage Use passthrough geometry shader to write 0x3F (6 bits set, views 0 to 5) to the viewport mask Usage: gl_ViewportMask[0] = 0x3F; // Replicate primitive 6 times Set swizzle state of each viewport index to refer to proper +X, -X, +Z,-Y, +Z, -Z cube map faces Requires NV_viewport_swizzle extension Caveat: Force the window-space Z to be an eye-space planar distance for proper depth testing Requires inverse W buffering for depth testing Swizzle each view’s “Z” into output W Make sure input clip-space W is 1.0 and swizzled to output Z Means window-space Z will be one over W or a planar eye-space distance from eye, appropriate for depth testing Requires to have floating-point depth buffer for W buffering Bonus Feature of Maxwell’s NV_viewport_array2 extension
  • 34. 34 (Naïve) Fast Single-pass Cube Map Rendering #define pX GL_VIEWPORT_SWIZZLE_POSITIVE_X_NV #define nX GL_VIEWPORT_SWIZZLE_NEGATIVE_X_NV #define pY GL_VIEWPORT_SWIZZLE_POSITIVE_Y_NV #define nY GL_VIEWPORT_SWIZZLE_NEGATIVE_Y_NV #define pZ GL_VIEWPORT_SWIZZLE_POSITIVE_Z_NV #define nZ GL_VIEWPORT_SWIZZLE_NEGATIVE_Z_NV #define pW GL_VIEWPORT_SWIZZLE_POSITIVE_W_NV glDisable(GL_SCISSOR_TEST); glViewport(0, 0, 1024, 1024); glViewportSwizzleNV(0, nZ, nY, pW, pX); // positive X face glViewportSwizzleNV(1, pZ, nY, pW, nX); // negative X face glViewportSwizzleNV(2, pX, pZ, pW, pY); // positive Y face glViewportSwizzleNV(3, pX, nZ, pW, nX); // negative Y face glViewportSwizzleNV(4, pX, nY, pW, pZ); // positive Z face glViewportSwizzleNV(5, nX, nY, pW, nZ); // negative Z face #extension GL_NV_geometry_shader_passthrough : require #extension GL_NV_viewport_array2 : require layout(triangles) in; // No output primitive layout qualifiers required. layout(viewport_relative) out highp int gl_Layer; // Redeclare gl_PerVertex to pass through "gl_Position". layout(passthrough) in gl_PerVertex { vec4 gl_Position; }; // Declare "Inputs" with "passthrough" to copy members attributes layout(passthrough) in Inputs { vec2 texcoord; vec4 baseColor; }; void main() { gl_ViewportMask[0] = 0x3F; // Replicate primitive 6 times gl_Layer = 0; } With Maxwell’s NV_viewport_array2 & NV_viewport_swizzle Viewport array state configuration Passthrough geometry shader non-naïve version would perform per-face culling in shader Getting swizzles from this table from the OpenGL 4.5 specification ensures your swizzles matches OpenGL’s cube map layout conventions
  • 35. 35 GPU Voxelization, typically for Global Illumination • Concept: desire to sample the volumetric coverage within a scene Ideally sampling the emittance color & directionality from the scene too Input: polygonal meshes Output: 3D grid (texture image) where voxels hold attribute values + coverage The Other Main Justification for Viewport Swizzle Voxelization pipeline Passthrough geometry shader + viewport swizzle makes this fast
  • 36. 36 What’s Tricky About Voxelization • Not your regular rasterization into a 2D image! • Instead voxelization needs rasterizing into a 3D grid Represented on the GPU as a 3D texture or other 3D array of voxels • BUT our GPU and OpenGL only know how to rasterize in 2D So exploit that by rasterizing into a “fake” 2D framebuffer ARB_framebuffer_no_attachments extension allows rasterizing to framebuffer lacking any attachments for color or depth-stencil The logical framebuffer has a width & height, but no pixel storage • Approach: Rasterize a given triangle within the voxelization region on an orthogonal axis direction where triangle has the largest area (X, Y, or Z axis) Then fragment shader does (atomic) image stores to store coverage & attributes at the appropriate (x,y,z) location in 3D grid Caveat: Use conservative rasterization to avoid missing features Skip rendering a 2D image with pixels... because we need a 3D result Exact details are involved, but a fast geometry shader & viewport swizzling make Dominant Axis Selection efficient
  • 37. 37 What’s the Point of Voxelization? Direct lighting feels over dark Feeds a GPU Global Illumination Algorithm
  • 38. 38 What’s the Point of Voxelization? Feeds a GPU Global Illumination Algorithm Global illumination with ambient occlusion avoids the over-dark feel
  • 39. 39 Direct lighting feels over dark What’s the Point of Voxelization? Feeds a GPU Global Illumination Algorithm
  • 40. 40 Global Illumination with specular effects capture subtle reflections in floor too What’s the Point of Voxelization? Feeds a GPU Global Illumination Algorithm
  • 41. 41 What’s the Point of Voxelization? Improving the Ambient Contribution on Surfaces Flat ambient (no diffuse or specular directional lighting shown)
  • 42. 42 What’s the Point of Voxelization? Improving the Ambient Contribution on Surfaces Screen-space ambient occlusion improves the sense of depth a little
  • 43. 43 What’s the Point of Voxelization? Improving the Ambient Contribution on Surfaces True global illumination for ambient makes the volumetric structure obvious
  • 49. 49 Complete Global Illumination is Complex • Complete implementation included in NVIDIA VXGI Implements Voxel Cone Tracing Part of Visual FX solutions • Implemented for DirectX 11 But all the underlying GPU technology is available as OpenGL extensions NV_viewport_array2 NV_viewport_swizzle NV_geometry_shader_passthrough NV_conservative_raster NVIDIA Provides Implementations
  • 50. 50 Conservative Rasterization • Mentioned on last slide as an extension used for global illumination Easy to enable: glEnable(GL_CONSERVATIVE_RASTERIZATION_NV); Additional functionality: Also provides ability to provide addition bits of sub-pixel precision • Conventional rasterization is based on point-sampling Pixel is covered if the pixel’s exact center is within the triangle Multisample antialiasing = multiple pixel locations per pixels Means rasterization can “miss” coverage if sample points for pixels or multisample locations are missed Point sampling can under-estimate ideal coverage • Conservative rasterization Guarantees coverage if any portion of triangle intersects (overlaps) the pixel square Caveat: after sub-pixel snapping to the sub-pixel grid However may rasterize “extra” pixels not overlapping pixel squares intersected by the triangle Conservative rasterization typically over-estimates ideal coverage Intended for algorithms such as GPU voxelization where missing coverage results in rendering artifacts—and be tolerant of over-estimated coverage Maxwell’s NV_conservative_raster extension
  • 51. 51 Conservative Rasterization Visualized • Green pixel squares have their pixel center covered by the triangle • Pink pixel squares intersect the triangle but do NOT have their pixel centered covered Consider Conventional Rasterization of a Triangle Pink pixel square indicate some degree of under-estimated coverage
  • 52. 52 Conservative Rasterization Visualized • Push triangle edges away from the triangle center (centroid) by half-pixel width • Constructs a new, larger (dilated) triangle covering more samples Consider Conventional Rasterization of a Dilated Triangle Notice all the pink pixel squares are within the dilated triangle
  • 53. 53 Conservative Rasterization Visualized • Yellow pixel square indicate pixels within dilated triangle but not intersected by the original triangle Overestimated Rasterization of a Dilated Triangle Notice all the yellow pixel squares are within the dilated triangle
  • 54. 54 Caveats Using Conservative Rasterization • Shared edges of non-overlapping rasterized triangles are guaranteed not to have either Double-hit pixels Pixel gaps • Rule is known as “watertight rasterization” Very useful property in practice Example: avoids double blending at edges Coverage can be under-estimated; long, skinny triangles might cover zero samples • Interpolation at a covered pixel center (or sample locations when multisampling) are guaranteed to return values within bounds of primitives vertex attributes • Conservative rasterization makes no such guarantee against double-hit pixels • Indeed double-hit pixels are effective guaranteed along shared triangle edges • Algorithms using conservative rasterization must be tolerant of over-estimated coverage Long, skinny triangles have more dilation over-estimated coverage error • Interpolation can become extrapolation when interpolation location is not within the original primitive! You have been warned shared edge
  • 55. 55 Conservative Rasterization Dilate Control Provides control to increase the amount of conservative dilation when GL_CONSERVATIVE_RASTERIZATION_NV is enabled Straightforward usage glConservativeRasterParameterfNV (GL_CONSERVATIVE_RASTER_DILATE_NV, 0.5f); 0.5 implies an additional half-pixel offset to the dilation, so extra conservative Actual value range is [0, 0.75] in increments of 0.25 Initial value is 0.0 Maxwell’s NV_conservative_raster_dilate extension
  • 56. 56 Conservative Rasterization versus Polygon Smooth • OpenGL supports polygon smooth rasterization mode since OpenGL 1.0 Example usage: glEnable(GL_POLYGON_SMOOTH) • glEnable(GL_CONSERVATIVE_RASTERIZATION_NV) is different from glEnable(GL_POLYGON_SMOOTH)? Subtle semantic difference • NVIDIA implements GL_POLYGON_SMOOTH by computing point-inside-primitive tests at multiple sample locations within each pixel square So computes fractional coverage used to modulate alpha component post-shading Typically recommended for use with glBlendFunc(GL_SRC_ALPHA_SATURATE, GL_ONE) blending enabled Polygon smooth should not over-estimate fractional coverage • Conservative rasterization works by dilation, as explained Conservative rasterization does not compute a fractional coverage So there is no modulation of alpha by the fractional coverage What’s the difference?
  • 57. 57 Maxwell Vector Graphics Improvements • Simple idea: mixed sample counts Improve antialiasing quality & performance of vector graphics rendering Every color samples gets N stencil/depth samples • Notion of stencil-depth test changes OLD notion: stencil & depth tests must either fail or pass, Boolean result NEW notion: multiple stencil & depth values per color sample mean the stencil & depth test can “fractionally pass” • GPU automatically modulates post-shader RGBA color by fractional test result Assumes blending configured Similar to fractional coverage blending in CPU-based vector graphics • Advantages Works very cleanly with NV_path_rendering Much reduced memory footprint ¼ at same coverage quality Much less memory bandwidth Superior path rendering anti-aliasing quality, up to 16x Minimal CPU overhead Maxwell provides super- efficient “cover” operation Maxwell’s NV_framebuffer_mixed_samples Extension glCoverageModulationNV(GL_RGBA);
  • 58. 58 16:1 Fractional Stencil Test Example 87.5% fractional stencil test (14 of 16) 1 color sample, 16 stencil samples 100% fractional stencil test (16 of 16) 0% fractional stencil test (0 of 16) 37.5% fractional stencil test (6 of 16) Examine Fractional Stencil Test Results
  • 59. 59 4 color samples, 16 stencil samples Each color sample separately modulated and blended! 0%, 100%, 0%, 50% fractional stencil test (1 of 4, 4 of 4, 0 of 4, 1 of 4) 0%, 0%, 0%, 0% fractional stencil test (0 of 4, 0 of 4, 0 of 4, 0 of 4) 100%, 100%, 100%, 100% fractional stencil test (4 of 4, 4 of 4, 4 of 4, 4 of 4) 100%, 100%, 100%, 50% fractional stencil test (4 of 4, 4 of 4, 4 of 4, 2 of 4) 16:4 Fractional Stencil Test Example Examine Fractional Stencil Test Results
  • 60. 60 Mixed Sample Configurations 8:8 8:4 8:2 8:1 8x1x 2x 4x 16x 1x 1:1 2:1 4:1 16:1 2x 2:2 4:2 16:2 4x 4:4 16:4 8x 16:8 Coverage/stencil samples per pixel Colorsamplesperpixel Maxwell’s NV_framebuffer_mixed_samples Extension
  • 61. 61 N = 1 2 4 8 16 M = 1 2 4 8 = pixel region LEGEND = color sample = sample location Mixed Samples Visualized Application determines the quality/performance/memory; many choices
  • 62. 62 Better Vector Graphics Performance Tiger SVG Scene GK204 (Kepler) vs. GM204 (Maxwell2) vs. GM204 with NV_framebuffer_mixed_samples 0.00 0.50 1.00 1.50 2.00 2.50 3.00 100x100 200x200 300x300 400x400 500x500 500x500 600x600 700x700 800x800 900x900 1000x1000 1100x1100 Window Resolution Millisecondsperframe GK104 16:16 GM20416:16 GM204 16:4 GM204 16:1 Kepler conventional 16x Maxwell 2 conventional 16x Maxwell 2, 16:4 & 16:1 Faster & ¼ memory footprint Smaller is better (faster!) While Using Much Less Framebuffer Memory
  • 63. 63 Fast, Flexible Vector Graphics Results NV_framebuffer_mixed_samples + NV_path_rendering combined Web pages Flash type games Text, even in with perspective Emojis!  Illustrations Mapping All rendering shown at 16:1 quality
  • 64. 64 NVIDIA OpenGL Features Integrated in Google’s Skia 2D Graphics Library • Skia is Google’s 2D graphics library • Primarily for web rendering • Used by Chromium, Firefox, and Google’s Chrome browser • Skia has support today for GPU-acceleration with OpenGL exploiting • NV_path_rendering for vector graphics filling & stroking • NV_framebuffer_mixed_samples for efficient framebuffer representation • EXT_blend_func_extended for extended Porter-Duff blending model • KHR_blend_equation_advanced for advanced Blend Modes
  • 65. 65 Naïve Mixed Sample Rendering Causes Artifacts • Easy to render paths with NV_path_rendering + NV_framebuffer_mixed_samples • Reason: two-step “Stencil, then Cover” approach guarantees proper coverage is fully resolved in first “stencil” pass, then color is updated in “cover” pass • Just works by design • But what if you want to render a simple convex shape like a rectangle with conventional rasterization & mixed samples? • Draw rectangle as two triangles • Into 16:1 mixed sample configuration • But fractional coverage modulation causes seam along internal edge! Requires Careful use of NV_framebuffer_mixed_samples 4x pixel magnification double blending crack   great 16x antialiasing on external edges
  • 66. 66 Examine the Situation Carefully • Two triangles A and B • Where A is 100% fine • Where B is 100% fine • External edge of A is properly antialiased • External edge of B is properly antialiased • PROBLEM is shared edge • Both triangles claim fractional coverage along this edge • Causes Double Blending • Can we “fix” rasterization so either A or B, but never both claim the shared edge? • YES, Maxwell GPUs can • Using NV_sample_mask_override_coverage extension Maxwell’s NV_sample_mask_override_coverage Extension Helps 100% A 100% B A’s antialiased edge B’s antialiased edge Problematic double-blended shared edge
  • 67. 67 Solution: Triangle A Claims Coverage or B Claims, But not Both void main() { gl_FragColor = gl_Color; } #version 400 compatibility #extension GL_NV_sample_mask_override_coverage : require layout(override_coverage) out int gl_SampleMask[]; const int num_samples = 16; const int all_sample_mask = 0xffff; void main() { gl_FragColor = gl_Color; if (gl_SampleMaskIn[0] == all_sample_mask) { gl_SampleMask[0] = all_sample_mask; } else { int mask = 0; for (int i=0; i<num_samples; i++) { vec2 st; st = interpolateAtSample(gl_TexCoord[0].xy, i); if (all(lessThan(abs(st),vec2(1)))) mask |= (1 << i); } int otherMask = mask & ~gl_SampleMaskIn[0]; if (otherMask > gl_SampleMaskIn[0]) gl_SampleMask[0] = 0; else gl_SampleMask[0] = mask; } } Handle in fragment shader: by overriding the sample mask coverage BEFORE: Simply output interpolated color AFTER: Interpolate color + resolve overlapping coverage claims trivial fragment shader
  • 68. 68 Solution: Triangle A Claims Coverage or B Claims, But not Both void main() { gl_FragColor = gl_Color; } #version 400 compatibility #extension GL_NV_sample_mask_override_coverage : require layout(override_coverage) out int gl_SampleMask[]; const int num_samples = 16; const int all_sample_mask = 0xffff; void main() { gl_FragColor = gl_Color; if (gl_SampleMaskIn[0] == all_sample_mask) { gl_SampleMask[0] = all_sample_mask; } else { int mask = 0; for (int i=0; i<num_samples; i++) { vec2 st; st = interpolateAtSample(gl_TexCoord[0].xy, i); if (all(lessThan(abs(st),vec2(1)))) mask |= (1 << i); } int otherMask = mask & ~gl_SampleMaskIn[0]; if (otherMask > gl_SampleMaskIn[0]) gl_SampleMask[0] = 0; else gl_SampleMask[0] = mask; } } Handle in fragment shader: by overriding the sample mask coverage BEFORE: Simply output interpolated color AFTER: Interpolate color + resolve overlapping coverage claims additional re-rasterization epilogue early accept optimization sample mask override coverage support
  • 69. 69 NV_sample_mask_override_coverage • BEFORE: Fragment shaders can access sample mask for multisample rasterization • Indicates which individual coverage samples with a pixel are covered by the fragment • Fragment shader can also “clear” bits in the sample mask to discard samples • But in standard OpenGL, no way to “set” bits to augment coverage • Fragment’s output sample mask is always bitwise AND’ed with original sample mask • NOW: Maxwell’s NV_sample_mask_override_coverage allows overriding coverage! • The fragment shader can completely rewrite the sample mask • Clearing bits still discards coverage • BUT setting bits not previously set augments coverage • Powerful capability enables programmable rasterization algorithms • Like example in previous slide to fix double blending artifacts What does it allow?
  • 70. 70 Other Sample Mask Coverage Override Uses • Handles per-sample stencil test for high-quality sub-pixel clipping • These techniques integrated today into Skia Works for general quadrilaterals, even in drawn in perspective Adapts well to drawing circles and ellipses And even rounded rectangles Example: 16x quality blended ellipses
  • 71. 71 Maxwell OpenGL Extensions • Voxelization, Global Illumination, and Virtual Reality NV_viewport_array2 NV_viewport_swizzle AMD_vertex_shader_viewport_index AMD_vertex_shader_layer • Vector Graphics extensions NV_framebuffer_mixed_samples EXT_raster_multisample NV_path_rendering_shared_edge • Advanced Rasterization NV_conservative_raster NV_conservative_raster_dilate NV_sample_mask_override_coverage NV_sample_locations, now ARB_sample_locations NV_fill_rectangle • Shader Improvements NV_geometry_shader_passthrough NV_shader_atomic_fp16_vector NV_fragment_shader_interlock, now ARB_fragment_shader_interlock EXT_post_depth_coverage, now ARB_post_depth_coverage Requires GeForce 950, Quadro M series, Tegra X1, or better New Graphics Features of NVIDIA’s Maxwell GPU Architecture Lacked time to talk about these extensions
  • 72. 72 • Graphics pipeline operation •ARB_fragment_shader_interlock •ARB_sample_locations •ARB_post_depth_coverage •ARB_ES3_2_compatibility •Tessellation bounding box •Multisample line width query •ARB_shader_viewport_layer_array • Texture mapping functionality •ARB_texture_filter_minmax •ARB_sparse_texture2 •ARB_sparse_texture_clamp • Shader functionality •ARB_ES3_2_compatibility •ES 3.2 shading language support •ARB_parallel_shader_compile •ARB_gpu_shader_int64 •ARB_shader_atomic_counter_ops •ARB_shader_clock •ARB_shader_ballot 2015: In Review OpenGL in 2015 ratified 13 new standard extensions
  • 73. 73 Need a Full Refresher on 2014 and 2015 OpenGL? • Honestly, lots of functionality in 2014 & 2015 if you’ve not followed carefully Available @ http://www.slideshare.net/Mark_Kilgard
  • 74. 74 Pascal GPU OpenGL Extensions • Pascal has 5 new OpenGL extensions • Major goal: improving Virtual Reality support • Several extensions used in combination • NV_stereo_view_rendering • efficiently render left & right eye views in single rendering pass • NV_viewport_array2 + NV_geometry_shader_passthrough—discussed already • NV_clip_space_w_scaling • extends viewport array state with per-viewport re-projection • EXT_window_rectangles • fast inclusive/exclusive rectangle testing during rasterization • Multi-vendor extension supported on all modern NVIDIA GPUs • High-end Virtual Reality with two GPUs • New explicit NV_gpu_multicast extension • Render left & right eyes with distinct GPUs New for 2016
  • 75. 75 Basic question Why should the Virtual Reality (VR) image shown in a Head Mounted Display (HMD) feel real? Ignoring head tracking and the realism of the image itself... just focused on the image generation
  • 76. 76 Why HMD’s Image ≈ Perception of Reality ≈ lens(lens-1(rendered image)) ≈ lens(screen) HMD image ≈ lens image screen ≈ lens-1(rendered image) ≈ rendered image rendered image ≈ pin hole image pin hole image ≈ eye view eye view ≈ perception of reality by warping by rendering model by anatomy by psychology by optics lens image = lens(screen) ≈ pin hole image ≈ eye view ≈ perception of reality image ≈ lens(lens-1(image)) by composition Portion of transformation involving GPU rendering & resampling Twin goals 1. Minimize HMD resampling error 2. Increase rendering efficiency
  • 77. 77 Goal of Head Mounted Display (HMD) Rendering •Goal: perceived HMD image ≈ visual perception of reality • Each image pair on HMD screen, as seen through its HMD lens, should be perceived as images of the real world •Assume pin hole camera image ≈ real world • Traditional computer graphics assumes this • Perspective 3D rasterization idealizes a pin hole camera • Human eye ball also approximately a pin hole camera •perceived HMD image = lens(screen image) • Function lens() warps image as optics of HMD lens does •screen image = lens-1 (pin hole camera image) • Function lens-1 () is inverse of the lens image warp •perceived image ≈ lens(lens-1 (pin hole camera image)) •pin hole camera image ≈ eye view
  • 78. 78 Pin Hole Camera Ideal Albrecht Dürer: Artist Drawing with Perspective Device Normal computer graphics generally good at rendering “pin hole” camera images And people are good at interpreting such images as 3D scenes But HMDs have a non-linear image warping due to lens distortion
  • 79. 79 Lens Distortion in HMD •Head-mounted Display (HMD) magnifies its screen with a lens •Why is a lens needed? • To feel immersive • Immersion necessitates a wide field- of-view • So HMD lens “widens” the HMD screen’s otherwise far too narrow field-of-view •Assume a radial symmetric magnify • Could be a fancier lens & optics • BUT consumer lens should be inexpensive & lightweight Graph paper viewed & magnified through HMD lens
  • 81. 81 Lens Performs a Radial Symmetric Warp Adding circles to image shows distortion increases as the radius increases Original Image Overlaid with circles
  • 82. 82 Pin-hole Camera Image Assumptions •Assume a conventionally rendered perspective image • In other words a pin-hole camera image •r is the distance of a pixel (x,y) relative to the center of the image at (0,0) so •Theta is the angle of the pixel relative to the origin •Assume pin hole camera image has maximum radius of 1 • So the X & Y extent of the images is [-1..1] 22 yxr += θ θ sin cos ry rx = =
  • 83. 83 Radius Remapping for an HMD Magnifying Lens • A lens in an HMD magnifies the image • What is magnification really? • Magnifying takes a pixel at a given radius and “moves it out” to a larger radius in the magnified image • In the HMD len’s image, each pin-hole camera pixel radius r is mapped to alternate radius rlensImage • This maps each pixel (x,y) in the pin-hole camera image to an alternate location (xlensImage,ylensImage) • Without changing theta agedisplaylensImage rrkrkr Im 4 2 2 1 ...)1( +++= ...1 4 2 2 1 +++ = rkrk r r lensImage gedisplayIma Essentially a Taylor series approximating actual optics of lens
  • 84. 84 Lens Function Coefficients for Google Cardboard Lens coefficients k1 & k2 are values that can be measured Additional coefficients (k3, etc.) are negligible Coefficients for typical lens in Google Cardboard k1 = 0.22 k2 = 0.26 Big question Can we render so the amount of resampling necessary to invert a particular lens’s distortion is minimized?
  • 85. 85 Radius Remapping for Lens Matched Shading (LMS) • Assume a conventionally rendered perspective image • In other words a pin-hole camera image • r is the distance of a pixel (x,y) relative to the center of the image at (0,0) so • Theta is the angle of the pixel relative to the origin • Lens Matched Shading provides an alternate radius rLMS for the same pixel (xLMS,yLMS) • This maps each pixel (x,y) to an alternate location • Without changing theta θ θ sin cos LMSLMS LMSLMS ry rx = = θ θ sin cos ry rx = = θθ sincos1 rprp r rLMS ++ = 22 yxr += OLD: Conventional “pin hold” camera rendering NEW: Lens Matched Shading rendering
  • 86. 86 Concentric circles in pin hole camera view gets “squished” by inverse lens transform HMD’s Inverse Lens Warp pin hole camera view (conventionally rendered image) inverse lens warp view (HMD screen) k1 = 0.22 k2 = 0.26 4 2 2 1 21 rkrk r r lensImage gedisplayIma ++ =
  • 87. 87 Lens Matched Shading p = 0.26007 θθ sincos1 rprp r rLMS ++ = pin hole camera view Lens Matched Shading (rendered framebuffer image) Concentric circles in pin hole camera view gets “projected” towards origin
  • 88. 88 Complete Process of Lens Matched Shading ideal pin hole camera view rendered image with lens matched shading lens warped image image as perceived viewed through HMD lens while different, these two images are “well matched” so warp between them minimizes pixel movement and resampling
  • 89. 89 What is Optimal Value for p? A reasonable measure of optimality is root mean square error of difference between LMS and inverse lens warp radii over entire lens So what p minimizes this integral for a particular lens’s coefficients When k1 = 0.22 & k2 = 0.26, optimal p ≈ 0.26007 θ θθ π ddrr rprp r rkrk r 2 2 0 1 0 4 2 2 1 sincos121∫∫         ++ − ++ * Analysis assumes a Google Cardboard-type device; Oculus has asymmetric visible screen region
  • 90. 90 Matched Overlap of Lens Matched Shading and Lens Warped Image k1 = 0.22 k2 = 0.26 p = 0.26007 Root Mean Square (RMS) error = 0.0598
  • 91. 91 Much Worse Overlap of Conventional Projection and Lens Warped Image Root Mean Square (RMS) error = 0.273 k1 = 0.22 k2 = 0.26 p = 0
  • 92. 92 Advantages of Lens Matched Shading •What is rendered by GPU is closer (less error) to what the HMD needs to display than conventional “pin hole” camera rendering •Means less resampling error • There’s still a non-linear re-warping necessary • However the “pixel movement” for the warp is greatly reduced •Another advantage: fewer pixels need be rendered for same wide field of view •Also want application to render left & right views with LMS in a single efficient rendering pass
  • 94. 94 Stereo Views of Same Scene Left and Right eye view of same simple scene Two scenes are slightly different if compared
  • 95. 95 Swapped Stereo Views Right and Left (swapped) eye view of same simple scene Two scenes are slightly different if compared
  • 96. 96 Image Difference of Two Views − + 0.5 = Left eye view Right eye view Clamped difference image
  • 97. 97 Lens Matched Shading Same left & right eye view but rendered with w scaling
  • 98. 98 Lens Matched Shading Quadrants Same left & right eye view but rendered with w scaling Each quadrant gets different projection to “tilt to center”
  • 99. 99 Visualization of Lens Matched Shading Rendering
  • 100. 100 Warped Lens Matched Shaped Warped version of lens shading to match HMD lens
  • 101. 101 Lens Matched Shading with Window Rectangle Testing Same Lens Matched Shading but with EXT_window_rectangles Nothing in black corners is shaded or even rasterized
  • 102. 102 Lens Matched Shading with Window Rectangle Testing Nothing in black corners is shaded or even rasterized Yellow lines show overlaid 8 inclusive window rectangles Same 8 window rectangles “shared” by each view’s texture array layer
  • 104. 104 NEW Window Rectangles Test in Per-fragment Operations Window Rectangles Test NEW stage
  • 105. 105 Straightforward API • glWindowRectanglesEXT(GLenum mode, GLsizei count, const GLint rects[]); • mode can be either GL_INCLUSIVE_EXT or GL_EXCLUSIVE_EXT • count can be from 0 to maximum number of supported window rectangles • Must be at least 4 (for AMD hardware) • NVIDIA hardware supports 8 • Rectangles allowed to overlap and/or disjoint • Each rectangle is (x,y,width,height) • width & height must be non-negative • Initial state • GL_EXCLUSIVE_NV with zero rectangles • Excluding rendering from zero rectangles means nothing is discarded by window rectangles test Multi-vendor EXT_window_rectangles Extension
  • 106. 106 Lens Matched Shading with Window Rectangle Testing Nothing in black corners is shaded or even rasterized Yellow lines show overlaid 8 inclusive window rectangles Same 8 window rectangles “shared” by each view’s texture array layer
  • 107. 107 Warped Lens Matched Shading with Window Rectangle Testing during Rendering Identical as “Lens Matched Shading” despite corners not being rasterized because corners don’t contribute to warped version
  • 108. 108 Warped Lens Matched Shading with Win. Rect. Testing during Rendering & Warping Same prior image, but warp now uses window rectangles Avoids wasting time warping corners not visible through lens
  • 109. 109 Visualizing Warp Window Rectangles Point: Window rectangle testing used TWICE #1 during Lens Matched Shading rendering pass #2 during warping pass
  • 110. 110 VR Rendering Pipeline LMS Right Eye View Warped Right Eye View LMS Left Eye View Warped Left Eye View Scene Displayed within HMD Single Rendering Pass Single Pass Stereo + Lens Matched Shading + Window Rectangle Testing Drawn with Single Triangle Fragment Shader Warping Window Rectangle Testing Perception to user is linear rendering HMD lens “undoes” warping to provide a perceived wide field-of-view Pascal does all this efficiently in a single rendering pass! 8 viewports, 1 pass
  • 111. 111 OpenGL Extensions Used in LMS VR Pipeline • Allows vertex shader to output two clip-space positions • (x1,y,z,w) and (x2,y,z,w) • Results in TWO primitives one for left eye & one for right eye • New GLSL built-ins • gl_SecondaryPositionNV • Like gl_Position but for “second eye’s view” • gl_SecondaryViewportMaskNV[] • Like gl_ViewportMaskNV[] but for “second eye’s view” • Also can steer primitives to different texture array slices • layout(secondary_view_offset = 1) int gl_Layer; Pascal’s NV_stereo_view_rendering Extension
  • 112. 112 OpenGL Extensions Used in LMS VR Pipeline Adds a new set of state to viewport array elements Each viewport index can recompute clip space as w = w + A x + B y Pascal’s NV_clip_space_w_scaling Extension Viewport array state 0 1 2 3 ... 15 xv yv wv hv n,f xs ys ws hs es xswyswzswwws A,B 0 0 1024 1024 0,1 0,0, 512,512,1 x+,y+,z+,w+ −0.26,−0.26 0 0 1024 1024 0,1 512,0, 512,512,1 y+,z+,x+,w+ +0.26,−02.6 0 0 1024 1024 0,1 512,0, 512,512,1 z+,x+,y+,w+ −0.26,−0.26 ... standard viewport array state swizzle state NEW w scaling 0 0 1024 1024 0,1 512,512, 512,512,1 z+,x+,y+,w+ +0.26,+0.26 Four quadrants for Lens Matched Shading
  • 113. 113 Example Lens Matched Shading Rendered Image Example image A=+0.2, B=+0.2A=−0.2, B=+0.2 A=−0.2, B=−0.2 A=+0.2, B=−0.2
  • 114. 114 More Information on NVIDIA Virtual Reality GPU Support Growing Software Development Kit (SDK) for Virtual Reality Focus on GPU efficiency Whitepapers and sample code Both OpenGL and Direct3D supported https://developer.nvidia.com/vrworks Get the VRWORKS 2.0 SDK
  • 115. 115 Still More Pascal OpenGL Extensions NVX_blend_equation_advanced_multi_draw_buffers • No API, simply relaxes error restriction so advanced blend modes from KHR_blend_equation_advanced & NV_blend_equation_advanced work with more than 1 color attachment • Important for CMYK rendering NV_conservative_raster_pre_snap_triangles • More Conservative Rasterization control • Allows conservative rendering dilation prior to sub-pixel snapping NV_shader_atomic_float64 • Atomic shader operations on double-precision values CYMK color space rendering with multiple color attachments Pascal’s non-Virtual Reality Enhancements
  • 116. 116 OpenGL extension exposing Khronos intermediate language for parallel compute and graphics New standard Khronos extension for OpenGL Just announced! July 22, 2016 Allows compiled SPIR-V code to be passed directly to OpenGL driver Accepts SPIR-V output from open source Glslang Khronos Reference compiler https://github.com/KhronosGroup/glslang Other compilers can target SPIR-V too Khronos standard extension ARB_gl_spirv +
  • 117. 117 SPIR-V Ecosystem LLVM Third party kernel and shader Languages •SPIR-V •Khronos defined and controlled cross-API intermediate language •Native support for graphics and parallel constructs •32-bit Word Stream •Extensible and easily parsed •Retains data object and control flow information for effective code generation and translation OpenCL C++OpenCL C GLSL Khronos has open sourced these tools and translators IHV Driver Runtimes Other Intermediate Forms SPIR-V Validator SPIR-V (Dis)Assembler LLVM to SPIR-V Bi-directional Translator Khronos plans to open source these tools soon https://github.com/KhronosGroup/SPIR/tree/spirv-1.1 Open source C++ front-end released HLSL Khronos has open sourced these tools and translators Khronos plans to open source these tools soon Khronos has open sourced these tools and translators HLSL Khronos plans to open source these tools soon Khronos has open sourced these tools and translators GLSLHLSL Khronos plans to open source these tools soon Khronos has open sourced these tools and translators OpenCL C GLSLHLSL Khronos plans to open source these tools soon Khronos has open sourced these tools and translators OpenCL C++OpenCL C GLSLHLSL Khronos plans to open source these tools soon Khronos has open sourced these tools and translators LLVM to SPIR-V Bi-directional Translator OpenCL C++OpenCL C GLSLHLSL Khronos plans to open source these tools soon Khronos has open sourced these tools and translators SPIR-V Validator LLVM to SPIR-V Bi-directional Translator OpenCL C++OpenCL C GLSLHLSL Khronos plans to open source these tools soon Khronos has open sourced these tools and translators SPIR-V (Dis)Assembler SPIR-V Validator LLVM to SPIR-V Bi-directional Translator OpenCL C++OpenCL C GLSLHLSL Khronos plans to open source these tools soon Khronos has open sourced these tools and translators New with ARB_gl_spirv
  • 118. 118 NVIDIA’s SIGGRAPH Driver Update •NVIDIA historically releases a “developer” driver at SIGGRAPH with support for all Khronos standard extensions announced at SIGGRAPH • This year too  •Monday (July 25, 2016) NVIDIA will put out a new SIGGRAPH driver • ARB_gl_spirv • Major extension in terms of compiler infrastructure & shader support • EXT_window_rectangles • Updates to Pascal OpenGL extensions • For Windows and Linux operating systems Developed driver with ARB_gl_spirv extension https://developer.nvidia.com/opengl-driver
  • 119. 119 GLEW Support Available NOW GLEW = The OpenGL Extension Wrangler Library Open source library Pre-built distribution: http://glew.sourceforge.net/ Source code: https://github.com/nigels-com/glew Your one-stop-shop for API support for all OpenGL extension APIs Just released GLEW 2.0 (July 2016) provides API support for ARB_gl_spirv EXT_window_rectangles All of NVIDIA’s Maxwell extensions All of NVIDIA’s Pascal extensions All other NVIDIA multi-GPU generation initiatives Examples: NV_path_rendering, NV_command_list, NV_gpu_multicast Thanks to Nigel Stewart, GLEW maintainer, for this
  • 120. 120 NVIDIA OpenGL in 2016 Provides OpenGL’s Maximally Available Superset Pascal Extensions 2015 ARB extensions OpenGL 4.5 Core Maxwell Extensions Legacy EXT & Other Compatibility Extensions OpenGL Complete Compatibility Path Rendering Multi-GPU. SLI Approaching Zero Driver Overhead NVIDIA Multi-generation GPU Initiatives DirectX inter-op Vulkan inter-op ES Enhancements Full OpenGL ES 3.2 Khronos Standard Expected Compatibility NVIDIA Initiatives GPU Generation Features
  • 121. 121 Last Words •Lots of new OpenGL features in NVIDIA’s 2016 Driver •Highlights • OpenGL 2015 Khronos standard extensions all supported by NVIDIA • Maxwell’s features for • GPU Voxelization & Global Illumination • Vector Graphics • And Pascal supports all these features too • Pascal’s features for efficient Virtual Reality rendering • NVIDIA supports new ARB_gl_spirv extension • Provides shader compilation inter-operability for Vulkan and OpenGL
  • 122. 122 SIGGRAPH Paper Using OpenGL to Check Out • Harnesses OpenGL-based GPU tessellation • Avoids the complex patch splitting in current OpenSubdiv approach • Wednesday, July 27 • Ballroom C/D/E • 3:45 to 5:55pm session

Editor's Notes

  • #5: Intro: 3-4m Exercise 1: 10m
  • #7: One consistent user experience across many platforms
  • #9: Released at Oculus Connect Immersive firefight experience ala The Matrix The engine completes a render pass on the left eye, then proceeds to the right The same render target is used for each eye