SlideShare a Scribd company logo

Syysgraph 2018 - Modern Graphics Abstractions & Real-Time Ray Tracing

Electronic Arts / DICE, profile picture
Electronic Arts / DICE

Modern Graphics Abstractions & Real-Time Ray Tracing discusses Halcyon, a graphics rendering system built from scratch using modern graphics APIs. Halcyon uses render handles, commands, backends, devices, and graphs to provide an efficient and flexible rendering system that works across APIs. It also details virtual multi-GPU capabilities that allow developers to test multi-GPU code even on single-GPU machines.

Technology
1 of 127
Downloaded 69 times
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
Modern Graphics Abstractions & Real-Time Ray Tracing Syysgraph 2018 Graham Wihlidal, Colin Barré-Brisebois SEED – Electronic Arts
Modern Graphics Abstractions
Halcyon Rendering ▪ “PICA PICA” and Halcyon built from scratch ▪ Implemented lots of bespoke technology ▪ Only modern abstractions (DX12, VK, Metal 2, …) ▪ Minimal effort to add a new API or platform ▪ Efficient and flexible rendering was a major focus S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
Halcyon Rendering S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing Render Handles Render Commands Render Backend Render Device Render Backend Render DeviceRender Device Render Backend Render Proxy Render Graph Render Graph Application
Render Backend
Render Backend ▪ Live-reloadable DLLs ▪ Enumerates adapters and capabilities ▪ Swap chain support ▪ Extensions (i.e. ray tracing, sub groups, …) ▪ Determine adapter(s) to use S E E D // Halcyon + Vulkan
Render Backend ▪ Provides debugging and profiling ▪ RenderDoc integration, validation layers, … ▪ Create and destroy render devices S E E D // Halcyon + Vulkan
Render Backend S E E D // Halcyon + Vulkan ▪ Direct3D 12 ▪ Vulkan 1.1 ▪ Metal 2 ▪ Proxy ▪ Mock
Render Backend S E E D // Halcyon + Vulkan ▪ Direct3D 12 ▪ Shader Model 6.X ▪ DirectX Ray Tracing ▪ Bindless Resources ▪ Explicit Multi-GPU ▪ DirectML (soon..) ▪ …
Render Backend S E E D // Halcyon + Vulkan ▪ Vulkan 1.1 ▪ Sub-groups ▪ Descriptor indexing ▪ External memory ▪ Multi-draw indirect ▪ Ray tracing (soon..) ▪ …
Render Backend S E E D // Halcyon + Vulkan ▪ Metal 2 ▪ Early development ▪ Primarily desktop ▪ Argument buffers ▪ Machine learning ▪ …
Render Backend S E E D // Halcyon + Vulkan ▪ Proxy ▪ Discussed later in the presentation
Render Backend S E E D // Halcyon + Vulkan ▪ Mock ▪ Performs resource tracking and validation ▪ Command stream is parsed and evaluated ▪ No submission to an API ▪ Useful for unit tests and debugging
Render Device
Render Device S E E D // Halcyon + Vulkan ▪ Abstraction of a logical GPU adapter ▪ e.g. VkDevice, ID3D12Device, … ▪ Provides interface to GPU queues ▪ Command list submission
Render Device S E E D // Halcyon + Vulkan ▪ Ownership of GPU resources ▪ Create & Destroy ▪ Lifetime tracking of resources ▪ Mapping render handles → device resources
Render Handles
S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing Render Handles ▪ Resources associated by handle ▪ Lightweight (64 bits) ▪ Constant-time lookup ▪ Type safety (i.e. buffer vs texture) ▪ Can be serialized or transmitted ▪ Generational for safety ▪ e.g. double-delete, usage after delete
S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing Render Handles ID3D12Resource ID3D12Resource DX12: Adapter 2 ID3D12Resource DX12: Adapter 3 ID3D12Resource DX12: Adapter 1DX12: Adapter 0 Render Handle ▪ Handles allow one-to-many cardinality [handle->devices] ▪ Each device can have a unique representation of the handle
S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing Render Handles ▪ Can query if a device has a handle loaded ▪ Safely add and remove devices ▪ Handle owned by application, representation can reload on device ID3D12Resource ID3D12Resource DX12: Adapter 2 ID3D12Resource DX12: Adapter 3 ID3D12Resource DX12: Adapter 1DX12: Adapter 0 Render Handle
S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing Render Handles ▪ Shared resources are supported ▪ Primary device owner, secondaries alias primary ID3D12Resource ID3D12Resource DX12: Adapter 2 ID3D12Resource DX12: Adapter 3 ID3D12Resource DX12: Adapter 1DX12: Adapter 0 Render Handle
S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing Render Handles ▪ Can also mix and match backends in the same process! ▪ Made debugging VK implementation much easier ▪ DX12 on left half of screen, VK on right half of screen ID3D12Resource ID3D12Resource VK: Adapter 0 VkImage Proxy: Adapter 0 Render Handle DX12: Adapter 1DX12: Adapter 0 Render Handle
Render Commands
Render Commands ▪ Draw ▪ DrawIndirect ▪ Dispatch ▪ DispatchIndirect ▪ UpdateBuffer ▪ UpdateTexture ▪ CopyBuffer ▪ CopyTexture ▪ Barriers ▪ Transitions ▪ BeginTiming ▪ EndTiming ▪ ResolveTimings ▪ BeginEvent ▪ EndEvent ▪ BeginRenderPass ▪ EndRenderPass ▪ RayTrace ▪ UpdateTopLevel ▪ UpdateBottomLevel ▪ UpdateShaderTable S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
Render Commands ▪ Queue type specified ▪ Spec validation ▪ Allowed to run? ▪ e.g. draws on compute ▪ Automatic scheduling ▪ Where can it run? ▪ Async compute S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
Render Commands S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
Render Command List ▪ Encodes high level commands ▪ Tracks queue types encountered ▪ Queue mask indicating scheduling rules ▪ Commands are stateless - parallel recording S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
Render Compilation ▪ Render command lists are “compiled” ▪ Translation to low level API ▪ Can compile once, submit multiple times ▪ Serial operation (memcpy speed) ▪ Perfect redundant state filtering S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
Render Graph
Render Graph ▪ Inspired by FrameGraph [O’Donnell 2017] ▪ Automatically handle transient resources ▪ Import explicitly managed resources ▪ Automatic resource transitions ▪ Render target batching ▪ DiscardResource ▪ Memory aliasing barriers ▪ … S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
Render Graph ▪ Basic memory management ▪ Not targeting current consoles ▪ Fine grained memory reuse sub-optimal with current PC drivers ▪ Lose ~5% on aliasing barriers and discards ▪ Automatic queue scheduling ▪ Ongoing research ▪ Need heuristics on task duration and bottlenecks ▪ e.g. Memory vs ALU ▪ Not enough to specify dependencies S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
Render Graph ▪ Frame Graph → Render Graph: No concept of a “frame” ▪ Fully automatic transitions and split barriers ▪ Single implementation, regardless of backend ▪ Translation from high level render command stream ▪ API differences hidden from render graph ▪ Support for mGPU ▪ Mostly implicit and automatic ▪ Can specify a scheduling policy S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
Render Graph ▪ Composition of multiple graphs at varying frequencies ▪ Same GPU: async compute ▪ mGPU: graphs per GPU ▪ Out-of-core: server cluster, remote streaming S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
Render Graph ▪ Composition of multiple graphs at varying frequencies ▪ e.g. translucency, refraction, global illumination S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
Render Graph ▪ Two phases ▪ Graph construction ▪ Specify inputs and outputs ▪ Serial operation (by design) ▪ Graph evaluation ▪ Highly parallelized ▪ Record high level render commands ▪ Automatic barriers and transitions S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
 Construction phase  Evaluation phase
Render Graph S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing ▪ Explicit heterogeneous mGPU ▪ Parallel fork-join approach ▪ Resources copied through system memory using copy queue ▪ ~1ms for every 15mb transferred ▪ Minimize PCI-E transfers ▪ Immutable data replicated ▪ Tightly pack data GPU1 GPU2 GPU3 GPU4 Partition1[Primary] Partition2[Secondary] Partition3[Secondary] Partition4[Secondary]
Render Graph S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing ▪ Workloads are divided into partitions ▪ Based on GPU device count ▪ Single primary device ▪ Other devices are secondaries ▪ Variety of scheduling and transfer patterns are necessary ▪ Simple rules engine GPU1 GPU2 GPU3 GPU4 Partition1[Primary] Partition2[Secondary] Partition3[Secondary] Partition4[Secondary]
Render Graph S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing ▪ Run ray generation on primary GPU ▪ Copy results in sub-regions to other GPUs ▪ Run tracing phases on separate GPUs ▪ Copy tracing results back to primary GPU ▪ Run filtering on primary GPU
Render Graph S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing ▪ Only width is divided ▪ Simplifies textures vs. buffers ▪ Passes are unaware of GPU count
▪ Lots of fun coordinate snapping bugs ▪ i.e. 3 GPUs partitioned to 0.33333…
▪ Lots of fun coordinate snapping bugs ▪ 16 GPUs! (because, why not?)
Render Graph S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing ▪ RenderGraphSchedule ▪ NoDevices → Pass is disabled ▪ AllDevices → Pass runs on all devices ▪ PrimaryDevice → Pass only runs on primary device ▪ SecondaryDevices → Pass runs on secondaries if count > 1, otherwise primary ▪ OnlySecondaryDevices → Pass only runs on secondary devices, disabled unless mGPU Requested Per Pass →
Render Graph S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing ▪ RenderTransferPartition ▪ PartitionAll → Select all partitions from device ▪ PartitionIsolated → Select isolated region from device ▪ RenderTransferFilter ▪ AllDevices → Transfer completes on all devices ▪ PrimaryDevice → Transfer completes on the primary device ▪ SecondaryDevices → Transfer completes on all secondary devices Requested Per Pass →
Render Graph S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing ▪ PartitionAll → PartitionAll ▪ Copies full resource on one GPU to full resource on all specified GPUs ▪ PartitionAll → PartitionIsolated ▪ Copies full resource on one GPU to isolated regions on all specified GPUs (partial copies) ▪ PartitionIsolated → PartitionAll ▪ (Invalid configuration) ▪ PartitionIsolated → PartitionIsolated ▪ Copies isolated region on one GPU to isolated regions on all specified GPUs (partial copies)
Devices this pass will run on Schedule transfers in or out Scaling work dimensions for each GPU
Some bugs were obvious
Some bugs were obvious
Render Graph S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing ▪ Some bugs were subtle ▪ Weird cell shading? ☺ ▪ Incorrect transfers? ▪ Transfers in (input data) ▪ Transfers out (result data) ▪ Incorrect scheduling? ▪ Pass not running ▪ Pass running when it shouldn’t ▪ Partition window
Render Graph Some of our render graph passes: S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing ▪ Bloom ▪ BottomLevelUpdate ▪ BrdfLut ▪ CocDerive ▪ DepthPyramid ▪ DiffuseSh ▪ Dof ▪ Final ▪ GBuffer ▪ Gtao ▪ IblReflection ▪ ImGui ▪ InstanceTransforms ▪ Lighting ▪ MotionBlur ▪ Present ▪ RayTracing ▪ RayTracingAccum ▪ ReflectionFilter ▪ ReflectionSample ▪ ReflectionTrace ▪ Rtao ▪ Screenshot ▪ Segmentation ▪ ShaderTableUpdate ▪ ShadowFilter ▪ ShadowMask ▪ ShadowCascades ▪ ShadowTrace ▪ Skinning ▪ Ssr ▪ SurfelGapFill ▪ SurfelLighting ▪ SurfelPositions ▪ SurfelSpawn ▪ Svgf ▪ TemporalAa ▪ TemporalReproject ▪ TopLevelUpdate ▪ TranslucencyTrace ▪ Velocity ▪ Visibility
Render Graph S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing ▪ Implicit data flow via explicit scopes ▪ “Long-distance” extensible parameter passing ▪ Scope given to each render pass ▪ Can create nested scope for sub-graph ▪ Results stored into scope ▪ Hygiene via nesting and shadowing { gbuffer <- render_opaque() gbuffer <- render_decals(gbuffer) { gbuffer <- render_opaque() render_lighting(gbuffer) } -> envmap apply_envmap(gbuffer, envmap) } struct RenderGraphAreaLight { RenderGraphResource triangleLightList; uint32 triangleCount; };
Render Graph S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing ▪ Lookup by type ▪ scope.get<T>() -> &T ▪ Parameters in “plain old data” structs ▪ RenderGraphResource, RenderHandle ▪ float, int, mat4, etc. { gbuffer <- render_opaque() gbuffer <- render_decals(gbuffer) { gbuffer <- render_opaque() render_lighting(gbuffer) } -> envmap apply_envmap(gbuffer, envmap) } struct RenderGraphAreaLight { RenderGraphResource triangleLightList; uint32 triangleCount; };
Render Graph DSL ▪ Experimental ▪ Macro Magic S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
Virtual Multi-GPU
S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing Virtual Multi-GPU ▪ Most developers have single GPU ▪ Uncommon for 2 GPU machines ▪ Rare for 3+ GPU ▪ Practical for show floor and cranking up to 11 ▪ Impractical for regular development ☺
S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing Virtual Multi-GPU ▪ Build device indirection table ▪ Virtual device index → adapter index DX12: Adapter 2 ID3D12ResourceID3D12Resource DX12: Adapter 1 ID3D12Resource DX12: Adapter 0 Device 0 Device 1 Device 2 Device 3 Device 4 Device 5
S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing Virtual Multi-GPU ▪ Create multiple instances of a device ▪ Virtual GPUs execute sequentially (WDDM) DX12: Adapter 2 ID3D12ResourceID3D12Resource DX12: Adapter 1 ID3D12Resource DX12: Adapter 0 Device 0 Device 1 Device 2 Device 3 Device 4 Device 5
S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing Virtual Multi-GPU ▪ Increases overall wall time (don’t use for profiling) ▪ Amazing for development and testing! DX12: Adapter 2 ID3D12ResourceID3D12Resource DX12: Adapter 1 ID3D12Resource DX12: Adapter 0 Device 0 Device 1 Device 2 Device 3 Device 4 Device 5
S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing Virtual Multi-GPU ▪ PICA PICA developers all had 1 GPU ▪ Limited testing with 2 GPUs ▪ Show floor at GDC 2018 was 4 GPUs ▪ Virtual-only testing… ▪ Crossed fingers ▪ Worked flawlessly!
S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing Virtual Multi-GPU ▪ Develop and debug multi-GPU with only a single GPU ▪ Virtual mGPU reliably reproduces most bugs! ▪ Entire features developed without physical mGPU ▪ i.e. Surfel GI (the night before GDC.. ☺)
Render Proxy
▪ Remote render backend ▪ Any API / Any OS Render Proxy S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
▪ Render API calls are routed remotely ▪ Uses gRPC (high performance RPC framework) ▪ Use an API on an incompatible OS ▪ e.g. Direct3D 12 on macOS or Linux Render Proxy S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
▪ Scale large workloads with a GPU cluster ▪ Some API as render graph mGPU ▪ Only rendering is routed, scene state is local ▪ Work from the couch! ▪ i.e. DirectX ray tracing on a MacBook ☺ Render Proxy S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
Render device → gRPC
Protobuf 3 Schema
▪ The possibilities are endless! Render Proxy S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
Machine Learning
Machine Learning ▪ Deep reinforcement learning ▪ Rendering 36 semantic views ▪ Training with TensorFlow ▪ On-premise GPU cluster ▪ Inference with TensorFlow ▪ CPU AVX2 ▪ In-process S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
Machine Learning ▪ Adding inferencing with DirectML ▪ Hardware accelerated inferencing operators ▪ Resource management ▪ Schedule ML work explicitly ▪ Interleave ML work with other GPU workloads ▪ Fall back for other APIs S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
Machine Learning ▪ Treat trained ML models like any other 3D asset ▪ Render Graph abstractions ▪ Reference the same render resources ▪ Similar to chaining compute passes ▪ Record “meta” render commands ▪ Backends can fuse or transform, if desired S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
Machine Learning ▪ Provide operators as render commands S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing ▪ Activation ▪ Convolution ▪ Elementwise ▪ FC ▪ GRU ▪ LSTM ▪ MatMul ▪ Normalization ▪ Pooling ▪ Random ▪ RNN ▪ etc.
Asset Pipelines
Asset Pipelines ▪ Geometry ▪ Animations ▪ Shaders ▪ Sounds ▪ Music ▪ Textures ▪ Scenes ▪ etc. S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
Asset Pipelines ▪ Everything is content addressable ▪ Hash of data is the identity ▪ Sha256 ▪ Merkle trees! ▪ Dependency evaluation S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
https://en.wikipedia.org/wiki/Merkle_tree
Asset Pipelines ▪ Containerized, running on Kubernetes ▪ Google Cloud Platform ▪ On-Premise Cluster ▪ AMD 1950X TR ▪ NV Titan V ▪ Communication using gRPC and Protobuf ▪ Google Cloud Storage S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
Asset Pipelines ▪ Analytics with Prometheus and Grafana ▪ Publish custom metrics ▪ Scraped into rich UI ▪ Collecting data is important! S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
Syysgraph 2018 - Modern Graphics Abstractions & Real-Time Ray Tracing
Shaders
Shaders ▪ Exclusively HLSL ▪ Shader Model 6.X ▪ Majority are compute shaders ▪ Performance is critical ▪ Group shared memory ▪ Wave-ops / Sub-groups S E E D // Halcyon + Vulkan
Shaders ▪ No reflection ▪ Avoid costly lookups ▪ Only explicit bindings ▪ … except for validation ▪ Extensive use of HLSL spaces ▪ Updates at varying frequency ▪ Bindless S E E D // Halcyon + Vulkan
Shaders S E E D // Halcyon + Vulkan SPIR-VDXIL Vulkan 1.1Direct3D 12 ISPCMSL SPIRV-CROSS AVX2, …Metal 2 DXC HLSL
Real-Time Ray Tracing
Hybrid Rendering Direct Shadows (ray trace or raster) Direct Lighting (compute) Reflections (ray trace or compute) G-Buffer (raster) Global Illumination (ray trace and compute) Post Processing (compute) Transparency & Translucency (ray trace and compute) Ambient Occlusion (ray trace or compute) S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
No Ray Tracing
Path Tracing (1 spp)
Path Tracing Accumulation (15s)
Hybrid Rendering
No Ray Tracing
Hybrid Rendering
Syysgraph 2018 - Modern Graphics Abstractions & Real-Time Ray Tracing
• Launch rays from G-Buffer • Trace at half resolution • Supports arbitrary normal & roughness • Extensive spacial & temporal reconstruction
Raw Reflections (1/4 rpp)
After Reconstruction & Filtering
▪ Launch ray towards light ▪ Handled by Miss Shader (DXR) ▪ Soft shadows? ▪ Random direction from cone [PBRT] ▪ SVGF-style reconstruction [Schied 2017]
Hard Raytraced Shadows Soft Raytraced Shadows (Unfiltered) Soft Raytraced Shadows (Filtered) S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
Transparent Shadows ▪ Hard to get right with raster ▪ [McGuire17] ▪ Trace towards light ▪ Like opaque shadow tracing ▪ Accumulate absorption ▪ Product of colors ▪ Thin film approximation ▪ Density absorption easy extension ▪ Can also be soft! S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
Transparent Shadows ▪ Keep tracing until ▪ Hit opaque surface or all light is absorbed or miss S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
Transparent Shadows (hard)
Transparent Shadows (soft)
ො𝑛 𝑝 𝐴 𝑝 = 1 𝜋 න Ω 𝑉𝑝, ෝ𝑤(ො𝑛 ⋅ ෝ𝑤)𝑑𝜔
Screen-Space AO
Raytraced AO
Transparency & Translucency ▪ Raytracing enables accurate light scattering ▪ Transparency ▪ Order-independent (OIT) ▪ Multiple index-of-refraction transitions ▪ Variable roughness, refractions and absorption ▪ Translucency ▪ Light scattering inside homogeneous medium ▪ We do this in texture-space ▪ Handle view-dependent terms & dynamic changes to the environment Texture-Space Glass and Translucency S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
Transparency Works for clear and rough glass ▪ Clear ▪ No filtering required ▪ Rough ▪ Microfacet Models for Refraction through Rough Surfaces [Walter2007] ▪ More samples + temporal filtering ▪ Apply phase function & BTDF S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
Translucency ▪ For every valid position & normal ▪ Flip normal and push (ray) inside ▪ Launch rays in uniform sphere dist. ▪ (Importance-sample phase function) ▪ Compute lighting at intersection ▪ Gather all samples ▪ Update value in texture S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
Syysgraph 2018 - Modern Graphics Abstractions & Real-Time Ray Tracing
GI ▪ Can’t solve GI every frame ▪ Our GI budget: 250k rays / frame ▪ World space surfels ▪ Easy to accumulate ▪ Distribute on the fly ▪ No parameterization ▪ Smooth result by construction ▪ Position, normal, radius, animation info ▪ Spawn surfels in world from view ▪ Surfel path tracing with incremental irradiance S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
Round 1: Victory! ▪ Initial DXR-enabled games to have a set of ray tracing features ▪ Reflections ▪ Shadows ▪ AO ▪ … ▪ Developers learning how to incorporate ray tracing in their existing pipelines ▪ Adapt existing backends to support the transition ▪ Akin to first round of launch titles on a new console ▪ Let’s get ready for Round 2? S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
Round 2: Open Problems
Transparency ▪ Transparency is far from being solved! ▪ Glass ▪ Clear/rough + filtered + shadowed ▪ Particles & Volumetric Effects ▪ Can use miss shader to update volumes ▪ Raymarching in hit shaders? ▪ Non-trivial blending & filtering ▪ PICA PICA: texture-space OIT with refractions and scattering ▪ Not perfect, but one step closer ▪ Denoising techniques don’t work so well with transparency (and 1-2 spp) Transparency in the Maxwell Renderer S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
Partial Coverage ▪ Foliage ▪ Can still do alpha test (i.e.: any-hit) ▪ Animated becomes a real problem ▪ Defocus effects such as motion blur and depth of field are still intractable ▪ Need partial coverage denoising @ 1-2 spp PBRT (Matt Pharr, Wenzel Jakob, Greg Humphreys) Disney / Pixar S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
Specialized Denoising ▪ PICA PICA: specialized denoising & reconstruction for view-dependent, light-dependent or any other term you can monitor variance across frames [Stachowiak2018] ▪ Can use general denoising on the whole image, but… ▪ Denoising filter around a specific term will achieve greater quality ▪ Potentially converges faster ▪ One algorithm to rule-them-all? ▪ Lately… ▪ Gradient Estimation for Real-Time Adaptive Temporal Filtering [Schied 2018] ▪ Lots of progress with DL approaches S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
Geometry++ ▪ Ray-Triangle intersection is the fast path ▪ Well-known problem [Möller 1997] [Baldwin 2016] ▪ Intersection of procedural geometry currently done via Intersection Shaders ▪ Allows custom/parametric intersection code, for arbitrary shapes ▪ Current (slow) path for procedural geometry ▪ Procedural: hair, water, and particles are still a challenge ▪ Hair: all done parametrically via intersection shaders? ▪ Water: can be done via triangles, but reflections + refractions are challenging ▪ Also, handle view-dependent LODing (ie.: water patches) vs ray-dependent LODing? ▪ DXR: Need ways to accelerate for non-obvious primitives ▪ Cone-BVH for ray-marching? (à-la-Aaltonen in Claybook) S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
Taking Advantage of DXR ▪ DirectX offers easy interoperability between raster, compute and raytracing ▪ Raytracing, rasterization and compute shaders can share code & types ▪ Evaluate your actual HLSL material shaders - directly usable for a hybrid raytracing pipeline ▪ The output from one stage can feed data for another ▪ Write to UAVs, read in next stage ▪ Prepare rays to be launched, and trace on another (i.e.: mGPU) ▪ Interop extends opportunities for solving new sparse problems ▪ Use the power of each stage to your advantage ▪ RT opens the door to an entirely new class of techniques for games ▪ DXR provides a playground where research & gamedevs can collaborate! S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
Performance (1/) ▪ Best tracing performance comes from one massive static BVH? ▪ Viable for a big static scene, but most games are not static… ▪ Animated Geometry: Acceleration Structure construction is fairly fast ▪ But not fast enough to run vertex shaders for all the animated geometry and build AS from scratch every frame ▪ Top Level: has to be rebuilt whenever anything interesting happens ▪ Animated objects, objects inserted/removed, LOD swap ▪ LOD selection/management and how that ties to Shader Table management ▪ Bottom Level: is there a sweet spot? ▪ Lots of individual rigid objects: might have to bake transforms and merge into larger ones ▪ How much memory can you spare for better tracing performance? S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
Performance (2/) ▪ 2+ levels hierarchies? ▪ + : Allow to split worlds into update frequencies, and avoid updates if nothing has changed ▪ - : Tracking the object-to-world transforms becomes difficult: ray needs a transform stack ▪ - : Apps can end up creating a large instance hierarchy of single-node trees → bad  ▪ Idea: have two bottom level BVH's, one for static and one for dynamic geo? ▪ Static: Built & left alone, traverse it first ▪ Dynamic: Rebuilt in the background as a long running task. Refitted every frame ▪ + Supports many animated characters ▪ While not taking a hit for rebuilding BVH for static geometry ▪ - Doesn’t solve massive number of materials that games typically have… S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
Performance (3/) ▪ Example: 1K materials, fire 100k secondary rays ▪ (Potentially) need to run 1k shaders each on a batch of 100 hits. Viable workload? ▪ Can’t just shove all material variations in the SBT and let IHVs optimize, right? ▪ How aggressively do shader variations / materials in game scenes need to be merged? 1 or 2 orders of magnitude? ▪ Uber Shaders + Texture Space Shading? ▪ PICA PICA Uber Shaders: Rebinding constants quite frequently was not an issue ▪ Use texture-space gbuffers, or is that too much memory? S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
Future Ray Tracing Research ▪ RTRT literature has focused on accelerating “correct” raytracing ▪ Lots new challenges with game ray tracing ▪ Literature has to adjust to game ray tracing constraints ▪ Games → budgeted amount rays/frame, rays/pixel, fixed frame times & memory budgets ▪ Games → Light transport caches: surfels, voxels, lightmaps ▪ Q: How many rays-per-pixel needed to have real-time perfect indirect diffuse? ▪ Q: What has to happen to get fully robust PBR real-time ray traced lighting? ▪ Perf R&D: Efficient sampling / integration strategies & reconstruction filtering ▪ Perf R&D: Caching of materials and partial solutions S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
Global Illumination ▪ Open problem even in offline rendering ▪ Variance still too high ▪ Can reduce frequency ▪ Prefiltering, path-space filtering ▪ Denoising & reconstruction ▪ Pinhole GI & lighting is not solved ▪ Incoherent shading → intractable performance ▪ Have to resort to caching to amortize shading ▪ PICA PICA: caching of GI via surfels ▪ Some issues: only spawn surfels from what you see ▪ Need to solve GI for user-generated content → [Ritchell 2011] S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
Summary ▪ Modern graphics abstractions are important for efficiency without verbosity ▪ Real-time ray tracing significantly improves visuals, but still have a lots to do to bridge offline and real-time ▪ DXR provides a playground where research and gamedevs can collaborate! S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
References S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing ▪ [Andersson 2018] Johan Andersson, Colin Barré-Brisebois.“DirectX: Evolving Microsoft's Graphics Platform”. available online ▪ [McGuire 2017] McGuire, Morgan and Mara, Michael. “Phenomenological Transparency”. available online ▪ [O’Donnell 2017] Yuriy O’Donnell. “Frame Graph: Extensible Rendering Architecture in Frostbite”. available online ▪ [Schied 2017] Schied, Christoph et. al. “Spatiotemporal Variance-Guided Filtering: Real-Time Reconstruction for Path- Traced Global Illumination”. available online ▪ [Stachowiak 2018] Tomasz Stachowiak. “Towards Effortless Photorealism Through Real-Time Raytracing” available online ▪ [Wihlidal 2018] Graham Wihlidal. “Halcyon + Vulkan”. available online
Thanks ▪ Matthäus Chajdas ▪ Rys Sommefeldt ▪ Timothy Lottes ▪ Tobias Hector ▪ Neil Henning ▪ John Kessenich ▪ Hai Nguyen ▪ Nuno Subtil ▪ Adam Sawicki ▪ Alon Or-bach ▪ Baldur Karlsson ▪ Cort Stratton ▪ Mathias Schott ▪ Rolando Caloca ▪ Sebastian Aaltonen ▪ Hans-Kristian Arntzen ▪ Yuriy O’Donnell ▪ Arseny Kapoulkine ▪ Tex Riddell ▪ Marcelo Lopez Ruiz ▪ Lei Zhang ▪ Greg Roth ▪ Noah Fredriks ▪ Qun Lin ▪ Ehsan Nasiri, ▪ Steven Perron ▪ Alan Baker ▪ Diego Novillo ▪ Tomasz Stachowiak
Thanks ▪ SEED ▪ Johan Andersson ▪ Jasper Bekkers ▪ Joakim Bergdahl ▪ Ken Brown ▪ Dean Calver ▪ Dirk de la Hunt ▪ Jenna Frisk ▪ Paul Greveson ▪ Henrik Halen ▪ Effeli Holst ▪ Andrew Lauritzen ▪ Magnus Nordin ▪ Niklas Nummelin ▪ Anastasia Opara ▪ Kristoffer Sjöö ▪ Ida Winterhaven ▪ Tomasz Stachowiak ▪ Microsoft ▪ Chas Boyd ▪ Ivan Nevraev ▪ Amar Patel ▪ Matt Sandy ▪ NVIDIA ▪ Tomas Akenine-Möller ▪ Nir Benty ▪ Jiho Choi ▪ Peter Harrison ▪ Alex Hyder ▪ Jon Jansen ▪ Aaron Lefohn ▪ Ignacio Llamas ▪ Henry Moreton ▪ Martin Stich
S E E D / / S E A R C H F O R E X T R A O R D I N A R Y E X P E R I E N C E S D I V I S I O N S T O C K H O L M – L O S A N G E L E S – M O N T R É A L – R E M O T E S E E D . E A . C O M W E ‘ R E H I R I N G !
Questions? Graham Wihlidal graham@ea.com @gwihlidal Colin Barré-Brisebois cbbrisebois@ea.com @ZigguratVertigo

More Related Content

PPT
Frostbite Rendering Architecture and Real-time Procedural Shading & Texturing...
repii
 
PPTX
Future Directions for Compute-for-Graphics
Electronic Arts / DICE
 
PPTX
HPG 2018 - Game Ray Tracing: State-of-the-Art and Open Problems
Electronic Arts / DICE
 
PPTX
Parallel Futures of a Game Engine (v2.0)
repii
 
PDF
Lighting Shading by John Hable
Naughty Dog
 
PPT
A Bit More Deferred Cry Engine3
guest11b095
 
PPTX
FrameGraph: Extensible Rendering Architecture in Frostbite
Electronic Arts / DICE
 
PPTX
A Certain Slant of Light - Past, Present and Future Challenges of Global Illu...
Electronic Arts / DICE
 
Frostbite Rendering Architecture and Real-time Procedural Shading & Texturing...
repii
 
Future Directions for Compute-for-Graphics
Electronic Arts / DICE
 
HPG 2018 - Game Ray Tracing: State-of-the-Art and Open Problems
Electronic Arts / DICE
 
Parallel Futures of a Game Engine (v2.0)
repii
 
Lighting Shading by John Hable
Naughty Dog
 
A Bit More Deferred Cry Engine3
guest11b095
 
FrameGraph: Extensible Rendering Architecture in Frostbite
Electronic Arts / DICE
 
A Certain Slant of Light - Past, Present and Future Challenges of Global Illu...
Electronic Arts / DICE
 

What's hot (20)

PDF
Practical SPU Programming in God of War III
Slide_N
 
PPT
Crysis Next-Gen Effects (GDC 2008)
Tiago Sousa
 
PDF
Graphics Gems from CryENGINE 3 (Siggraph 2013)
Tiago Sousa
 
PPTX
Triangle Visibility buffer
Wolfgang Engel
 
PPTX
Rendering Technologies from Crysis 3 (GDC 2013)
Tiago Sousa
 
PPTX
Khronos Munich 2018 - Halcyon and Vulkan
Electronic Arts / DICE
 
PDF
Deferred Rendering in Killzone 2
Guerrilla
 
PDF
Advanced Scenegraph Rendering Pipeline
Narann29
 
PPTX
Terrain in Battlefield 3: A Modern, Complete and Scalable System
Electronic Arts / DICE
 
PPTX
Frostbite on Mobile
Electronic Arts / DICE
 
PPTX
Optimizing the Graphics Pipeline with Compute, GDC 2016
Graham Wihlidal
 
PDF
Screen Space Reflections in The Surge
Michele Giacalone
 
PPTX
Past, Present and Future Challenges of Global Illumination in Games
Colin Barré-Brisebois
 
PPTX
Moving Frostbite to Physically Based Rendering
Electronic Arts / DICE
 
PPTX
Parallel Futures of a Game Engine
repii
 
PPTX
Parallel Graphics in Frostbite - Current & Future (Siggraph 2009)
repii
 
PDF
Rendering Tech of Space Marine
Pope Kim
 
PDF
Practical Spherical Harmonics Based PRT Methods
Naughty Dog
 
PDF
An introduction to Realistic Ocean Rendering through FFT - Fabio Suriano - Co...
Codemotion
 
PDF
Penner pre-integrated skin rendering (siggraph 2011 advances in real-time r...
JP Lee
 
Practical SPU Programming in God of War III
Slide_N
 
Crysis Next-Gen Effects (GDC 2008)
Tiago Sousa
 
Graphics Gems from CryENGINE 3 (Siggraph 2013)
Tiago Sousa
 
Triangle Visibility buffer
Wolfgang Engel
 
Rendering Technologies from Crysis 3 (GDC 2013)
Tiago Sousa
 
Khronos Munich 2018 - Halcyon and Vulkan
Electronic Arts / DICE
 
Deferred Rendering in Killzone 2
Guerrilla
 
Advanced Scenegraph Rendering Pipeline
Narann29
 
Terrain in Battlefield 3: A Modern, Complete and Scalable System
Electronic Arts / DICE
 
Frostbite on Mobile
Electronic Arts / DICE
 
Optimizing the Graphics Pipeline with Compute, GDC 2016
Graham Wihlidal
 
Screen Space Reflections in The Surge
Michele Giacalone
 
Past, Present and Future Challenges of Global Illumination in Games
Colin Barré-Brisebois
 
Moving Frostbite to Physically Based Rendering
Electronic Arts / DICE
 
Parallel Futures of a Game Engine
repii
 
Parallel Graphics in Frostbite - Current & Future (Siggraph 2009)
repii
 
Rendering Tech of Space Marine
Pope Kim
 
Practical Spherical Harmonics Based PRT Methods
Naughty Dog
 
An introduction to Realistic Ocean Rendering through FFT - Fabio Suriano - Co...
Codemotion
 
Penner pre-integrated skin rendering (siggraph 2011 advances in real-time r...
JP Lee
 
Ad

Similar to Syysgraph 2018 - Modern Graphics Abstractions & Real-Time Ray Tracing (20)

PDF
DX12 & Vulkan: Dawn of a New Generation of Graphics APIs
AMD Developer Central
 
PDF
SEED - Halcyon Architecture
Electronic Arts / DICE
 
PDF
Keynote (Johan Andersson) - Mantle for Developers - by Johan Andersson, Techn...
AMD Developer Central
 
PPTX
Mantle for Developers
Electronic Arts / DICE
 
PDF
Umbra Ignite 2015: Jérémy Virga – Dishonored 2 rendering engine architecture ...
Umbra Software
 
PPTX
The Rendering Pipeline - Challenges & Next Steps
repii
 
PPTX
Graphics Processing unit ppt
VictorAbhinav
 
PPTX
Graphics pipelining
Areena Javed
 
PPTX
[Unite Seoul 2019] Mali GPU Architecture and Mobile Studio
Owen Wu
 
PDF
Optimizing HDRP with NVIDIA Nsight Graphics – Unite Copenhagen 2019
Unity Technologies
 
PPTX
The next generation of GPU APIs for Game Engines
Pooya Eimandar
 
PDF
3 boyd direct3_d12 (1)
mistercteam
 
PDF
[2018 GDC] Real-Time Ray-Tracing Techniques for Integration into Existing Ren...
Takahiro Harada
 
PDF
Modern Graphics Pipeline Overview
slantsixgames
 
PDF
Minko - Flash Conference #5
Minko3D
 
PDF
IRJET-A Study on Parallization of Genetic Algorithms on GPUS using CUDA
IRJET Journal
 
PPTX
Graphics processing unit ppt
Sandeep Singh
 
PDF
Mobile crossplatformchallenges siggraph
changehee lee
 
PDF
Mobile crossplatformchallenges siggraph
JP Lee
 
PDF
[03 1][gpu용 개발자 도구 - parallel nsight 및 axe] miller axe
laparuma
 
DX12 & Vulkan: Dawn of a New Generation of Graphics APIs
AMD Developer Central
 
SEED - Halcyon Architecture
Electronic Arts / DICE
 
Keynote (Johan Andersson) - Mantle for Developers - by Johan Andersson, Techn...
AMD Developer Central
 
Mantle for Developers
Electronic Arts / DICE
 
Umbra Ignite 2015: Jérémy Virga – Dishonored 2 rendering engine architecture ...
Umbra Software
 
The Rendering Pipeline - Challenges & Next Steps
repii
 
Graphics Processing unit ppt
VictorAbhinav
 
Graphics pipelining
Areena Javed
 
[Unite Seoul 2019] Mali GPU Architecture and Mobile Studio
Owen Wu
 
Optimizing HDRP with NVIDIA Nsight Graphics – Unite Copenhagen 2019
Unity Technologies
 
The next generation of GPU APIs for Game Engines
Pooya Eimandar
 
3 boyd direct3_d12 (1)
mistercteam
 
[2018 GDC] Real-Time Ray-Tracing Techniques for Integration into Existing Ren...
Takahiro Harada
 
Modern Graphics Pipeline Overview
slantsixgames
 
Minko - Flash Conference #5
Minko3D
 
IRJET-A Study on Parallization of Genetic Algorithms on GPUS using CUDA
IRJET Journal
 
Graphics processing unit ppt
Sandeep Singh
 
Mobile crossplatformchallenges siggraph
changehee lee
 
Mobile crossplatformchallenges siggraph
JP Lee
 
[03 1][gpu용 개발자 도구 - parallel nsight 및 axe] miller axe
laparuma
 
Ad

More from Electronic Arts / DICE (20)

PPTX
GDC2019 - SEED - Towards Deep Generative Models in Game Development
Electronic Arts / DICE
 
PPT
SIGGRAPH 2010 - Style and Gameplay in the Mirror's Edge
Electronic Arts / DICE
 
PDF
CEDEC 2018 - Towards Effortless Photorealism Through Real-Time Raytracing
Electronic Arts / DICE
 
PPTX
CEDEC 2018 - Functional Symbiosis of Art Direction and Proceduralism
Electronic Arts / DICE
 
PPTX
SIGGRAPH 2018 - PICA PICA and NVIDIA Turing
Electronic Arts / DICE
 
PPTX
SIGGRAPH 2018 - Full Rays Ahead! From Raster to Real-Time Raytracing
Electronic Arts / DICE
 
PDF
EPC 2018 - SEED - Exploring The Collaboration Between Proceduralism & Deep Le...
Electronic Arts / DICE
 
PDF
DD18 - SEED - Raytracing in Hybrid Real-Time Rendering
Electronic Arts / DICE
 
PDF
Creativity of Rules and Patterns: Designing Procedural Systems
Electronic Arts / DICE
 
PPTX
Shiny Pixels and Beyond: Real-Time Raytracing at SEED
Electronic Arts / DICE
 
PPTX
Physically Based Sky, Atmosphere and Cloud Rendering in Frostbite
Electronic Arts / DICE
 
PPTX
High Dynamic Range color grading and display in Frostbite
Electronic Arts / DICE
 
PPTX
4K Checkerboard in Battlefield 1 and Mass Effect Andromeda
Electronic Arts / DICE
 
PPTX
Lighting the City of Glass
Electronic Arts / DICE
 
PPTX
Photogrammetry and Star Wars Battlefront
Electronic Arts / DICE
 
PPTX
Physically Based and Unified Volumetric Rendering in Frostbite
Electronic Arts / DICE
 
PPTX
Stochastic Screen-Space Reflections
Electronic Arts / DICE
 
PPTX
Rendering Battlefield 4 with Mantle
Electronic Arts / DICE
 
PPTX
Battlefield 4 + Frostbite + Mantle
Electronic Arts / DICE
 
PPTX
5 Major Challenges in Real-time Rendering (2012)
Electronic Arts / DICE
 
GDC2019 - SEED - Towards Deep Generative Models in Game Development
Electronic Arts / DICE
 
SIGGRAPH 2010 - Style and Gameplay in the Mirror's Edge
Electronic Arts / DICE
 
CEDEC 2018 - Towards Effortless Photorealism Through Real-Time Raytracing
Electronic Arts / DICE
 
CEDEC 2018 - Functional Symbiosis of Art Direction and Proceduralism
Electronic Arts / DICE
 
SIGGRAPH 2018 - PICA PICA and NVIDIA Turing
Electronic Arts / DICE
 
SIGGRAPH 2018 - Full Rays Ahead! From Raster to Real-Time Raytracing
Electronic Arts / DICE
 
EPC 2018 - SEED - Exploring The Collaboration Between Proceduralism & Deep Le...
Electronic Arts / DICE
 
DD18 - SEED - Raytracing in Hybrid Real-Time Rendering
Electronic Arts / DICE
 
Creativity of Rules and Patterns: Designing Procedural Systems
Electronic Arts / DICE
 
Shiny Pixels and Beyond: Real-Time Raytracing at SEED
Electronic Arts / DICE
 
Physically Based Sky, Atmosphere and Cloud Rendering in Frostbite
Electronic Arts / DICE
 
High Dynamic Range color grading and display in Frostbite
Electronic Arts / DICE
 
4K Checkerboard in Battlefield 1 and Mass Effect Andromeda
Electronic Arts / DICE
 
Lighting the City of Glass
Electronic Arts / DICE
 
Photogrammetry and Star Wars Battlefront
Electronic Arts / DICE
 
Physically Based and Unified Volumetric Rendering in Frostbite
Electronic Arts / DICE
 
Stochastic Screen-Space Reflections
Electronic Arts / DICE
 
Rendering Battlefield 4 with Mantle
Electronic Arts / DICE
 
Battlefield 4 + Frostbite + Mantle
Electronic Arts / DICE
 
5 Major Challenges in Real-time Rendering (2012)
Electronic Arts / DICE
 

Recently uploaded (20)

PDF
GamePlan Trading System Review: Professional Trader's Honest Take
SOFTTECHHUB
 
PDF
7 ChatGPT Prompts to Help You Define Your Ideal Customer Profile.pdf
SOFTTECHHUB
 
PPT
“AI and Expert System Decision Support & Business Intelligence Systems”
ZubinRadhakrishnan
 
PDF
Machine learning based COVID-19 study performance prediction
IAESIJAI
 
PDF
Review of recent advances in non-invasive hemoglobin estimation
IAESIJAI
 
PPTX
Big Data Technologies - Introduction.pptx
kuthubussaman1
 
PDF
The Rise and Fall of 3GPP – Time for a Sabbatical?
3G4G
 
PDF
Advanced Soft Computing BINUS July 2025.pdf
University of Hertfordshire
 
PPTX
Effective Security Operations Center (SOC) A Modern, Strategic, and Threat-In...
ReZa AdineH
 
PPTX
Detection-First SIEM: Rule Types, Dashboards, and Threat-Informed Strategy
ReZa AdineH
 
PPTX
VMware vSphere Foundation How to Sell Presentation-Ver1.4-2-14-2024.pptx
blackmambaettijean
 
PDF
Dropbox Q2 2025 Financial Results & Investor Presentation
Dropbox
 
PDF
Optimiser vos workloads AI/ML sur Amazon EC2 et AWS Graviton
Julien SIMON
 
PDF
Unlocking AI with Model Context Protocol (MCP)
Brian McKeiver
 
PDF
Spectral efficient network and resource selection model in 5G networks
IAESIJAI
 
PDF
KodekX | Application Modernization Development
KodekX
 
PPTX
Cloud computing and distributed systems.
kkkkkhan
 
PDF
GDG Cloud Iasi [PUBLIC] Florian Blaga - Unveiling the Evolution of Cybersecur...
athlonica
 
PDF
solutions_manual_-_materials___processing_in_manufacturing__demargo_.pdf
AbdullahSani29
 
PDF
Per capita expenditure prediction using model stacking based on satellite ima...
IAESIJAI
 
GamePlan Trading System Review: Professional Trader's Honest Take
SOFTTECHHUB
 
7 ChatGPT Prompts to Help You Define Your Ideal Customer Profile.pdf
SOFTTECHHUB
 
“AI and Expert System Decision Support & Business Intelligence Systems”
ZubinRadhakrishnan
 
Machine learning based COVID-19 study performance prediction
IAESIJAI
 
Review of recent advances in non-invasive hemoglobin estimation
IAESIJAI
 
Big Data Technologies - Introduction.pptx
kuthubussaman1
 
The Rise and Fall of 3GPP – Time for a Sabbatical?
3G4G
 
Advanced Soft Computing BINUS July 2025.pdf
University of Hertfordshire
 
Effective Security Operations Center (SOC) A Modern, Strategic, and Threat-In...
ReZa AdineH
 
Detection-First SIEM: Rule Types, Dashboards, and Threat-Informed Strategy
ReZa AdineH
 
VMware vSphere Foundation How to Sell Presentation-Ver1.4-2-14-2024.pptx
blackmambaettijean
 
Dropbox Q2 2025 Financial Results & Investor Presentation
Dropbox
 
Optimiser vos workloads AI/ML sur Amazon EC2 et AWS Graviton
Julien SIMON
 
Unlocking AI with Model Context Protocol (MCP)
Brian McKeiver
 
Spectral efficient network and resource selection model in 5G networks
IAESIJAI
 
KodekX | Application Modernization Development
KodekX
 
Cloud computing and distributed systems.
kkkkkhan
 
GDG Cloud Iasi [PUBLIC] Florian Blaga - Unveiling the Evolution of Cybersecur...
athlonica
 
solutions_manual_-_materials___processing_in_manufacturing__demargo_.pdf
AbdullahSani29
 
Per capita expenditure prediction using model stacking based on satellite ima...
IAESIJAI
 

Syysgraph 2018 - Modern Graphics Abstractions & Real-Time Ray Tracing

  • 1. Modern Graphics Abstractions & Real-Time Ray Tracing Syysgraph 2018 Graham Wihlidal, Colin Barré-Brisebois SEED – Electronic Arts
  • 3. Halcyon Rendering ▪ “PICA PICA” and Halcyon built from scratch ▪ Implemented lots of bespoke technology ▪ Only modern abstractions (DX12, VK, Metal 2, …) ▪ Minimal effort to add a new API or platform ▪ Efficient and flexible rendering was a major focus S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
  • 4. Halcyon Rendering S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing Render Handles Render Commands Render Backend Render Device Render Backend Render DeviceRender Device Render Backend Render Proxy Render Graph Render Graph Application
  • 6. Render Backend ▪ Live-reloadable DLLs ▪ Enumerates adapters and capabilities ▪ Swap chain support ▪ Extensions (i.e. ray tracing, sub groups, …) ▪ Determine adapter(s) to use S E E D // Halcyon + Vulkan
  • 7. Render Backend ▪ Provides debugging and profiling ▪ RenderDoc integration, validation layers, … ▪ Create and destroy render devices S E E D // Halcyon + Vulkan
  • 8. Render Backend S E E D // Halcyon + Vulkan ▪ Direct3D 12 ▪ Vulkan 1.1 ▪ Metal 2 ▪ Proxy ▪ Mock
  • 9. Render Backend S E E D // Halcyon + Vulkan ▪ Direct3D 12 ▪ Shader Model 6.X ▪ DirectX Ray Tracing ▪ Bindless Resources ▪ Explicit Multi-GPU ▪ DirectML (soon..) ▪ …
  • 10. Render Backend S E E D // Halcyon + Vulkan ▪ Vulkan 1.1 ▪ Sub-groups ▪ Descriptor indexing ▪ External memory ▪ Multi-draw indirect ▪ Ray tracing (soon..) ▪ …
  • 11. Render Backend S E E D // Halcyon + Vulkan ▪ Metal 2 ▪ Early development ▪ Primarily desktop ▪ Argument buffers ▪ Machine learning ▪ …
  • 12. Render Backend S E E D // Halcyon + Vulkan ▪ Proxy ▪ Discussed later in the presentation
  • 13. Render Backend S E E D // Halcyon + Vulkan ▪ Mock ▪ Performs resource tracking and validation ▪ Command stream is parsed and evaluated ▪ No submission to an API ▪ Useful for unit tests and debugging
  • 15. Render Device S E E D // Halcyon + Vulkan ▪ Abstraction of a logical GPU adapter ▪ e.g. VkDevice, ID3D12Device, … ▪ Provides interface to GPU queues ▪ Command list submission
  • 16. Render Device S E E D // Halcyon + Vulkan ▪ Ownership of GPU resources ▪ Create & Destroy ▪ Lifetime tracking of resources ▪ Mapping render handles → device resources
  • 18. S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing Render Handles ▪ Resources associated by handle ▪ Lightweight (64 bits) ▪ Constant-time lookup ▪ Type safety (i.e. buffer vs texture) ▪ Can be serialized or transmitted ▪ Generational for safety ▪ e.g. double-delete, usage after delete
  • 19. S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing Render Handles ID3D12Resource ID3D12Resource DX12: Adapter 2 ID3D12Resource DX12: Adapter 3 ID3D12Resource DX12: Adapter 1DX12: Adapter 0 Render Handle ▪ Handles allow one-to-many cardinality [handle->devices] ▪ Each device can have a unique representation of the handle
  • 20. S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing Render Handles ▪ Can query if a device has a handle loaded ▪ Safely add and remove devices ▪ Handle owned by application, representation can reload on device ID3D12Resource ID3D12Resource DX12: Adapter 2 ID3D12Resource DX12: Adapter 3 ID3D12Resource DX12: Adapter 1DX12: Adapter 0 Render Handle
  • 21. S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing Render Handles ▪ Shared resources are supported ▪ Primary device owner, secondaries alias primary ID3D12Resource ID3D12Resource DX12: Adapter 2 ID3D12Resource DX12: Adapter 3 ID3D12Resource DX12: Adapter 1DX12: Adapter 0 Render Handle
  • 22. S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing Render Handles ▪ Can also mix and match backends in the same process! ▪ Made debugging VK implementation much easier ▪ DX12 on left half of screen, VK on right half of screen ID3D12Resource ID3D12Resource VK: Adapter 0 VkImage Proxy: Adapter 0 Render Handle DX12: Adapter 1DX12: Adapter 0 Render Handle
  • 24. Render Commands ▪ Draw ▪ DrawIndirect ▪ Dispatch ▪ DispatchIndirect ▪ UpdateBuffer ▪ UpdateTexture ▪ CopyBuffer ▪ CopyTexture ▪ Barriers ▪ Transitions ▪ BeginTiming ▪ EndTiming ▪ ResolveTimings ▪ BeginEvent ▪ EndEvent ▪ BeginRenderPass ▪ EndRenderPass ▪ RayTrace ▪ UpdateTopLevel ▪ UpdateBottomLevel ▪ UpdateShaderTable S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
  • 25. Render Commands ▪ Queue type specified ▪ Spec validation ▪ Allowed to run? ▪ e.g. draws on compute ▪ Automatic scheduling ▪ Where can it run? ▪ Async compute S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
  • 26. Render Commands S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
  • 27. Render Command List ▪ Encodes high level commands ▪ Tracks queue types encountered ▪ Queue mask indicating scheduling rules ▪ Commands are stateless - parallel recording S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
  • 28. Render Compilation ▪ Render command lists are “compiled” ▪ Translation to low level API ▪ Can compile once, submit multiple times ▪ Serial operation (memcpy speed) ▪ Perfect redundant state filtering S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
  • 30. Render Graph ▪ Inspired by FrameGraph [O’Donnell 2017] ▪ Automatically handle transient resources ▪ Import explicitly managed resources ▪ Automatic resource transitions ▪ Render target batching ▪ DiscardResource ▪ Memory aliasing barriers ▪ … S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
  • 31. Render Graph ▪ Basic memory management ▪ Not targeting current consoles ▪ Fine grained memory reuse sub-optimal with current PC drivers ▪ Lose ~5% on aliasing barriers and discards ▪ Automatic queue scheduling ▪ Ongoing research ▪ Need heuristics on task duration and bottlenecks ▪ e.g. Memory vs ALU ▪ Not enough to specify dependencies S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
  • 32. Render Graph ▪ Frame Graph → Render Graph: No concept of a “frame” ▪ Fully automatic transitions and split barriers ▪ Single implementation, regardless of backend ▪ Translation from high level render command stream ▪ API differences hidden from render graph ▪ Support for mGPU ▪ Mostly implicit and automatic ▪ Can specify a scheduling policy S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
  • 33. Render Graph ▪ Composition of multiple graphs at varying frequencies ▪ Same GPU: async compute ▪ mGPU: graphs per GPU ▪ Out-of-core: server cluster, remote streaming S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
  • 34. Render Graph ▪ Composition of multiple graphs at varying frequencies ▪ e.g. translucency, refraction, global illumination S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
  • 35. Render Graph ▪ Two phases ▪ Graph construction ▪ Specify inputs and outputs ▪ Serial operation (by design) ▪ Graph evaluation ▪ Highly parallelized ▪ Record high level render commands ▪ Automatic barriers and transitions S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
  • 36.  Construction phase  Evaluation phase
  • 37. Render Graph S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing ▪ Explicit heterogeneous mGPU ▪ Parallel fork-join approach ▪ Resources copied through system memory using copy queue ▪ ~1ms for every 15mb transferred ▪ Minimize PCI-E transfers ▪ Immutable data replicated ▪ Tightly pack data GPU1 GPU2 GPU3 GPU4 Partition1[Primary] Partition2[Secondary] Partition3[Secondary] Partition4[Secondary]
  • 38. Render Graph S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing ▪ Workloads are divided into partitions ▪ Based on GPU device count ▪ Single primary device ▪ Other devices are secondaries ▪ Variety of scheduling and transfer patterns are necessary ▪ Simple rules engine GPU1 GPU2 GPU3 GPU4 Partition1[Primary] Partition2[Secondary] Partition3[Secondary] Partition4[Secondary]
  • 39. Render Graph S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing ▪ Run ray generation on primary GPU ▪ Copy results in sub-regions to other GPUs ▪ Run tracing phases on separate GPUs ▪ Copy tracing results back to primary GPU ▪ Run filtering on primary GPU
  • 40. Render Graph S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing ▪ Only width is divided ▪ Simplifies textures vs. buffers ▪ Passes are unaware of GPU count
  • 41. ▪ Lots of fun coordinate snapping bugs ▪ i.e. 3 GPUs partitioned to 0.33333…
  • 42. ▪ Lots of fun coordinate snapping bugs ▪ 16 GPUs! (because, why not?)
  • 43. Render Graph S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing ▪ RenderGraphSchedule ▪ NoDevices → Pass is disabled ▪ AllDevices → Pass runs on all devices ▪ PrimaryDevice → Pass only runs on primary device ▪ SecondaryDevices → Pass runs on secondaries if count > 1, otherwise primary ▪ OnlySecondaryDevices → Pass only runs on secondary devices, disabled unless mGPU Requested Per Pass →
  • 44. Render Graph S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing ▪ RenderTransferPartition ▪ PartitionAll → Select all partitions from device ▪ PartitionIsolated → Select isolated region from device ▪ RenderTransferFilter ▪ AllDevices → Transfer completes on all devices ▪ PrimaryDevice → Transfer completes on the primary device ▪ SecondaryDevices → Transfer completes on all secondary devices Requested Per Pass →
  • 45. Render Graph S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing ▪ PartitionAll → PartitionAll ▪ Copies full resource on one GPU to full resource on all specified GPUs ▪ PartitionAll → PartitionIsolated ▪ Copies full resource on one GPU to isolated regions on all specified GPUs (partial copies) ▪ PartitionIsolated → PartitionAll ▪ (Invalid configuration) ▪ PartitionIsolated → PartitionIsolated ▪ Copies isolated region on one GPU to isolated regions on all specified GPUs (partial copies)
  • 46. Devices this pass will run on Schedule transfers in or out Scaling work dimensions for each GPU
  • 47. Some bugs were obvious
  • 48. Some bugs were obvious
  • 49. Render Graph S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing ▪ Some bugs were subtle ▪ Weird cell shading? ☺ ▪ Incorrect transfers? ▪ Transfers in (input data) ▪ Transfers out (result data) ▪ Incorrect scheduling? ▪ Pass not running ▪ Pass running when it shouldn’t ▪ Partition window
  • 50. Render Graph Some of our render graph passes: S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing ▪ Bloom ▪ BottomLevelUpdate ▪ BrdfLut ▪ CocDerive ▪ DepthPyramid ▪ DiffuseSh ▪ Dof ▪ Final ▪ GBuffer ▪ Gtao ▪ IblReflection ▪ ImGui ▪ InstanceTransforms ▪ Lighting ▪ MotionBlur ▪ Present ▪ RayTracing ▪ RayTracingAccum ▪ ReflectionFilter ▪ ReflectionSample ▪ ReflectionTrace ▪ Rtao ▪ Screenshot ▪ Segmentation ▪ ShaderTableUpdate ▪ ShadowFilter ▪ ShadowMask ▪ ShadowCascades ▪ ShadowTrace ▪ Skinning ▪ Ssr ▪ SurfelGapFill ▪ SurfelLighting ▪ SurfelPositions ▪ SurfelSpawn ▪ Svgf ▪ TemporalAa ▪ TemporalReproject ▪ TopLevelUpdate ▪ TranslucencyTrace ▪ Velocity ▪ Visibility
  • 51. Render Graph S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing ▪ Implicit data flow via explicit scopes ▪ “Long-distance” extensible parameter passing ▪ Scope given to each render pass ▪ Can create nested scope for sub-graph ▪ Results stored into scope ▪ Hygiene via nesting and shadowing { gbuffer <- render_opaque() gbuffer <- render_decals(gbuffer) { gbuffer <- render_opaque() render_lighting(gbuffer) } -> envmap apply_envmap(gbuffer, envmap) } struct RenderGraphAreaLight { RenderGraphResource triangleLightList; uint32 triangleCount; };
  • 52. Render Graph S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing ▪ Lookup by type ▪ scope.get<T>() -> &T ▪ Parameters in “plain old data” structs ▪ RenderGraphResource, RenderHandle ▪ float, int, mat4, etc. { gbuffer <- render_opaque() gbuffer <- render_decals(gbuffer) { gbuffer <- render_opaque() render_lighting(gbuffer) } -> envmap apply_envmap(gbuffer, envmap) } struct RenderGraphAreaLight { RenderGraphResource triangleLightList; uint32 triangleCount; };
  • 53. Render Graph DSL ▪ Experimental ▪ Macro Magic S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
  • 55. S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing Virtual Multi-GPU ▪ Most developers have single GPU ▪ Uncommon for 2 GPU machines ▪ Rare for 3+ GPU ▪ Practical for show floor and cranking up to 11 ▪ Impractical for regular development ☺
  • 56. S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing Virtual Multi-GPU ▪ Build device indirection table ▪ Virtual device index → adapter index DX12: Adapter 2 ID3D12ResourceID3D12Resource DX12: Adapter 1 ID3D12Resource DX12: Adapter 0 Device 0 Device 1 Device 2 Device 3 Device 4 Device 5
  • 57. S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing Virtual Multi-GPU ▪ Create multiple instances of a device ▪ Virtual GPUs execute sequentially (WDDM) DX12: Adapter 2 ID3D12ResourceID3D12Resource DX12: Adapter 1 ID3D12Resource DX12: Adapter 0 Device 0 Device 1 Device 2 Device 3 Device 4 Device 5
  • 58. S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing Virtual Multi-GPU ▪ Increases overall wall time (don’t use for profiling) ▪ Amazing for development and testing! DX12: Adapter 2 ID3D12ResourceID3D12Resource DX12: Adapter 1 ID3D12Resource DX12: Adapter 0 Device 0 Device 1 Device 2 Device 3 Device 4 Device 5
  • 59. S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing Virtual Multi-GPU ▪ PICA PICA developers all had 1 GPU ▪ Limited testing with 2 GPUs ▪ Show floor at GDC 2018 was 4 GPUs ▪ Virtual-only testing… ▪ Crossed fingers ▪ Worked flawlessly!
  • 60. S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing Virtual Multi-GPU ▪ Develop and debug multi-GPU with only a single GPU ▪ Virtual mGPU reliably reproduces most bugs! ▪ Entire features developed without physical mGPU ▪ i.e. Surfel GI (the night before GDC.. ☺)
  • 62. ▪ Remote render backend ▪ Any API / Any OS Render Proxy S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
  • 63. ▪ Render API calls are routed remotely ▪ Uses gRPC (high performance RPC framework) ▪ Use an API on an incompatible OS ▪ e.g. Direct3D 12 on macOS or Linux Render Proxy S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
  • 64. ▪ Scale large workloads with a GPU cluster ▪ Some API as render graph mGPU ▪ Only rendering is routed, scene state is local ▪ Work from the couch! ▪ i.e. DirectX ray tracing on a MacBook ☺ Render Proxy S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
  • 67. ▪ The possibilities are endless! Render Proxy S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
  • 69. Machine Learning ▪ Deep reinforcement learning ▪ Rendering 36 semantic views ▪ Training with TensorFlow ▪ On-premise GPU cluster ▪ Inference with TensorFlow ▪ CPU AVX2 ▪ In-process S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
  • 70. Machine Learning ▪ Adding inferencing with DirectML ▪ Hardware accelerated inferencing operators ▪ Resource management ▪ Schedule ML work explicitly ▪ Interleave ML work with other GPU workloads ▪ Fall back for other APIs S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
  • 71. Machine Learning ▪ Treat trained ML models like any other 3D asset ▪ Render Graph abstractions ▪ Reference the same render resources ▪ Similar to chaining compute passes ▪ Record “meta” render commands ▪ Backends can fuse or transform, if desired S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
  • 72. Machine Learning ▪ Provide operators as render commands S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing ▪ Activation ▪ Convolution ▪ Elementwise ▪ FC ▪ GRU ▪ LSTM ▪ MatMul ▪ Normalization ▪ Pooling ▪ Random ▪ RNN ▪ etc.
  • 74. Asset Pipelines ▪ Geometry ▪ Animations ▪ Shaders ▪ Sounds ▪ Music ▪ Textures ▪ Scenes ▪ etc. S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
  • 75. Asset Pipelines ▪ Everything is content addressable ▪ Hash of data is the identity ▪ Sha256 ▪ Merkle trees! ▪ Dependency evaluation S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
  • 77. Asset Pipelines ▪ Containerized, running on Kubernetes ▪ Google Cloud Platform ▪ On-Premise Cluster ▪ AMD 1950X TR ▪ NV Titan V ▪ Communication using gRPC and Protobuf ▪ Google Cloud Storage S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
  • 78. Asset Pipelines ▪ Analytics with Prometheus and Grafana ▪ Publish custom metrics ▪ Scraped into rich UI ▪ Collecting data is important! S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
  • 81. Shaders ▪ Exclusively HLSL ▪ Shader Model 6.X ▪ Majority are compute shaders ▪ Performance is critical ▪ Group shared memory ▪ Wave-ops / Sub-groups S E E D // Halcyon + Vulkan
  • 82. Shaders ▪ No reflection ▪ Avoid costly lookups ▪ Only explicit bindings ▪ … except for validation ▪ Extensive use of HLSL spaces ▪ Updates at varying frequency ▪ Bindless S E E D // Halcyon + Vulkan
  • 83. Shaders S E E D // Halcyon + Vulkan SPIR-VDXIL Vulkan 1.1Direct3D 12 ISPCMSL SPIRV-CROSS AVX2, …Metal 2 DXC HLSL
  • 85. Hybrid Rendering Direct Shadows (ray trace or raster) Direct Lighting (compute) Reflections (ray trace or compute) G-Buffer (raster) Global Illumination (ray trace and compute) Post Processing (compute) Transparency & Translucency (ray trace and compute) Ambient Occlusion (ray trace or compute) S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
  • 93. • Launch rays from G-Buffer • Trace at half resolution • Supports arbitrary normal & roughness • Extensive spacial & temporal reconstruction
  • 96. ▪ Launch ray towards light ▪ Handled by Miss Shader (DXR) ▪ Soft shadows? ▪ Random direction from cone [PBRT] ▪ SVGF-style reconstruction [Schied 2017]
  • 97. Hard Raytraced Shadows Soft Raytraced Shadows (Unfiltered) Soft Raytraced Shadows (Filtered) S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
  • 98. Transparent Shadows ▪ Hard to get right with raster ▪ [McGuire17] ▪ Trace towards light ▪ Like opaque shadow tracing ▪ Accumulate absorption ▪ Product of colors ▪ Thin film approximation ▪ Density absorption easy extension ▪ Can also be soft! S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
  • 99. Transparent Shadows ▪ Keep tracing until ▪ Hit opaque surface or all light is absorbed or miss S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
  • 102. ො𝑛 𝑝 𝐴 𝑝 = 1 𝜋 න Ω 𝑉𝑝, ෝ𝑤(ො𝑛 ⋅ ෝ𝑤)𝑑𝜔
  • 105. Transparency & Translucency ▪ Raytracing enables accurate light scattering ▪ Transparency ▪ Order-independent (OIT) ▪ Multiple index-of-refraction transitions ▪ Variable roughness, refractions and absorption ▪ Translucency ▪ Light scattering inside homogeneous medium ▪ We do this in texture-space ▪ Handle view-dependent terms & dynamic changes to the environment Texture-Space Glass and Translucency S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
  • 106. Transparency Works for clear and rough glass ▪ Clear ▪ No filtering required ▪ Rough ▪ Microfacet Models for Refraction through Rough Surfaces [Walter2007] ▪ More samples + temporal filtering ▪ Apply phase function & BTDF S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
  • 107. Translucency ▪ For every valid position & normal ▪ Flip normal and push (ray) inside ▪ Launch rays in uniform sphere dist. ▪ (Importance-sample phase function) ▪ Compute lighting at intersection ▪ Gather all samples ▪ Update value in texture S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
  • 109. GI ▪ Can’t solve GI every frame ▪ Our GI budget: 250k rays / frame ▪ World space surfels ▪ Easy to accumulate ▪ Distribute on the fly ▪ No parameterization ▪ Smooth result by construction ▪ Position, normal, radius, animation info ▪ Spawn surfels in world from view ▪ Surfel path tracing with incremental irradiance S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
  • 110. Round 1: Victory! ▪ Initial DXR-enabled games to have a set of ray tracing features ▪ Reflections ▪ Shadows ▪ AO ▪ … ▪ Developers learning how to incorporate ray tracing in their existing pipelines ▪ Adapt existing backends to support the transition ▪ Akin to first round of launch titles on a new console ▪ Let’s get ready for Round 2? S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
  • 112. Transparency ▪ Transparency is far from being solved! ▪ Glass ▪ Clear/rough + filtered + shadowed ▪ Particles & Volumetric Effects ▪ Can use miss shader to update volumes ▪ Raymarching in hit shaders? ▪ Non-trivial blending & filtering ▪ PICA PICA: texture-space OIT with refractions and scattering ▪ Not perfect, but one step closer ▪ Denoising techniques don’t work so well with transparency (and 1-2 spp) Transparency in the Maxwell Renderer S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
  • 113. Partial Coverage ▪ Foliage ▪ Can still do alpha test (i.e.: any-hit) ▪ Animated becomes a real problem ▪ Defocus effects such as motion blur and depth of field are still intractable ▪ Need partial coverage denoising @ 1-2 spp PBRT (Matt Pharr, Wenzel Jakob, Greg Humphreys) Disney / Pixar S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
  • 114. Specialized Denoising ▪ PICA PICA: specialized denoising & reconstruction for view-dependent, light-dependent or any other term you can monitor variance across frames [Stachowiak2018] ▪ Can use general denoising on the whole image, but… ▪ Denoising filter around a specific term will achieve greater quality ▪ Potentially converges faster ▪ One algorithm to rule-them-all? ▪ Lately… ▪ Gradient Estimation for Real-Time Adaptive Temporal Filtering [Schied 2018] ▪ Lots of progress with DL approaches S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
  • 115. Geometry++ ▪ Ray-Triangle intersection is the fast path ▪ Well-known problem [Möller 1997] [Baldwin 2016] ▪ Intersection of procedural geometry currently done via Intersection Shaders ▪ Allows custom/parametric intersection code, for arbitrary shapes ▪ Current (slow) path for procedural geometry ▪ Procedural: hair, water, and particles are still a challenge ▪ Hair: all done parametrically via intersection shaders? ▪ Water: can be done via triangles, but reflections + refractions are challenging ▪ Also, handle view-dependent LODing (ie.: water patches) vs ray-dependent LODing? ▪ DXR: Need ways to accelerate for non-obvious primitives ▪ Cone-BVH for ray-marching? (à-la-Aaltonen in Claybook) S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
  • 116. Taking Advantage of DXR ▪ DirectX offers easy interoperability between raster, compute and raytracing ▪ Raytracing, rasterization and compute shaders can share code & types ▪ Evaluate your actual HLSL material shaders - directly usable for a hybrid raytracing pipeline ▪ The output from one stage can feed data for another ▪ Write to UAVs, read in next stage ▪ Prepare rays to be launched, and trace on another (i.e.: mGPU) ▪ Interop extends opportunities for solving new sparse problems ▪ Use the power of each stage to your advantage ▪ RT opens the door to an entirely new class of techniques for games ▪ DXR provides a playground where research & gamedevs can collaborate! S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
  • 117. Performance (1/) ▪ Best tracing performance comes from one massive static BVH? ▪ Viable for a big static scene, but most games are not static… ▪ Animated Geometry: Acceleration Structure construction is fairly fast ▪ But not fast enough to run vertex shaders for all the animated geometry and build AS from scratch every frame ▪ Top Level: has to be rebuilt whenever anything interesting happens ▪ Animated objects, objects inserted/removed, LOD swap ▪ LOD selection/management and how that ties to Shader Table management ▪ Bottom Level: is there a sweet spot? ▪ Lots of individual rigid objects: might have to bake transforms and merge into larger ones ▪ How much memory can you spare for better tracing performance? S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
  • 118. Performance (2/) ▪ 2+ levels hierarchies? ▪ + : Allow to split worlds into update frequencies, and avoid updates if nothing has changed ▪ - : Tracking the object-to-world transforms becomes difficult: ray needs a transform stack ▪ - : Apps can end up creating a large instance hierarchy of single-node trees → bad  ▪ Idea: have two bottom level BVH's, one for static and one for dynamic geo? ▪ Static: Built & left alone, traverse it first ▪ Dynamic: Rebuilt in the background as a long running task. Refitted every frame ▪ + Supports many animated characters ▪ While not taking a hit for rebuilding BVH for static geometry ▪ - Doesn’t solve massive number of materials that games typically have… S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
  • 119. Performance (3/) ▪ Example: 1K materials, fire 100k secondary rays ▪ (Potentially) need to run 1k shaders each on a batch of 100 hits. Viable workload? ▪ Can’t just shove all material variations in the SBT and let IHVs optimize, right? ▪ How aggressively do shader variations / materials in game scenes need to be merged? 1 or 2 orders of magnitude? ▪ Uber Shaders + Texture Space Shading? ▪ PICA PICA Uber Shaders: Rebinding constants quite frequently was not an issue ▪ Use texture-space gbuffers, or is that too much memory? S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
  • 120. Future Ray Tracing Research ▪ RTRT literature has focused on accelerating “correct” raytracing ▪ Lots new challenges with game ray tracing ▪ Literature has to adjust to game ray tracing constraints ▪ Games → budgeted amount rays/frame, rays/pixel, fixed frame times & memory budgets ▪ Games → Light transport caches: surfels, voxels, lightmaps ▪ Q: How many rays-per-pixel needed to have real-time perfect indirect diffuse? ▪ Q: What has to happen to get fully robust PBR real-time ray traced lighting? ▪ Perf R&D: Efficient sampling / integration strategies & reconstruction filtering ▪ Perf R&D: Caching of materials and partial solutions S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
  • 121. Global Illumination ▪ Open problem even in offline rendering ▪ Variance still too high ▪ Can reduce frequency ▪ Prefiltering, path-space filtering ▪ Denoising & reconstruction ▪ Pinhole GI & lighting is not solved ▪ Incoherent shading → intractable performance ▪ Have to resort to caching to amortize shading ▪ PICA PICA: caching of GI via surfels ▪ Some issues: only spawn surfels from what you see ▪ Need to solve GI for user-generated content → [Ritchell 2011] S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
  • 122. Summary ▪ Modern graphics abstractions are important for efficiency without verbosity ▪ Real-time ray tracing significantly improves visuals, but still have a lots to do to bridge offline and real-time ▪ DXR provides a playground where research and gamedevs can collaborate! S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
  • 123. References S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing ▪ [Andersson 2018] Johan Andersson, Colin Barré-Brisebois.“DirectX: Evolving Microsoft's Graphics Platform”. available online ▪ [McGuire 2017] McGuire, Morgan and Mara, Michael. “Phenomenological Transparency”. available online ▪ [O’Donnell 2017] Yuriy O’Donnell. “Frame Graph: Extensible Rendering Architecture in Frostbite”. available online ▪ [Schied 2017] Schied, Christoph et. al. “Spatiotemporal Variance-Guided Filtering: Real-Time Reconstruction for Path- Traced Global Illumination”. available online ▪ [Stachowiak 2018] Tomasz Stachowiak. “Towards Effortless Photorealism Through Real-Time Raytracing” available online ▪ [Wihlidal 2018] Graham Wihlidal. “Halcyon + Vulkan”. available online
  • 124. Thanks ▪ Matthäus Chajdas ▪ Rys Sommefeldt ▪ Timothy Lottes ▪ Tobias Hector ▪ Neil Henning ▪ John Kessenich ▪ Hai Nguyen ▪ Nuno Subtil ▪ Adam Sawicki ▪ Alon Or-bach ▪ Baldur Karlsson ▪ Cort Stratton ▪ Mathias Schott ▪ Rolando Caloca ▪ Sebastian Aaltonen ▪ Hans-Kristian Arntzen ▪ Yuriy O’Donnell ▪ Arseny Kapoulkine ▪ Tex Riddell ▪ Marcelo Lopez Ruiz ▪ Lei Zhang ▪ Greg Roth ▪ Noah Fredriks ▪ Qun Lin ▪ Ehsan Nasiri, ▪ Steven Perron ▪ Alan Baker ▪ Diego Novillo ▪ Tomasz Stachowiak
  • 125. Thanks ▪ SEED ▪ Johan Andersson ▪ Jasper Bekkers ▪ Joakim Bergdahl ▪ Ken Brown ▪ Dean Calver ▪ Dirk de la Hunt ▪ Jenna Frisk ▪ Paul Greveson ▪ Henrik Halen ▪ Effeli Holst ▪ Andrew Lauritzen ▪ Magnus Nordin ▪ Niklas Nummelin ▪ Anastasia Opara ▪ Kristoffer Sjöö ▪ Ida Winterhaven ▪ Tomasz Stachowiak ▪ Microsoft ▪ Chas Boyd ▪ Ivan Nevraev ▪ Amar Patel ▪ Matt Sandy ▪ NVIDIA ▪ Tomas Akenine-Möller ▪ Nir Benty ▪ Jiho Choi ▪ Peter Harrison ▪ Alex Hyder ▪ Jon Jansen ▪ Aaron Lefohn ▪ Ignacio Llamas ▪ Henry Moreton ▪ Martin Stich
  • 126. S E E D / / S E A R C H F O R E X T R A O R D I N A R Y E X P E R I E N C E S D I V I S I O N S T O C K H O L M – L O S A N G E L E S – M O N T R É A L – R E M O T E S E E D . E A . C O M W E ‘ R E H I R I N G !