SIGGRAPH 2018 - Full Rays Ahead! From Raster to Real-Time Raytracing

Full Rays Ahead!
From Raster to Real-Time Raytracing
Colin Barré-Brisebois - @ZigguratVertigo
SEED – Electronic Arts

SEED // Full Rays Ahead! From Raster to Real-Time Raytracing
Who Am I?
 Senior Rendering Engineer at SEED - EA
 Cross-discipline pioneering group within Electronic Arts
 Combining creativity with applied research
 Exploring the future of interactive entertainment with the goal to
enable anyone to create their own games and experiences
 Offices in Stockholm, Los Angeles and Montréal
 Previously
 Principal Rendering Engineer / Technical Director at WB Games Montréal
 Rendering Engineer at EA Montréal and DICE

PICA PICA
 Exploratory mini-game & world
 For self-learning AI agents to play,
not humans 
 “Imitation Learning with Concurrent
Actions in 3D Games” [Harmer 2018]
 SEED’s Halcyon R&D framework
 60 FPS @ 1080p on a TitanV
 Goals
 Explore hybrid rendering with DXR
 Clean and consistent visuals
 Procedural worlds [Opara 2018]
 No precomputation

Hybrid Rendering Pipeline
Direct shadows
(raytrace or raster)
Lighting
(compute + raytrace)
Reflections
(raytrace or compute)
Deferred shading
(raster)
Global Illumination
(compute and raytrace)
Post processing
(compute)
Transparency & Translucency
(raytrace and compute)
Ambient occlusion
(raytrace or compute)

Goals
 Tips-and-Tricks on bringing your engine up with DXR
 Bringing some of the core techniques to production
 Some challenges you might have to face

I’m a dev and I want to move to DXR…
What should I do?

Preface
 Transition to DXR is not automagical
 Some effects are easy to add: hard shadows/reflections, ambient occlusion
 The fun starts when things get blurry & soft 
 Can’t just implement offline raytracing techniques into real-time
 Most of the literature assumes offline – we have work to do here
 Embrace the transition, and filtering! 
 DXR is pretty intuitive!
 Nice evolution from previous raster + compute pipelines
 Easy to get quickly up and improve!
 Having a good DX12 implementation will make your job easier!

Some warm-up might be required…

Warm-up for DXR
 Brush-up raytracing & DXR work out there!
 This course, NV’s Falcor, MS’ DirectXSamples, our GDC/DD talks 
 Check Matt Pharr’s Adopting Lessons From Offline Ray Tracing to Real-Time Ray Tracing
 Break down passes so you can easily swap & reuse!
 HLSL makes it easy: DXR interops with Rasterization and Compute
 Build shared functions that you will call from both Rasterization and Compute
 Make sure your engine can quickly toggle between techniques
 Will be essential to compare when optimizing, cutting corners, choosing ray counts
 (“Bonus”): Build a reference pathtracer in engine
 Prepare your passes for the transition: swapping inputs & outputs
 Start thinking about how to handle noise (TAA and other filtering)

First Thing
 A few techniques should be implemented first (in difficulty order)
 Shadows
 Ambient Occlusion
 Reflections

Shadows
 Launch a ray towards light
 Ray misses  Not in shadow
 RAY_FLAG_SKIP_CLOSEST_HIT_SHADER
 Handled by Miss Shader
 Shadowed = !payload.miss;
 Soft shadows?
 Random direction from cone [PBRT]
 Cone width drives penumbra
 [1;N] rays & filtering
 We used SVGF [Schied 2017]
 Temporal accumulation
 Multi-pass weighted blur
 Variance-driven kernel size

AO
 Random cosine hemi sampling
 Launch from g-buffer normals
 AO = payload.miss ? 1.0 : 0.0
Integral of the visibility function
over the hemisphere Ω for the
point p on a surface with normal
n with respect to the projected
solid angle 𝑛
𝑝
𝐴 𝑝 =
1
𝜋 Ω
𝑉𝑝, 𝑤( 𝑛 ⋅ 𝑤)𝑑𝜔

Reflections
 Launch rays from G-Buffer
 Trace at half resolution
 ¼ ray/pixel for reflection
 ¼ ray/pixel for reflected shadow
 Reconstruct at full resolution
 Also supports:
 Arbitrary normals
 Spatially-varying roughness
 Extended info:
 GDC 2018 & DD 2018

Reflection Pipeline
Importance
sampling
Screen-space
reflection
Raytracing
Spatial
reconstruction
Envmap gap fill
Bilateral cleanup
Temporal
accumulation

Raw Raytrace Output Raytracing
Importance
sampling

Spatial Reconstruction Spatial
reconstruction

+Temporal Accumulation Temporal
accumulation

+Bilateral cleanup Bilateral cleanup

Validate Against Ground Truth!
 Validating against ground truth is key when building RTRT!
 Toggle between hybrid and path-tracer when working on a feature
 Rapidly compare results against ground truth
 Toggle between non-RT techniques and RT
 i.e.: SSR  RT reflections, SSAO  RTAO
 Check performance & check quality (and where you can cut corners)
 PICA PICA: used constantly during production
 Multi-layer material & specular, RTAO vs SSAO, Surfel GI vs path-traced GI
 No additional maintenance required between shared code
 Because of interop!

Pathtracing ~15 seconds accumulation

Take Advantage of Interop
 DirectX offers easy interoperability between raster, compute and raytracing
 Raytracing, rasterization and compute shaders can share code & types
 Evaluate your actual HLSL material shaders - directly usable for a hybrid raytracing pipeline
 The output from one stage can feed data for another
 i.e.: Write to UAVs, read in next stage
 i.e.: Prepare rays to be launched, and trace on another (i.e.: mGPU)
 i.e.: Can update Shader Table from the GPU
 Interop extends opportunities for solving new sparse problems
 Interleave raster, compute and raytracing
 Use the power of each stage to your advantage
 Interop will become your new best friend as we move towards this transition

Tips-and-Tricks
 Different payloads for different problems
 Make your rays lean-and-mean
 Pack your payload with get()/set() functions
 Embrace lean recursiveness
 Minimize the data you carry
 Often don’t need infinite recursiveness
 Take advantage of Shader Spaces
 Not mandatory, but can make your life easier
 For frequency updates
 Space 0: default space, object static data
 Space 1: RT space, instance data

Speaking of Rays…
 Handling coherency is key for RTRT performance
 Coherent  adjacent work performing similar operations & memory access
 Camera rays, texture-space shading
 Incoherent  trash caches, kills performance
 Reflection, shadows, refraction, Monte Carlo
 Use rays sparingly
 Trace only where necessary
 Tune ray count to importance
 Adaptive techniques
 Reconstruct & denoise
 Reuse results from spatial and temporal domains

Texture Level-of-Detail
What about texture level of detail?
 Mipmapping [Williams 1983] is the standard method to avoid texture aliasing:
 Screen-space pixel maps to approximately one texel in the mipmap hierarchy
 Supported by all GPUs for rasterization via shading quad and derivatives
Left: level-of-detail (λ), partial derivatives and the parallelogram-approximated texture-space footprint of a pixel. Right: mipmap chain

No shading quads for ray tracing!
 Traditionally: Ray Differentials
 Estimates the footprint of a pixel by
computing world-space derivatives of the
ray with respect to the image plane
 Have to differentiate (virtual offset) rays
 Heavier payload (12 floats) for subsequent
rays (can) affect performance. Optimize!
 Alternative: always sample mip 0 with bilinear filtering (with extra samples)
 Leads to aliasing and additional performance cost
Ray Differentials [Igehy99]
Surface
P(x + ∂x)
D(x + ∂x)P(x)
∂D(x)
∂P(x)
D(x)
D(x)
x
y
R(x)
R(x + ∂x)
∂x

Together with Research, we
developed a texture LOD technique for ray tracing:
 Heuristic based on triangle properties, a curvature
estimate, distance, and incident angle
 Similar quality to ray differentials with single trilinear lookup
 Single value stored in the payload
 Upcoming publication in Ray Tracing Gems
 Tomas Akenine-Möller (NV), Jim Nilsson (NV), Magnus
Andersson (NV), Colin Barré-Brisebois (EA), Robert Toth
 Barely scratched the surface – still work to do!
 Preprint: https://t.co/opJPBiZ6au

Open Problems
 Noise vs Ghosting vs Performance
 Managing Coherency & Ray Batches
 Transparency & Procedural Geometry
 Specialized denoising & reconstruction
 Real-Time Global Illumination
 DXR’s Top & Bottom Accel  best for RTRT?
 Managing Animations
 New Hybrid rendering approaches?
 Texture LOD?
Buckle-up, we have work to do… and this is awesome! 

S E E D // Towards effortless photorealism through real-time raytracing
Summary
 Real-time visuals with (almost) path traced quality
 Achievable today on consumer hardware
 RTRT opens the door to an entirely new class of techniques for games
 Solve sparse and incoherent problems
 And now with raytracing hardware, it’s getting really fun! 
 Tomorrow: will talk about PICA PICA performance on NVIDIA Turing
 Still… we’re at the start of the journey
 Plenty of research opportunities
 Looking forward to see how you will take advantage of raytracing for your work!

PICA PICA ASSETS
Now available on Sketchfab & SEED’s website. Free use for your R&D & papers!
glTF + FBX + textures. License: CC BY-NC 4.0  https://skfb.ly/6AJCp

Thanks
 Chris Wyman (NV)
 SEED PICA PICA Team
 Alex Hyder (NV)
 Aaron Lefohn (NV)
 Ignacio Llamas (NV)
 Martin Stich (NV)
S E E D
S E A R C H F O R E X T R A O R D I N A R Y E X P E R I E N C E S D I V I S I O N
S T O C K H O L M – L O S A N G E L E S – M O N T R É A L – R E M O T E
W W W . E A . C O M / S E E D
W E ‘ R E H I R I N G !

Questions?

SIGGRAPH 2018 - Full Rays Ahead! From Raster to Real-Time Raytracing

More Related Content

What's hot (20)

Similar to SIGGRAPH 2018 - Full Rays Ahead! From Raster to Real-Time Raytracing (20)

More from Electronic Arts / DICE (18)

Recently uploaded (20)

SIGGRAPH 2018 - Full Rays Ahead! From Raster to Real-Time Raytracing

Editor's Notes