Using Umbra Spatial Data for Visibility and Audio Propagation in Quantum Break

Editor's Notes

• #3: Hi, Thank you for coming to our session, I know there’s a lot of sessions out there and that you are busy, and that expensive All Access Pass is better served by going to all the lectures around, so thank you for attending. We are going present a packed 30 minutes for you. With me on the stage today is mr Jan Benes, the audio programmer from Remedy from Finland and he is going to providing the interesting part of this presentation. The point of our talk, the takeaway if you will, is that Umbra and the spatial data it generates, is useful for many things in games and even outside of games. Specifically we are going to concentrate on how Remedy is leveraging our tech in Quantum Break for not just visibility but audio propagation. What are not going to do is going into detail about the inner workings of Umbra, half an hour is not enough for that.
• #5: The core of what our tech does is occlusion culling, that’s why you are going to be licensing it. Our tech does that really well. I dare say that. The footage is captured from the Umbra debugger which is our brand new tool that ships with the Umbra SDK. It’s something you can use to evaluate Umbra as you can import your content into it, but it’s main task is to help visualize how Umbra works with your data Umbra Cloud The Umbra Cloud performs the geometry processing in the cloud instead of locally, and thus offers near-unlimited computation and storage capabilities. Cloud integration improves usability and improves content iteration times and productivity. Umbra Cloud generates the database in the cloud, and thus offers near-unlimited computation and storage capabilities.  
• #6: Here you see the data computation working and you can do that in the cloud now. Umbra Debugger The Umbra Debugger is part of the SDK and is a tool for inspecting, visualizing and debugging Umbra’s behavior in the user’s worlds. The Debugger visualizes the effect of various parameters. It’s also a valuable tool for getting started with the evaluation and integration process. The Debugger can be used to iterate the scene geometry export and to find the right set of computation parameters without having to implement a full runtime integration into the user’s actual renderer.
• #7: Umbra has extremely accurate information about space, we have spatial connectivity data Which is very useful for many things and many of our licensees use our tech in various ways...
• #8: The games massive game worlds are split into multiple blocks which are dynamically streamed in the game. Umbra supports this seamlessly, as Umbra’s data is also streamable. They also use Umbra’s built-in LOD culling support to automatically determine the visibility of different LOD levels of their objects As well as accelerate their shadow mapping by using Umbra to cull shadow casters that would cast occluded shadows.
• #9: Visibility query is cheap to begin with, you can split into multiple jobs running in parallel.
• #10: Destiny uses Umbra’s spatial data to split the world into various logical sections. They then also use this spatial data for various purposes, like streaming, AI and audio. Destiny also uses Umbra’s unique “predicted camera” feature, which allows them to launch the visibility query very early on in the frame - even before the camera location is accurately known - and still get correct results.
• #11: Today we are going to talk about Quantum Break from Remedy and how Umbra is used in this upcoming Xbox one exclusive, which is the most expensive finnish cultural production in our nations history
• #12: So Jan, take it away...
• #14: Making games for almost two decades, building in-house tech for almost two decades.
• #15: The current version of the Remedy game engine is an evolution of the Alan Wake tech, which means it’s still a streaming-heavy open world engine. For the purpose of the talk let’s mention game cells, a 128x128 m grid for streaming static data. Key middleware for the different subsystems - the middleware data need to support streaming as well. Renderer: deferred since AW.
• #16: The engine needs to cope with big levels, varying environments and a lot of detail. In many levels you can experience big dynamic changes during the playthrough, so static data need to be built for all the versions (permutations). We’re trying to support fast iterations in all pripelines and make the generation of middleware data transparent to the users as much as possible.
• #17: Umbra helps a lot with occlusion culling, which is obviously its main purpose. We also found that Umbra provides a useful toolset for our ‘slightly more advanced’ 3D audio systems.
• #18: Scene hierarchy and raycasts are pretty useful for audio, bi-directional visibility improves occlusion culling. Streaming support is a clear requirement given by the engine architecture.
• #19: Mostly ‘standard’ approach for occlusion culling (see also The Witcher 3 and Destiny talks). Queries run on a job thread so they don’t block the renderer. Bi-directional visibility is a nice optimization that Umbra implemented for us, it combines dynamic occlusion buffer for the camera with occlusion buffers baked during export for individual objects.
• #20: Building from scratch can take a while (but bear in mind that in this example it means building three versions of the level). Usually this will be handled by a build server anyway. Iterations are more important and quite quick, we only rebuild data for modified geometry (at the scope of game cells, see slide 14).
• #21: The big challenge for sound design in Quantum Break is building two completely different and contrasting worlds (normal world – realism meets art, players should feel the locations; stutter world – out of time, surreal, broken). On the engine side, we’re exposing as much state information as possible to Wwise to give sound designers control and independence. Using runtime effects helps with iterations (live editing) and simplifies pipelines as there’s no need to bake anything to the data. Probably wouldn’t be possible on previous generation hardware. Grain synth plugin allows us to dynamically play sounds forwards or backwards and freeze them when needed. Signal analysis plugin sends a set of values to the renderer so we can for example emit particles of shake geometry by playing specific sounds.
• #22: Sound environments mark areas with different effects, on the Wwise side this is implemented using auxiliary sends – splitting the signal between a main dry ‘route’ and possibly several effect buses. We’re using Umbra data as scene subdivision, to have a quick access to the effects affecting given position in space. The scope is Umbra clusters, convex areas of ‘reasonable size’.
• #24: Also there’s blend time in Wwise to smooth the transitions.
• #25: A lot of raycasts, a lot of sound sources. Note that also bullet hits or collision sounds are actual 3D sounds that need to be processed by the system.
• #26: Slowly getting to sound propagation, so let’s introduce this feature. Funny fact: the audio occluders was the first use case/implementation we did. Now we also have the generic gates in place. Most common example for these is doors, which block both audio and visibility as long as they are closed. Placing and scripting the gates means some ammount of manual editing, but if often comes for free, for example gates in door prefabs.
• #27: Sound propagation system kicks in when a sound source is completely obstructed and when there is at least one gate between the source and listener (camera). The expected result is: muffled sound using effects and filters, hearing it coming through the gate. Set of parameters is sent to Wwise for each sound source to give the information about occlusion, propagation and some specific details.
• #28: What’s happening undr the hood - one more important step that’s worth mentioning: when the environment volumes load, they run a ‘query connected region’ search from their center point, which marks all reachable clusters in their volume (it won’t bleed through walls). This step also discovers the connecting gates. Virtual positions and attenuations – keeping distance is quite important as we usually drive quite a few parameters based on the attenuation (volume, low and high pass filters, reverb sends, …). Following video shows the occlusion raycasts and a few different cases of the sound propagation (with a commentary).