Real-time CG animation in Unity: unpacking the Sherman project - Unite Copenhagen 2019
4 likes1,939 views
This document summarizes the animated short film "Sherman" which was created entirely with Unity. It discusses how the film was blocked out and animated without storyboards. Advanced animation techniques like Alembic were used. Custom fur and visual effects were also implemented. Lighting was achieved through light probes, reflection probes, and baked global illumination. The project structure and Unity's capabilities for film production are discussed, as well as future directions like modular projects and Addressables.
Real-time CG animation in Unity: unpacking the Sherman project - Unite Copenhagen 2019
- 1. Sherman An animated short film Made with Unity
- 2. Real-time CG animation in Unity: unpacking the Sherman project Mike Wuetherick Head of Tech M&E Innovation Group Unity Technologies @gekido
- 3. Topics Sherman ● Key Stats ● Blocking out the Short ● Lookdev in HDRP ● Camera Layout with Cinemachine ● Advanced animation with Alembic ● Lighting for Linear animation ● Fur & VFX ● Evolution of a Unity Project
- 5. Sherman – Key Stats ● No storyboards, straight into 3D to ‘find the story’ ● Small core team ○ 1 1/2 animators, 1 modeler, 1 graphics engineer, prod ● Unity Technology used: ○ Timeline, Cinemachine, Alembic, HD Render Pipeline, Post Processing, Visual FX Graph, Shader Graph, Unity Recorder, Progressive Lightmapper, FBX Exporter ● Concept to Final frames with Unity 2018.4 ● Fur experimental implementation, designed for ‘Offline Render Quality’
- 6. On The Shoulders of Giants Marza Animation Planet + Unity Japan - The Gift https://www.youtube.com/watch?v=tVHH3-bP-fE
- 7. Blocking out the Short
- 8. Blocking out the Short
- 10. LookDev with HDRP High fur quality bar Inspired by Dreamworks - Bilby, Disneyland Commercial
- 11. LookDev with HDRP Steven Shmuely - Modeler ● Simple ENV/ low asset count to ensure Quality ● Characters Modeled in Maya/ 3ds Max ● Surfaced in Substance Painter ● *Some ENV assets reused from Book of the Dead* ○ Styled to better fit the film
- 12. LookDev with HDRP ● Optimizing for Quality, not Performance ● Developed a detail settings system ○ Switch from fast working environment to highest quality rendered output ● Global Post Processing Profile - Overall Color Grading ● CM Shot per shot post controls - Exposure, DoF, Vignette, etc. ● CM Storyboard feature - Splitview to view Visual Target over Rendered Image
- 13. Steven Shmuely - Lone Modeler ● Rendered turntables in Unity for review ● Most assets use the HD/Lit Shader
- 14. Custom Materials - ShaderGraph
- 15. Camera Layout
- 16. Camera Layout with Cinemachine Cinemachine used to layout all cameras FBX exporter to pull cameras from Unity to Maya for Animators Initially shot wide to allow Bryan room to explore, CM used to refine shots
- 17. Camera Layout with Cinemachine Experimenting with new cameras via Timeline after Anim complete to push cinematic quality Combination of CM clips and Animation tracks to animate Cameras - quick exploration
- 18. Animation
- 19. Advanced Animation with Alembic Baymax Dreams used FBX file as primary format for asset transfer, wanted to explore alternatives FBX Limitations (at the time) - Skin weighting limitation prior to Unity 2019.x - Sherman created with Unity 2018.3 - each vertex weighted to 4 bone maximum - 2019.x now supports 256 bones/vertex - Scale Compensation (squash and stretch)
- 20. Squash and Stretch Squash and Stretch especially important for high quality animation Option: ● Flatten Rig Hierarchy (bag of bones) Challenge: ● Character is difficult to manage during previz Solution : Alembic *Alembic Package out of preview in 2019.1, now fully supported
- 21. Alembic Challenges File Size / Performance ● Each frame of anim is a baked snapshot of the vertex positions for the model. Higher poly mesh/ Longer animation = Big File size ● On Sherman - character animation data roughly 7Gb ( 12Gb total project size ) ● Alembic streams from disk – performance challenges Attaching Objects ● Attaching Lights/ Reflection probes and CM cameras not easily achieved ● Skeleton data not included by default – possible to export lets you get full skeleton rig – careful of super heavy nested game object hierarchies!
- 22. Alembic Challenges Material Management ● Material Definitions not stored needed to be reassigned with every new export ● Remapping materials with dozens of alembic clips is time consuming Wrote several tools to simplify - available in the Film/TV Toolbox package
- 23. USD and the Future Pixar’s open source USD format gaining traction with studios & tool vendors First version of Unity USD support available via Package Manager, new dedicated team!
- 24. Lighting
- 25. Lighting Strategies for Linear Content Lighting Supervisor - Jean-Philippe (JP) Leroux. Real Time lighting for Sherman achieved through: ● Light Probes ● Reflection Probes ● Cascade Shadow Maps ● Baked Global Illumination
- 26. Global Illumination Indirect lighting precalculated for high quality localized ambient lighting - Static objects are marked for lightmapping - Used progressive lightmapper Baked Indirect
- 27. Light Probe Array ● Small objects lit dynamically with a probe array ● Larger objects, not lightmapped - more refined probe lighting through Proxy volumes ● Did not bake lights, only the sky and bounce of directional lighting
- 28. Reflections Spherical Real-Time reflection probe Planar Real-Time reflection probe
- 29. Lights With Probes & Baked GI – all lighting is dynamic Light prefabs with a pivot offset – easily orbit objects
- 32. Fur & VFX
- 33. Fur & VFX
- 34. Visual Effects Graph ● Effects Artist : Vlad Neykov ● Node-based visual effects tool ● Available in Preview for HDRP ● GPU-accelerated effects
- 37. Fur
- 40. Authored ‘Strand’ patch, baked to SDF
- 43. Fur in Sherman ● Experimental implementation using 18.4 HDRP ● Shell-based approach ○ Multiple-SDF layers ● Shading uses HDRP ‘Hair’ master node Future Work: ● Updating tech to 19.x ● Collaboration with Graphics R&D on ‘proper’ Fur implementation ● Authoring / Groom tools in-editor
- 44. Other Tools & Features
- 45. Rendering ● Unity Recorder package ● Timeline integration ● Renders at 4k ● Dedicated ‘Render Farm’ of 3 consumer PC’s ● On demand, anyone can remote in to start renders ● Nightly renders for review
- 46. Filmic Motion Blur/ Super Sampling ● Initially developed for Baymax Dreams final QC ● Super Sampling ‘Offline” renderer ● Also used for final Fur Converge ● Renders at 960 fps (24 fps * 40 samples) ● Final frames @ 4k ○ roughly 5 FPM - full render of the short at max quality in roughly 9 hours ● Future work: stochastic accumulation rendering ● opens possibility for DOF / higher quality super sampled rendering for offline use cases
- 47. Render Window ● Allowed ‘final quality’ render of frames on demand ● Improved Iteration times for final Fur Groom / Lighting – on-demand convergence for Fur
- 48. Sherman – Key Stats ● No storyboards, straight into 3D to ‘find the story’ ● Unity Technology used: ○ Timeline, Cinemachine, Alembic, HD Render Pipeline, Post Processing, Visual FX Graph, Shader Graph, Unity Recorder, Progressive Lightmapper, FBX Exporter ● Concept to Final frames with Unity 2018.4 ● Custom Fur implementation, designed for ‘Offline Render Quality’ ● Project available (including all custom package) for everyone!
- 49. Evolution of a Unity project
- 50. Vestibulum congueAssembly Content ● 3d models ● Animations ● Textures ● Audio Code ● Unity packages ● Custom tools ● Workflow utilities ● Internal code libraries Assembly ● Scenes ● Timeline sequences ● Configuration files ● Build / platform info Project Composition
- 51. ‘Classic’ project structure - pre 2018.x Unity - Monolithic ‘Unity’ runtime / editor - Unity project, contains all: - User code - Assets - Configuration info - Monolithic Unity runtime Unity Editor Unity Project Code Content Custom Code 3rd party tools Models Anims Audio Textures Asset store content Unity Runtime Config Scenes Config files Build info
- 52. ‘Modern’ project structure - 2018.x Unity - Modular Unity Editor - Package-based feature delivery - Features migrated into c# ‘user space’ - eg. Scriptable Renderer - Unity project, contains all: - User code - Assets - Configuration info - Modular Unity runtime - Code stripping of unwanted features Unity Editor Unity Runtime Package Manager Modular Runtime Unity Project Code Content Custom Code 3rd party tools Models Anims Audio Textures Asset store content Config Scenes Config files Build info
- 53. Unity Editor Package Manager Content Custom Packages (scoped registries) ‘Near-future’ project structure - 2019.x Unity - Modular Unity Editor - Package-based feature delivery - Features migrated into c# ‘user space’ - eg. Scriptable Renderer - Asset store / 3rd party Content & Code - Unity project, only contains: - Configuration info - Custom user package delivery - Code - Content - Modular Unity runtime - Code stripping of unwanted features Unity Runtime Modular Runtime Unity Project Code Content Custom Code Models Anims Audio Textures Asset store content Config Scenes Config files Build info Code Unity features 3rd party tools Shared Libraries
- 54. Episodic Project structure Unity Editor Package Manager Content Asset store content Code Unity features 3rd party tools Lookdev Unity Project Config Scenes (?) Config files Build info Episode / Shot Assembly Unity Project Config Scenes (?) Config files Build info Custom in-house Packages Code Content Custom Code Models Anims Audio Textures Shared Libraries
- 55. Unity for Large-scale productions Unity Editor Package Manager Content Custom in-house Packages Unity Project Code Content Custom Code Models Anims Audio Textures Asset store content Config Scenes (?) Config files Build info Code Unity features 3rd party tools Shared Libraries Unity Project Config Scenes (?) Config files Build info Unity Project Config Scenes (?) Config files Build info Unity Project Config Scenes (?) Config files Build info Unity Project Config Scenes (?) Config files Build info Unity Project Config Scenes (?) Config files Build info Unity Project Config Scenes (?) Config files Build info Unity Project Config Scenes (?) Config files Build info Episodes (TV Production) Scenes (feature film)
- 56. Modular Projects Advantages ● Explicit versioning of code & assets ● Centralized management ● Access control (Scoped Registries) ● Coordination across multiple projects ○ Multiple scenes / episodes in a TV series or movie for example ● Integration with asset pipeline tools: ○ Shotgun, FTrack etc ● Common libraries, tools & code across multiple projects ● Reusable content (asset libraries) Disadvantages ● No documentation for how to set this up ‘out of the box’ ● Content in ‘package’ format still requires lengthy import times for first load ○ Cache server can be used to ‘pre-warm’ assets via a build machine ● Limited documentation to setup on-prem / custom package registry ● 3rd party Registry options (MyGet / GitHub) ● No tooling to easily create custom packages is, requires advanced technical knowledge
- 58. Addressables & Cloud Projects - The vast majority (ie almost all) assets are static (are not modified by the Unity user) during the assembly stage of a project - Look dev / asset prep occurs with ‘source’ asset, which is published as an addressable asset - Eliminates the need to ‘warm’ or preload assets for fast project loading - Assets should be able to be loaded immediately without ‘import’ times - Needs custom workflow tooling to implement currently
- 59. Addressables Architecture Lookdev Unity Project Config Scenes (?) Config files Build info Episode / Shot Unity Project Config Scenes (?) Config files Build infoAddressable Content Hosting Service Asset Database Artists publish asset versions to Asset Database Episode / Assembly project references latest approved version of asset
- 60. For more info ● Sherman Blog (2-part): https://blogs.unity3d.com/2019/06/11/introducing- sherman-part-1/ ● Unity Film Solutions Page: https://unity.com/solutions/film-animation-cinematics ● Other Unity Demos / Examples: https://unity3d.com/unity/demos ● Made with Unity: www.unity.com/madewith Unity for Look Development: https://youtu.be/urew479-Wlw Unity for Lighting & Compositing: https://youtu.be/0H3TUnUfAIU Unity for Layout: https://youtu.be/qZEXIXYOcoM
- 61. Thank you! Mike Wuetherick Head of Tech M&E Innovation Group Unity Technologies @gekido