From AI-Generated Stories to Interactive Volumetric Content by Adam Myhill, Unity | SIC 2022
Download as PPTX, PDF0 likes129 views
Adam Myhill discusses the potential of the metaverse and the challenges in content creation, focusing on the integration of AI-generated stories and interactive content. His company, Unity Technologies, aims to revolutionize fan engagement in sports through 3D volumetric capture and dynamic user experiences. The document emphasizes the importance of user experience in developing compelling, high-quality content in the metaverse, highlighting the need for innovative solutions to create personalized experiences.
From AI-Generated Stories to Interactive Volumetric Content by Adam Myhill, Unity | SIC 2022
- 1. Building for the Metaverse From AI-generated stories to interactive volumetric content Adam Myhill, Creative Director @ Unity Technologies Date: Wednesday October 26th Time: 11:00 AM - 11:40 AM Location: Room 1 - Upstairs @adamjcmyhill
- 2. The Metaverse has a content creation problem
- 3. Adam Myhill 20 years in video games 5 years in film as a DP Created Cinemachine, sold to Unity Created Cinecast, Metacast 7 patents in camera, UX, streaming, volumetric file formats Technical Emmy Creative Director of Metacast / Sports @ Unity Technologies Investor and Advisor to Transitional Forms Owner of 3 alpacas
- 4. What even is the metaverse?
- 10. Adam’s hot take on the Metaverse A persistent shared environment On a decentralized open standard Proof of Position + Spatial Contracts Dynamic subscription to content layers - modified by time, position, biometrics While purely creative sub-worlds will be cool (pick the blue or red door) The data-overlaid layers of content over IRL will be where it really takes hold
- 12. An incredible amount of content to be made
- 13. CREATE + PRESENT How do we make systems that Compelling, high quality experiences? (when you’re not there to do it)
- 15. Transitional Forms - leaders in AI generated content
- 16. WE’RE GIVING CONTENT A MIND OF ITS OWN.
- 18. 18 WE’VE BUILT A REAL-TIME SHOW PLATFORM WITH MASSIVE POTENTIAL. Our Transforms TV prototype is already proving that Artificial Intelligence is a viable creative partner for real- time content through: ★ Live human input ★ Autonomous content generation ★ Dynamic scene assembly ★ Real-time effects Prototype Reel (1min)
- 19. Prompt Art
- 20. Prompt Story
- 21. Prompts Show!
- 28. Learnings Subtle changes in input can have such huge changes on output The differences between GPT-2 and GPT-3 are astounding and each one has its quirks and character It can be totally dynamic - the show is rendered in real-time - so many possibilities of how people can interact and modify Building a completely procedural ‘show’ is a possibility
- 29. Unity Metacast
- 31. Metacast Vision / Mission: Revolutionize how fans engage with sports 3D volumetric capture of the event Streamed in 3D to your mobile Full agency over time and POV Super hard We have it working, it’s real
- 32. Metacast
- 33. How does it work?
- 34. Build your own capture studio Solve volumetric data (partnership) Run ML pose estimation Make your own ML pose estimation Make your own volumetric file format Make your own compression Create custom AI camera system Make custom volumetric NLE Figure out how to stream giant data Design app to render 3D quickly Design 3D interaction UX
- 35. Build your own capture studio Solve volumetric data (partnership) Run ML pose estimation Make your own ML pose estimation Make your own volumetric file format Make your own compression Create custom AI camera system Make custom volumetric NLE Figure out how to stream giant data Design app to render 3D quickly Design 3D interaction UX Difficult
- 36. Volumetric Pipeline & Platform End to end delivery of Volumetric data to fans and broadcasters Raw Volumetric Data First Citizen Unity Format Storage & Streaming Unity CCD Unity Volumetric Editor Unity App Active Distribution Networks
- 40. Estimate Depth Generate Point Sets Depth/PC Fusion (Registration) Color Reprojection Meshing & Reduction Algorithm Overview Type #1 - Calibrated Environment (Metastage etc)
- 42. Commercial pose estimation breaks down… We had to build our own, which also means training
- 43. How do you film a variable scenario (when you’re not there)
- 44. Tech partnership with INRIA in France
- 46. Nobody can be there to ‘film’ a variable scenario in real-time
- 51. New ways to monetize Virtually try-on and purchase gear from within the app, personalized in-world ads Drive and measure engagement through integrations with e-commerce and social media platforms.
- 52. Be respectful of the medium
- 53. “Metatok” gives a compressed TikTok- like UX to sports broadcasts Swipe left/right for highlights Swipe up/down for different fights or games At any time you can still touch-orbit
- 56. What did we learn?
- 57. Start with the user experience first DO NOT scale up until you’ve got something compelling
- 58. Objectives: - How can Unity render point cloud data with dynamic lighting - How can Unity power cinematic sports replays without keframe cameras?
- 59. Focus on the UX If it doesn’t feel like magic, you’re not done
- 60. We’ve been iterating on this UX for years It’s hard to make things feel effortless Simple is hard Complex is easy
- 61. Consumers don’t really care about technology People only care about how things make them feel
- 62. We have been building content Bottom up, or inside-out
- 65. AI and ML are beginning to take over They’re top-down You start, with the end
- 66. $$$$ Months of work Some kid on his gaming PC after dinner
- 70. THE ENORMOUS SCALE OF DYNAMIC, REAL-TIME CONTENT IS LITERALLY BEYOND HUMAN. There simply aren’t enough creators on earth to fulfill the promise of endless, immersive, hyper-personalized content.
- 71. Procedural generation Machine Learning Artificial Intelligence
Editor's Notes
- #17: In short, we’re bringing entertainment to life, by giving it a mind of its own.
- #19: In our Transforms TV prototype, we’ve proven that dynamic content is possible. With the right support, our plan is to design and scale this tech to empower studios and creators alike to co-create with cutting edge machine intelligence.
- #71: The sheer volume needed to scale endless, hyper-personalized content, is quite literally beyond human.