SlideShare a Scribd company logo

Ph.D. Defense: Expressive Sound Synthesis for Animation

Cecile P-L, profile picture
Cecile P-L

Ce document présente une thèse de doctorat sur la synthèse sonore expressif pour l'animation, abordant des méthodes comme la synthèse sonore basée sur la physique, le modélage de contact, et la synthèse basée sur des exemples. Il discute des défis et des perspectives d'un modèle hybride pour la génération de son interactif dans des environnements virtuels. Les contributions incluent une analyse automatique des enregistrements préexistants et des approches pour la synthèse sonore flexible adaptée à l'animation guidée par la physique.

TechnologyBusiness
Related topics:
Virtual Reality
1 of 65
Downloaded 28 times
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
t Expressive Sound Synthesis For Animation Cécile Picard-Limpens University of Nice/Sophia-Antipolis École Doctorale STIC REVES INRIA Sophia-Antipolis, France Advisors: George Drettakis, INRIA Sophia Antipolis (Reves) François Faure, INRIA Rhône-Alpes (Evasion) Nicolas Tsingos, DOLBY Laboratories, CA, USA Defense for Ph.D. in Computer Science C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 1
t Outline 1 Sound and Virtuality 2 Physics-Based Sound Synthesis Contact Modeling Resonator Modeling 3 Example-Based Synthesis Flexible Sound Synthesis 4 Perspectives on a Hybrid Model Motivation and Application 5 Conclusion and Discussion Contributions Extensions and Applications C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 2
t Sound Rendering Sound and Virtuality General Background for Virtual Reality and Games Motivation Physics-Based Synthesis Example-Based Interactive Audio Rendering Synthesis Perspectives on a Hybrid Model Conclusion and Discussion (R. Vantielcke - WipeoutHD on Playstation 3) C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 3
t Sound Rendering Sound and Virtuality General Background for Virtual Reality and Games Motivation Physics-Based Synthesis Example-Based Interactive Audio Rendering Synthesis Perspectives on a Hybrid Model Conclusion and Discussion (R. Vantielcke - WipeoutHD on Playstation 3) Traditional Approach Pre-Recordings Triggered + : Easy to implement – : Repetitive audio, discrepancies, lack of flexibility C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 3
t From Playback of Samples Sound and Virtuality General Background to Synthesis Motivation Physics-Based Synthesis Digital Sound Synthesis Example-Based Synthesis Source modeling ← Perspectives on Sound propagation, Sound reception a Hybrid Model Conclusion and Techniques Discussion Rigid body simulation Finite Element Method (FEM) (ArtiSynth) C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 4
t From Playback of Samples Sound and Virtuality General Background to Synthesis Motivation Physics-Based Synthesis Digital Sound Synthesis Example-Based Synthesis Source modeling ← Perspectives on Sound propagation, Sound reception a Hybrid Model Conclusion and Techniques Discussion Rigid body simulation Finite Element Method (FEM) (ArtiSynth) Physical Sound Simulation + : Physical approach, easy parametrization, Low memory usage – : Preprocess computation, Interface between physics and sound system C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 4
t Controlling the Sound Simulation Sound and Virtuality Challenges General Background Motivation Physics-Based Synthesis Example-Based Synthesis Perspectives on Sound Coherent With Visuals a Hybrid Model Conclusion and Unpredictable character of sounds Discussion Real-time sound synthesis Parametrization and Expressiveness Control and interactivity Authoring C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 5
t Our Contribution Sound and Virtuality Three Research Axes General Background Motivation Physics-Based Synthesis Example-Based Synthesis Physics-Based Sound synthesis Perspectives on a Hybrid Model Contact modeling Conclusion and Resonator modeling Discussion C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 6
t Our Contribution Sound and Virtuality Three Research Axes General Background Motivation Physics-Based Synthesis Example-Based Synthesis Physics-Based Sound synthesis Perspectives on a Hybrid Model Contact modeling Conclusion and Resonator modeling Discussion Example-Based Sound Synthesis Automatic analysis of pre-recordings Flexible synthesis for physics-driven animation C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 6
t Our Contribution Sound and Virtuality Three Research Axes General Background Motivation Physics-Based Synthesis Example-Based Synthesis Physics-Based Sound synthesis Perspectives on a Hybrid Model Contact modeling Conclusion and Resonator modeling Discussion Example-Based Sound Synthesis Automatic analysis of pre-recordings Flexible synthesis for physics-driven animation Perspectives on a Hybrid Model C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 6
t Overview Sound and Virtuality Physics-Based Synthesis 1 Sound and Virtuality Contact Modeling Audio Texture Synthesis For Complex Contacts 2 Physics-Based Sound Synthesis Resonator Modeling Contact Modeling A Robust and Multi-Scale Modal Analysis Resonator Modeling Example-Based Synthesis 3 Example-Based Synthesis Perspectives on Flexible Sound Synthesis a Hybrid Model Conclusion and 4 Perspectives on a Hybrid Model Discussion Motivation and Application 5 Conclusion and Discussion Contributions Extensions and Applications C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 7
t Sound from Contacts Sound and Virtuality Physics-Based Synthesis Contact Modeling Audio Texture Synthesis For Complex Dichotomy Contacts Resonator Modeling Impacts A Robust and Multi-Scale Modal Analysis Continuous contacts Example-Based Synthesis Two Schemes for Contact Force Modelling Perspectives on a Hybrid Model Feed-forward scheme Conclusion and [van den Doel et al. 01] Discussion Additive synthesis Direct computation of contact forces [Avanzini et al. 02] Bristle model C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 8
t Contact Modeling Sound and Virtuality Physics-Based Synthesis Contact Modeling What Are The Current Limitations Audio Texture Synthesis For Complex for Continuous Contacts? Contacts Resonator Modeling Rate for physics engine report A Robust and Multi-Scale Modal Analysis No geometric details when using visual textures Example-Based Synthesis Authoring and control are challenging Perspectives on a Hybrid Model Conclusion and Discussion C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 9
t Contact Modeling Sound and Virtuality Physics-Based Synthesis Contact Modeling What Are The Current Limitations Audio Texture Synthesis For Complex for Continuous Contacts? Contacts Resonator Modeling Rate for physics engine report A Robust and Multi-Scale Modal Analysis No geometric details when using visual textures Example-Based Synthesis Authoring and control are challenging Perspectives on a Hybrid Model HOW Can We Solve Them? Conclusion and Discussion By extracting Excitation profiles from visual textures with Adaptive resolution [Picard et al., VRIPHYS 08] C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 9
t Method for Impact Sounds Sound and Virtuality Physics-Based Synthesis Contact Modeling Audio Texture Synthesis For Complex Contacts Resonator Modeling A Robust and Multi-Scale Modal Analysis Example-Based Synthesis Perspectives on a Hybrid Model Conclusion and Discussion C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 10
t Method for Continuous Contact Sounds Sound and Virtuality Extraction of Excitation Profiles Physics-Based Synthesis Contact Modeling Audio Texture Synthesis For Complex Contacts Resonator Modeling A Robust and Multi-Scale Modal Analysis Example-Based Synthesis Perspectives on a Hybrid Model Conclusion and Discussion C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 11
t Synthesis of Excitation Profiles Sound and Virtuality For the Audio Force Modelling Physics-Based Synthesis Contact Modeling Audio Texture Synthesis For Complex Contacts Technique Resonator Modeling A Robust and Multi-Scale Modal Analysis Extraction from the visual texture image Example-Based Re-sampling along the trajectory Synthesis of the contact interaction (60Hz vs 44kHz) Perspectives on a Hybrid Model Conclusion and Based on the Complexity of the Histogram Discussion Simple texture image: Gradient of the image intensity Complex texture image: Isocurves of constant brightness (isophotes) C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 12
t Complex Textures Sound and Virtuality Coding the Excitation Profiles Physics-Based Synthesis Contact Modeling Audio Texture Synthesis For Complex Isophotes = Large amount of data Contacts Resonator Modeling How Can We Lighten the Info? A Robust and Multi-Scale Modal Analysis Example-Based By Coding the Excitation Profiles Synthesis = Main Features + Noise Part Perspectives on a Hybrid Model Conclusion and Discussion = + Noise Part: Statistical approximation C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 13
t Real-Time Audio Management Sound and Virtuality A Flexible Audio Pipeline Physics-Based Synthesis Contact Modeling Audio Texture Synthesis For Complex Contacts Resonator Modeling Simulations Driven by Ageia’s PhysX (now NVIDIA) A Robust and Multi-Scale Modal Analysis Example-Based Synthesis Perspectives on a Hybrid Model Conclusion and Discussion C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 14
t Audio Texture Synthesis Sound and Virtuality A Solution for Interactive Simulations Physics-Based Synthesis Contact Modeling Audio Texture Synthesis For Complex Contacts Resonator Modeling A Robust and Multi-Scale Modal Analysis Example-Based A Sound in Coherence with Visuals Synthesis Perspectives on a Hybrid Model Flexible Resolution Conclusion and Discussion Adapted to Procedural Generation C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 15
t Overview Sound and Virtuality Physics-Based Synthesis 1 Sound and Virtuality Contact Modeling Audio Texture Synthesis For Complex Contacts 2 Physics-Based Sound Synthesis Resonator Modeling Contact Modeling A Robust and Multi-Scale Modal Analysis Resonator Modeling Example-Based Synthesis 3 Example-Based Synthesis Perspectives on Flexible Sound Synthesis a Hybrid Model Conclusion and 4 Perspectives on a Hybrid Model Discussion Motivation and Application 5 Conclusion and Discussion Contributions Extensions and Applications C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 16
t Vibration Models Sound and Virtuality Modal Analysis Physics-Based Synthesis Contact Modeling Audio Texture Synthesis For Complex Contacts Generating Sounds Based on Physics Simulation Resonator Modeling A Robust and Multi-Scale Modal In computer musics Analysis [Iovino et al. 97, Cook 02] Example-Based Synthesis In computer graphics Perspectives on [Van Den Doel 01, O Brien et al. 02] a Hybrid Model Conclusion and Discussion Improvements for Interactive Sound Rendering Modal parameter tracking [Maxwell et al. 07] Frequency content sparsity [Bonneel et al. 08] C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 17
t Vibration Models Sound and Virtuality Modal Analysis Physics-Based Synthesis Contact Modeling Audio Texture 1 Get a Sounding Object and its Geometry Synthesis For Complex Contacts Resonator Modeling A Robust and Multi-Scale Modal Analysis Example-Based Synthesis 2 Construct the FEM (ex: Tetrahedral Mesh) Perspectives on a Hybrid Model 3 Apply Newton Second Law to DOF Conclusion and Discussion ¨ ˙ M d + C d + Kd = f (1) 4 Eigendecomposition ⇒ Modal Parameters M = LL−T ; L−1 KL−T = V ΛV T (2) where V = matrix of eigenvectors Λ = diagonal matrix of eigenvalues C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 18
t Vibration Models Sound and Virtuality Modal Analysis Physics-Based Synthesis Contact Modeling Audio Texture Synthesis For Complex Contacts In Real-time: Resonator Modeling A Robust and Modal synthesis Multi-Scale Modal Analysis Example-Based Synthesis Perspectives on a Hybrid Model Conclusion and Discussion 1 s(t) = ai sin(wi t)e −di t (3) n Control for vibration models C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 19
t Vibration Models Sound and Virtuality Modal Analysis Physics-Based Synthesis Contact Modeling Audio Texture What Are Synthesis For Complex Contacts The Current Limitations? Resonator Modeling A Robust and Multi-Scale Modal Meshing is difficult Analysis Example-Based No real control on the FEM resolution Synthesis No clear interface between physics and audio Perspectives on a Hybrid Model Conclusion and Discussion C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 20
t Vibration Models Sound and Virtuality Modal Analysis Physics-Based Synthesis Contact Modeling Audio Texture What Are Synthesis For Complex Contacts The Current Limitations? Resonator Modeling A Robust and Multi-Scale Modal Meshing is difficult Analysis Example-Based No real control on the FEM resolution Synthesis No clear interface between physics and audio Perspectives on a Hybrid Model Conclusion and Discussion HOW Can We Solve Them? By proposing A robust and multi-scale modal analysis which is Coherent with the physics simulation [Picard et al., DAFx 09] C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 20
t Our Deformation Model Sound and Virtuality Physics-Based Synthesis Inspired from Work by Nesme et al. Contact Modeling Audio Texture [Nesme et al. 06] Synthesis For Complex Contacts Resonator Modeling A Robust and Technique Multi-Scale Modal Analysis Merged voxels used as Hexahedral Finite Elements Example-Based Synthesis Perspectives on a Hybrid Model Conclusion and Discussion Implementation with the Sofa Framework Validation of the Model Tests on a metal cube C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 21
t Robustness Sound and Virtuality Physics-Based Synthesis Contact Modeling Audio Texture Robust Even for Non-Manifold Geometries Synthesis For Complex Contacts Resonator Modeling A Robust and Multi-Scale Modal Analysis Example-Based Synthesis Perspectives on a Hybrid Model Conclusion and Discussion Material: Aluminium C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 22
t Multi-Scale for Efficient Memory Usage Sound and Virtuality Physics-Based A Squirrel in Pine Wood Synthesis Contact Modeling Audio Texture Synthesis For Complex Contacts Resonator Modeling A Robust and Multi-Scale Modal Analysis Example-Based Synthesis Perspectives on a Hybrid Model Conclusion and Discussion C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 23
t Multi-Scale for Efficient Memory Usage Sound and Virtuality Physics-Based A Squirrel in Pine Wood: Different FE resolutions Synthesis Contact Modeling Audio Texture 3x3x3 4x4x4 8x8x8 9x9x9 Synthesis For Complex Contacts Resonator Modeling A Robust and Multi-Scale Modal Analysis Example-Based Synthesis Perspectives on a Hybrid Model Frequency Content = f (Hexahedral FE Resolution) Conclusion and Discussion Higher resolution models Frequency centroid shift Convergence of the frequency content C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 24
t Comparison with Classical Approach Sound and Virtuality Physics-Based Sounding Bowl - Material: Aluminium Synthesis Contact Modeling Audio Texture Classical Approach Our Approach Synthesis For Complex Contacts (816 modes) (75 modes) Resonator Modeling A Robust and Multi-Scale Modal Analysis Example-Based Synthesis Perspectives on a Hybrid Model Conclusion and Discussion C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 25
t A Robust and Multi-Scale Modal Analysis Sound and Virtuality A Solution for Sound Synthesis Physics-Based Synthesis Contact Modeling Audio Texture Synthesis For Complex Contacts Resonator Modeling A Robust and Multi-Scale Modal Analysis Example-Based Realistic Synthesis Perspectives on a Hybrid Model Adapted to Non-Manifold Geometries Conclusion and Discussion Resources Flexibility C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 26
t Overview Sound and Virtuality Physics-Based Synthesis 1 Sound and Virtuality Example-Based Synthesis 2 Physics-Based Sound Synthesis Flexible Sound Synthesis Contact Modeling Retargetting Example Sounds Resonator Modeling Perspectives on a Hybrid Model 3 Example-Based Synthesis Conclusion and Discussion Flexible Sound Synthesis 4 Perspectives on a Hybrid Model Motivation and Application 5 Conclusion and Discussion Contributions Extensions and Applications C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 27
t Implementation of Signal-Based Models Sound and Virtuality Physics-Based Synthesis Example-Based Synthesis Flexible Sound Concatenative Synthesis Synthesis Retargetting Example [Roads 91, Schwarz 06] Sounds Perspectives on Sound Textures Based on Physics a Hybrid Model [Cook 99] Conclusion and Discussion [Dobashi et al. 03, Zheng et al. 09] Dobashi et al. 03 Authoring and Interactive Control [Cook 02] Cook 99 C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 28
t Implementation of Signal-Based Models Sound and Virtuality Physics-Based Synthesis Example-Based Synthesis What Are Flexible Sound Synthesis The Current Limitations? Retargetting Example Sounds Processing is not generic Perspectives on a Hybrid Model Parametrizing is difficult Conclusion and Discussion C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 29
t Implementation of Signal-Based Models Sound and Virtuality Physics-Based Synthesis Example-Based Synthesis What Are Flexible Sound Synthesis The Current Limitations? Retargetting Example Sounds Processing is not generic Perspectives on a Hybrid Model Parametrizing is difficult Conclusion and Discussion HOW Can We Solve Them? By Retargetting example sounds To physics-driven animation [Picard et al., AES 09] C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 29
t Our Approach Sound and Virtuality Physics-Based Synthesis SINUSOIDAL AUDIO RECORDING Example-Based TRANSIENT Synthesis OBJECT GEOMETRY VIRTUAL ENVIRONMENT Flexible Sound 1 DICTIONARY OF AUDIO GRAINS Synthesis Impulsive / Continuous Retargetting Example BUILD COLLISION Sounds STRUCTURES Perspectives on 2 CORRELATION PATTERNS a Hybrid Model DEFINE PROCEDURES PREPROCESSING INTERACTIVE Conclusion and RETARGETTING RIGID-BODY Discussion TO ANIMATION SIMULATION AUDIO RENDERER VIDEO RENDERER ANIMATION WITH AUDIO Amplitude Time Our Contributions C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 30
t Preprocess: A Generic Analysis Sound and Virtuality Physics-Based Synthesis Example-Based Synthesis Flexible Sound Synthesis Impulsive and Continuous Contacts Retargetting Example Sounds Spectral Modeling Synthesis (SMS) [Serra 97] Perspectives on a Hybrid Model Conclusion and Automatic Extraction of Audio Grains Discussion Dictionary: Impulsive/Continuous Generation of Correlation Patterns between original recordings and audio grains C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 31
t On-Line: Flexible Sound Synthesis Sound and Virtuality Physics-Based Synthesis Example-Based Synthesis Resynthesis of the Original Recordings Flexible Sound Synthesis Candidate grains: max. correlation amplitude Retargetting Example Sounds Perspectives on a Hybrid Model Conclusion and Discussion C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 32
t On-Line: Flexible Sound Synthesis Sound and Virtuality Physics-Based Synthesis Example-Based Synthesis Resynthesis of the Original Recordings Flexible Sound Synthesis Candidate grains: max. correlation amplitude Retargetting Example Sounds Perspectives on Interactive Physics-Driven Animations a Hybrid Model Physics Info for Retargetting Conclusion and Discussion Contact type: impulsive or continuous? Penetration force and relative velocity C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 32
t On-Line: Flexible Sound Synthesis Sound and Virtuality Physics-Based Synthesis Example-Based Synthesis Resynthesis of the Original Recordings Flexible Sound Synthesis Candidate grains: max. correlation amplitude Retargetting Example Sounds Perspectives on Interactive Physics-Driven Animations a Hybrid Model Physics Info for Retargetting Conclusion and Discussion Contact type: impulsive or continuous? Penetration force and relative velocity Flexible Audio Shading Approach Additional, User-defined Resynthesis Schemes Spectral domain adaptation/modification C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 32
t Resynthesis of the Original Recordings Sound and Virtuality Physics-Based Synthesis Example-Based Synthesis Flexible Sound 94 recordings (14.6Mb) Synthesis ≈ 5000 grains + 94 Correlation Patterns (20% Gain) Retargetting Example Sounds Perspectives on Breaking Glass a Hybrid Model Conclusion and Shooting Gun Discussion Rolling Additional Material: http://www-sop.inria.fr/members/Cecile.Picard/ "‘Supplemental AES"’ C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 33
t Flexible Audio Shading Approach Sound and Virtuality Physics-Based Synthesis Example-Based Synthesis Easy Implementation of Time-Scaling Flexible Sound Synthesis Retargetting Example Faster Rolling Sounds Perspectives on Slower Breaking a Hybrid Model Conclusion and Discussion Synthesis of An Infinity Similar Audio Events by varying the audio content Rythmic pattern from Breaking Stone New material content: stone and gun Rythmic pattern from Breaking Glass New material content: ceramic C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 34
t Interactive Physics-Driven Animations Sound and Virtuality Physics-Based Synthesis Example-Based Synthesis Flexible Sound Simulations Driven by Sofa Framework Synthesis Retargetting Example Sounds Perspectives on a Hybrid Model Conclusion and Discussion C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 35
t Retargetting Example Sounds Sound and Virtuality A Solution for Interactive Simulations Physics-Based Synthesis Example-Based Synthesis Flexible Sound Synthesis Retargetting Example Variety Sounds Perspectives on a Hybrid Model Adapted to Scenarios Conclusion and Discussion Small Memory Footprint Real-Time Rendering An attractive solution for industrial applications (Eden Games, an ATARI game studio) C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 36
t Overview Sound and Virtuality Physics-Based Synthesis 1 Sound and Virtuality Example-Based Synthesis 2 Physics-Based Sound Synthesis Perspectives on a Hybrid Model Contact Modeling Motivation Resonator Modeling A Hybrid Model for Fracture Events Conclusion and 3 Example-Based Synthesis Discussion Flexible Sound Synthesis 4 Perspectives on a Hybrid Model Motivation and Application 5 Conclusion and Discussion Contributions Extensions and Applications C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 37
t Sound Modeling Sound and Virtuality When Nonlinearity Occurs Physics-Based Synthesis Example-Based Synthesis Perspectives on a Hybrid Model Motivation Problems of Single Models A Hybrid Model for Fracture Events Vibration models assume linearity Conclusion and Discussion Example-based sounds are hard to parametrize C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 38
t Sound Modeling Sound and Virtuality When Nonlinearity Occurs Physics-Based Synthesis Example-Based Synthesis Perspectives on a Hybrid Model Motivation Problems of Single Models A Hybrid Model for Fracture Events Vibration models assume linearity Conclusion and Discussion Example-based sounds are hard to parametrize Previous Work Modeling nonlinearities [O Brien et al. 01, Chadwick et al. 09] [Cook 02] C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 38
t Fracture Events Sound and Virtuality Physics-Based Synthesis Example-Based Background Synthesis Frequently occur in virtual environments Perspectives on a Hybrid Model Motivation Visual rendering A Hybrid Model for Fracture Events [O Brien et al. 99, 02] Conclusion and [Parker and O Brien. 09] Discussion Sound rendering: Little research [Warren et al. 84] [Rath et al. 03] C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 39
t Fracture Events Sound and Virtuality Physics-Based Synthesis Example-Based Background Synthesis Frequently occur in virtual environments Perspectives on a Hybrid Model Motivation Visual rendering A Hybrid Model for Fracture Events [O Brien et al. 99, 02] Conclusion and [Parker and O Brien. 09] Discussion Sound rendering: Little research [Warren et al. 84] [Rath et al. 03] Challenges Event depends on the material involved Differents phases emerge from fracture event C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 39
t Parametrization of Our Hybrid Model Sound and Virtuality Physics-Based Synthesis Example-Based Synthesis Selection Criteria Perspectives on Hybrid model applied when nonlinearity occurs a Hybrid Model Motivation A Hybrid Model for Fracture Events Techniques Conclusion and Discussion FM synthesis Audio grains FM synthesis Parametrization Smooth transition with vibration model Coherence inside the hybrid model C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 40
t Discussion Sound and Virtuality Physics-Based Synthesis Example-Based Synthesis Perspectives on a Hybrid Model Motivation Prospective model A Hybrid Model for Fracture Events Conclusion and Possible problem: report from the physics engine Discussion Simplicity of the tools allows real-time rendering C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 41
t Overview Sound and Virtuality Physics-Based Synthesis 1 Sound and Virtuality Example-Based Synthesis 2 Physics-Based Sound Synthesis Perspectives on a Hybrid Model Contact Modeling Conclusion and Resonator Modeling Discussion Contributions 3 Example-Based Synthesis Extensions and Applications Flexible Sound Synthesis 4 Perspectives on a Hybrid Model Motivation and Application 5 Conclusion and Discussion Contributions Extensions and Applications C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 42
t Synthesis of Sounds for Animation Sound and Virtuality Difficulties Physics-Based Synthesis Example-Based Synthesis Perspectives on a Hybrid Model Conclusion and Audio-Visual Coherence Discussion Contributions Extensions and Applications Extremely Dynamic Character Precision of Synthesis Large Variety of Objects C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 43
t Contributions Sound and Virtuality An Overview Physics-Based Synthesis Example-Based Complex Contact Modeling Synthesis Perspectives on 2D visual textures used as roughness maps a Hybrid Model Audible and position-dependent variations Conclusion and Detail-layer mechanisms Discussion Contributions Extensions and Applications C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 44
t Contributions Sound and Virtuality An Overview Physics-Based Synthesis Example-Based Complex Contact Modeling Synthesis Perspectives on 2D visual textures used as roughness maps a Hybrid Model Audible and position-dependent variations Conclusion and Detail-layer mechanisms Discussion Contributions Extensions and Applications Improved Modal Analysis for Resonator Modeling Complex non-manifold geometries can be handled Multi-scale resolution Coherence between simulation and audio C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 44
t Contributions Sound and Virtuality An Overview Physics-Based Synthesis Example-Based Complex Contact Modeling Synthesis Perspectives on 2D visual textures used as roughness maps a Hybrid Model Audible and position-dependent variations Conclusion and Detail-layer mechanisms Discussion Contributions Extensions and Applications Improved Modal Analysis for Resonator Modeling Complex non-manifold geometries can be handled Multi-scale resolution Coherence between simulation and audio Flexibility of Sound Design Audio grains and correlation patterns Dynamic retargetting to events Extended sound synthesis capabilities C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 44
t Contributions Sound and Virtuality Perspectives Physics-Based Synthesis Example-Based Synthesis Perspectives on a Hybrid Model Conclusion and Discussion A Prospective Hybrid Model Contributions for Complex Physical Phenomena Extensions and Applications Focus on Nonlinearity Combination of physically based and example-based methods Application Case: Fracture Events C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 45
t Overview Sound and Virtuality Physics-Based Synthesis 1 Sound and Virtuality Example-Based Synthesis 2 Physics-Based Sound Synthesis Perspectives on a Hybrid Model Contact Modeling Conclusion and Resonator Modeling Discussion Contributions 3 Example-Based Synthesis Extensions and Applications Flexible Sound Synthesis 4 Perspectives on a Hybrid Model Motivation and Application 5 Conclusion and Discussion Contributions Extensions and Applications C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 46
t Promising Directions for Future Work Sound and Virtuality Physics-Based Synthesis Complex Contact Modeling Example-Based Synthesis Two interacting textures Perspectives on Surface-based interactions a Hybrid Model Adequate perceptual experiments Conclusion and Discussion Contributions Extensions and Applications C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 47
t Promising Directions for Future Work Sound and Virtuality Physics-Based Synthesis Complex Contact Modeling Example-Based Synthesis Two interacting textures Perspectives on Surface-based interactions a Hybrid Model Adequate perceptual experiments Conclusion and Discussion Contributions Improved Modal Analysis for Resonator Modeling Extensions and Applications Recent work from [Nesme et al. Siggraph 09] Investigations with GPU for in-line computation Complete integration in a virtual scene C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 47
t Promising Directions for Future Work Sound and Virtuality Physics-Based Synthesis Complex Contact Modeling Example-Based Synthesis Two interacting textures Perspectives on Surface-based interactions a Hybrid Model Adequate perceptual experiments Conclusion and Discussion Contributions Improved Modal Analysis for Resonator Modeling Extensions and Applications Recent work from [Nesme et al. Siggraph 09] Investigations with GPU for in-line computation Complete integration in a virtual scene Example-Based Technique Clustering of similar grains Statistical analysis of correlation patterns Physics engine design C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 47
t Promising Directions for Future Work Sound and Virtuality Physics-Based Synthesis Example-Based Synthesis Perspectives on a Hybrid Model Conclusion and Discussion Hybrid Model for Fracture Events Contributions Extensions and Fracture sound simulation framework Applications Tracking of relevant physical data C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 48
t Conclusion Sound and Virtuality Physics-Based Synthesis Example-Based Synthesis Perspectives on a Hybrid Model New Physically Based Algorithms Conclusion and Discussion for Sound Rendering Contributions Extensions and Flexibility of Sound Modeling Applications Ideas on an Adequate Hybrid Sound Model Additional info: http://www-sop.inria.fr/members/Cecile.Picard/ C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 49
t Acknowledgements Sound and Virtuality Physics-Based Synthesis Example-Based Synthesis Perspectives on a Hybrid Model George Drettakis, François Faure, Conclusion and and Nicolas Tsingos Discussion Contributions REVES Team Extensions and Applications Marie-Paule Cani and the Evasion Team Paul G. Kry at the McGill University, Montréal Eden Games, an ATARI game studio, Lyon C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 50

More Related Content

PDF
Adria Recasens, DeepMind – Multi-modal self-supervised learning from videos
Codiax
 
PPT
Multimodal Analysis for Bridging Semantic Gap with Biologically Inspired Algo...
techkrish
 
PDF
igarss.pdf
grssieee
 
PDF
2001 Tf Lcmv Signal Enhancement Using Beamforming And Nonstationarity With Ap...
luca
 
PPT
Graphics
sitifadzilahmatnoor
 
PDF
IRJET- A Survey on Sound Recognition
IRJET Journal
 
PDF
APPRAISAL AND ANALOGY OF MODIFIED DE-NOISING AND LOCAL ADAPTIVE WAVELET IMAGE...
International Journal of Technical Research & Application
 
PDF
Lossy Compression Using Stationary Wavelet Transform and Vector Quantization
Omar Ghazi
 
Adria Recasens, DeepMind – Multi-modal self-supervised learning from videos
Codiax
 
Multimodal Analysis for Bridging Semantic Gap with Biologically Inspired Algo...
techkrish
 
igarss.pdf
grssieee
 
2001 Tf Lcmv Signal Enhancement Using Beamforming And Nonstationarity With Ap...
luca
 
Graphics
sitifadzilahmatnoor
 
IRJET- A Survey on Sound Recognition
IRJET Journal
 
APPRAISAL AND ANALOGY OF MODIFIED DE-NOISING AND LOCAL ADAPTIVE WAVELET IMAGE...
International Journal of Technical Research & Application
 
Lossy Compression Using Stationary Wavelet Transform and Vector Quantization
Omar Ghazi
 

What's hot (9)

PDF
Audio Steganography Coding Using the Discreet Wavelet Transforms
CSCJournals
 
PPTX
Spatio-temporal control of light in complex media
Sébastien Popoff
 
PDF
A Brief Introduction of Anomalous Sound Detection: Recent Studies and Future...
Yuma Koizumi
 
PDF
Analysis of PEAQ Model using Wavelet Decomposition Techniques
idescitation
 
PDF
Introduction to compressive sensing
Ahmed Nasser Agag
 
PDF
Performance Analysis of Digital Watermarking Of Video in the Spatial Domain
paperpublications3
 
PDF
Brunelli 2008: template matching techniques in computer vision
zukun
 
PDF
Miller - Remote Sensing and Imaging Physics - Spring Review 2012
The Air Force Office of Scientific Research
 
PDF
On the scattering of light : various models and methods used in computer grap...
Toru Tamaki
 
Audio Steganography Coding Using the Discreet Wavelet Transforms
CSCJournals
 
Spatio-temporal control of light in complex media
Sébastien Popoff
 
A Brief Introduction of Anomalous Sound Detection: Recent Studies and Future...
Yuma Koizumi
 
Analysis of PEAQ Model using Wavelet Decomposition Techniques
idescitation
 
Introduction to compressive sensing
Ahmed Nasser Agag
 
Performance Analysis of Digital Watermarking Of Video in the Spatial Domain
paperpublications3
 
Brunelli 2008: template matching techniques in computer vision
zukun
 
Miller - Remote Sensing and Imaging Physics - Spring Review 2012
The Air Force Office of Scientific Research
 
On the scattering of light : various models and methods used in computer grap...
Toru Tamaki
 
Ad

Similar to Ph.D. Defense: Expressive Sound Synthesis for Animation (20)

PDF
Adaptive Music and Interactive Audio
Adam Sporka
 
PDF
Foundations In Sound Design For Linear Media A Multidisciplinary Approach Mic...
gijonyenorwg
 
PDF
Designing Sound For Animation Robin Beauchamp
arnalpsi
 
PPT
Lecture6 audio
Mr SMAK
 
PPT
Lecture6 audio
Mr SMAK
 
PPT
Towards User-friendly Audio Creation
Jean Vanderdonckt
 
PDF
Computer sound design synthesis techniques and programming 2nd Edition Eduard...
jurcyszibaia
 
PPT
Multimedia tools (sound)
dhruv patel
 
PPT
Sound
Jay Patel
 
PDF
Echoes, Whispers, and Footsteps from the Conflux of Sonic Interaction Design ...
Eoin Brazil
 
PDF
Interactive Physically-Based Sound Design of 3D Model using Material Optimiza...
yamo_o
 
PPTX
Project - Sound Model Similarity Search
Sudarshan Bala
 
PDF
Automated Sound Design 1st Edition Eric Lyon.
subriyahagi
 
PDF
A Generalized Wave Field Synthesis Theory (Ph.D. Thesis Booklet)
Lori Moore
 
PDF
AudioSIG28Nov2011
Nevin Eronde
 
PDF
Experimental Media Voodoo™
SkyRonDotOrg
 
PDF
Computer Sound Design Synthesis Techniques And Programming 2nd Edition Eduard...
gxfqisu378
 
PDF
Multimedia elements
Niteshwar Bhardwaj
 
DOCX
Mit 8116
Josh Bwiira
 
PDF
Survey On Speech Synthesis
CSCJournals
 
Adaptive Music and Interactive Audio
Adam Sporka
 
Foundations In Sound Design For Linear Media A Multidisciplinary Approach Mic...
gijonyenorwg
 
Designing Sound For Animation Robin Beauchamp
arnalpsi
 
Lecture6 audio
Mr SMAK
 
Lecture6 audio
Mr SMAK
 
Towards User-friendly Audio Creation
Jean Vanderdonckt
 
Computer sound design synthesis techniques and programming 2nd Edition Eduard...
jurcyszibaia
 
Multimedia tools (sound)
dhruv patel
 
Sound
Jay Patel
 
Echoes, Whispers, and Footsteps from the Conflux of Sonic Interaction Design ...
Eoin Brazil
 
Interactive Physically-Based Sound Design of 3D Model using Material Optimiza...
yamo_o
 
Project - Sound Model Similarity Search
Sudarshan Bala
 
Automated Sound Design 1st Edition Eric Lyon.
subriyahagi
 
A Generalized Wave Field Synthesis Theory (Ph.D. Thesis Booklet)
Lori Moore
 
AudioSIG28Nov2011
Nevin Eronde
 
Experimental Media Voodoo™
SkyRonDotOrg
 
Computer Sound Design Synthesis Techniques And Programming 2nd Edition Eduard...
gxfqisu378
 
Multimedia elements
Niteshwar Bhardwaj
 
Mit 8116
Josh Bwiira
 
Survey On Speech Synthesis
CSCJournals
 
Ad

Recently uploaded (20)

PDF
Reach Out and Touch Someone: Haptics and Empathic Computing
Mark Billinghurst
 
PDF
Network Security Unit 5.pdf for BCA BBA.
Serpent6
 
PDF
Diabetes mellitus diagnosis method based random forest with bat algorithm
IAESIJAI
 
PPTX
Group 1 Presentation -Planning and Decision Making .pptx
MatthewLewis227954
 
PDF
Approach and Philosophy of On baking technology
30anthuong17
 
PPTX
Spectroscopy.pptx food analysis technology
nawahassan45
 
PDF
Spectral efficient network and resource selection model in 5G networks
IAESIJAI
 
PPTX
KOM of Painting work and Equipment Insulation REV00 update 25-dec.pptx
muhammadmuneebnaeem7
 
PPTX
A Presentation on Artificial Intelligence
memembruh336
 
PPTX
Programs and apps: productivity, graphics, security and other tools
4mqw9zch22
 
PDF
Accuracy of neural networks in brain wave diagnosis of schizophrenia
IAESIJAI
 
PDF
Univ-Connecticut-ChatGPT-Presentaion.pdf
ry7r52mzb4
 
PDF
Mobile App Security Testing_ A Comprehensive Guide.pdf
flufftailshop
 
PDF
A comparative study of natural language inference in Swahili using monolingua...
IAESIJAI
 
PDF
A comparative analysis of optical character recognition models for extracting...
IAESIJAI
 
PPTX
1. Introduction to Computer Programming.pptx
bonakiduza1024
 
PDF
Mushroom cultivation and it's methods.pdf
mailmeonrishi
 
PDF
Blue Purple Modern Animated Computer Science Presentation.pdf.pdf
Akar16
 
PDF
Assigned Numbers - 2025 - Bluetooth® Document
Bloombase
 
PDF
Advanced methodologies resolving dimensionality complications for autism neur...
IAESIJAI
 
Reach Out and Touch Someone: Haptics and Empathic Computing
Mark Billinghurst
 
Network Security Unit 5.pdf for BCA BBA.
Serpent6
 
Diabetes mellitus diagnosis method based random forest with bat algorithm
IAESIJAI
 
Group 1 Presentation -Planning and Decision Making .pptx
MatthewLewis227954
 
Approach and Philosophy of On baking technology
30anthuong17
 
Spectroscopy.pptx food analysis technology
nawahassan45
 
Spectral efficient network and resource selection model in 5G networks
IAESIJAI
 
KOM of Painting work and Equipment Insulation REV00 update 25-dec.pptx
muhammadmuneebnaeem7
 
A Presentation on Artificial Intelligence
memembruh336
 
Programs and apps: productivity, graphics, security and other tools
4mqw9zch22
 
Accuracy of neural networks in brain wave diagnosis of schizophrenia
IAESIJAI
 
Univ-Connecticut-ChatGPT-Presentaion.pdf
ry7r52mzb4
 
Mobile App Security Testing_ A Comprehensive Guide.pdf
flufftailshop
 
A comparative study of natural language inference in Swahili using monolingua...
IAESIJAI
 
A comparative analysis of optical character recognition models for extracting...
IAESIJAI
 
1. Introduction to Computer Programming.pptx
bonakiduza1024
 
Mushroom cultivation and it's methods.pdf
mailmeonrishi
 
Blue Purple Modern Animated Computer Science Presentation.pdf.pdf
Akar16
 
Assigned Numbers - 2025 - Bluetooth® Document
Bloombase
 
Advanced methodologies resolving dimensionality complications for autism neur...
IAESIJAI
 

Ph.D. Defense: Expressive Sound Synthesis for Animation

  • 1. t Expressive Sound Synthesis For Animation Cécile Picard-Limpens University of Nice/Sophia-Antipolis École Doctorale STIC REVES INRIA Sophia-Antipolis, France Advisors: George Drettakis, INRIA Sophia Antipolis (Reves) François Faure, INRIA Rhône-Alpes (Evasion) Nicolas Tsingos, DOLBY Laboratories, CA, USA Defense for Ph.D. in Computer Science C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 1
  • 2. t Outline 1 Sound and Virtuality 2 Physics-Based Sound Synthesis Contact Modeling Resonator Modeling 3 Example-Based Synthesis Flexible Sound Synthesis 4 Perspectives on a Hybrid Model Motivation and Application 5 Conclusion and Discussion Contributions Extensions and Applications C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 2
  • 3. t Sound Rendering Sound and Virtuality General Background for Virtual Reality and Games Motivation Physics-Based Synthesis Example-Based Interactive Audio Rendering Synthesis Perspectives on a Hybrid Model Conclusion and Discussion (R. Vantielcke - WipeoutHD on Playstation 3) C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 3
  • 4. t Sound Rendering Sound and Virtuality General Background for Virtual Reality and Games Motivation Physics-Based Synthesis Example-Based Interactive Audio Rendering Synthesis Perspectives on a Hybrid Model Conclusion and Discussion (R. Vantielcke - WipeoutHD on Playstation 3) Traditional Approach Pre-Recordings Triggered + : Easy to implement – : Repetitive audio, discrepancies, lack of flexibility C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 3
  • 5. t From Playback of Samples Sound and Virtuality General Background to Synthesis Motivation Physics-Based Synthesis Digital Sound Synthesis Example-Based Synthesis Source modeling ← Perspectives on Sound propagation, Sound reception a Hybrid Model Conclusion and Techniques Discussion Rigid body simulation Finite Element Method (FEM) (ArtiSynth) C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 4
  • 6. t From Playback of Samples Sound and Virtuality General Background to Synthesis Motivation Physics-Based Synthesis Digital Sound Synthesis Example-Based Synthesis Source modeling ← Perspectives on Sound propagation, Sound reception a Hybrid Model Conclusion and Techniques Discussion Rigid body simulation Finite Element Method (FEM) (ArtiSynth) Physical Sound Simulation + : Physical approach, easy parametrization, Low memory usage – : Preprocess computation, Interface between physics and sound system C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 4
  • 7. t Controlling the Sound Simulation Sound and Virtuality Challenges General Background Motivation Physics-Based Synthesis Example-Based Synthesis Perspectives on Sound Coherent With Visuals a Hybrid Model Conclusion and Unpredictable character of sounds Discussion Real-time sound synthesis Parametrization and Expressiveness Control and interactivity Authoring C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 5
  • 8. t Our Contribution Sound and Virtuality Three Research Axes General Background Motivation Physics-Based Synthesis Example-Based Synthesis Physics-Based Sound synthesis Perspectives on a Hybrid Model Contact modeling Conclusion and Resonator modeling Discussion C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 6
  • 9. t Our Contribution Sound and Virtuality Three Research Axes General Background Motivation Physics-Based Synthesis Example-Based Synthesis Physics-Based Sound synthesis Perspectives on a Hybrid Model Contact modeling Conclusion and Resonator modeling Discussion Example-Based Sound Synthesis Automatic analysis of pre-recordings Flexible synthesis for physics-driven animation C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 6
  • 10. t Our Contribution Sound and Virtuality Three Research Axes General Background Motivation Physics-Based Synthesis Example-Based Synthesis Physics-Based Sound synthesis Perspectives on a Hybrid Model Contact modeling Conclusion and Resonator modeling Discussion Example-Based Sound Synthesis Automatic analysis of pre-recordings Flexible synthesis for physics-driven animation Perspectives on a Hybrid Model C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 6
  • 11. t Overview Sound and Virtuality Physics-Based Synthesis 1 Sound and Virtuality Contact Modeling Audio Texture Synthesis For Complex Contacts 2 Physics-Based Sound Synthesis Resonator Modeling Contact Modeling A Robust and Multi-Scale Modal Analysis Resonator Modeling Example-Based Synthesis 3 Example-Based Synthesis Perspectives on Flexible Sound Synthesis a Hybrid Model Conclusion and 4 Perspectives on a Hybrid Model Discussion Motivation and Application 5 Conclusion and Discussion Contributions Extensions and Applications C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 7
  • 12. t Sound from Contacts Sound and Virtuality Physics-Based Synthesis Contact Modeling Audio Texture Synthesis For Complex Dichotomy Contacts Resonator Modeling Impacts A Robust and Multi-Scale Modal Analysis Continuous contacts Example-Based Synthesis Two Schemes for Contact Force Modelling Perspectives on a Hybrid Model Feed-forward scheme Conclusion and [van den Doel et al. 01] Discussion Additive synthesis Direct computation of contact forces [Avanzini et al. 02] Bristle model C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 8
  • 13. t Contact Modeling Sound and Virtuality Physics-Based Synthesis Contact Modeling What Are The Current Limitations Audio Texture Synthesis For Complex for Continuous Contacts? Contacts Resonator Modeling Rate for physics engine report A Robust and Multi-Scale Modal Analysis No geometric details when using visual textures Example-Based Synthesis Authoring and control are challenging Perspectives on a Hybrid Model Conclusion and Discussion C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 9
  • 14. t Contact Modeling Sound and Virtuality Physics-Based Synthesis Contact Modeling What Are The Current Limitations Audio Texture Synthesis For Complex for Continuous Contacts? Contacts Resonator Modeling Rate for physics engine report A Robust and Multi-Scale Modal Analysis No geometric details when using visual textures Example-Based Synthesis Authoring and control are challenging Perspectives on a Hybrid Model HOW Can We Solve Them? Conclusion and Discussion By extracting Excitation profiles from visual textures with Adaptive resolution [Picard et al., VRIPHYS 08] C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 9
  • 15. t Method for Impact Sounds Sound and Virtuality Physics-Based Synthesis Contact Modeling Audio Texture Synthesis For Complex Contacts Resonator Modeling A Robust and Multi-Scale Modal Analysis Example-Based Synthesis Perspectives on a Hybrid Model Conclusion and Discussion C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 10
  • 16. t Method for Continuous Contact Sounds Sound and Virtuality Extraction of Excitation Profiles Physics-Based Synthesis Contact Modeling Audio Texture Synthesis For Complex Contacts Resonator Modeling A Robust and Multi-Scale Modal Analysis Example-Based Synthesis Perspectives on a Hybrid Model Conclusion and Discussion C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 11
  • 17. t Synthesis of Excitation Profiles Sound and Virtuality For the Audio Force Modelling Physics-Based Synthesis Contact Modeling Audio Texture Synthesis For Complex Contacts Technique Resonator Modeling A Robust and Multi-Scale Modal Analysis Extraction from the visual texture image Example-Based Re-sampling along the trajectory Synthesis of the contact interaction (60Hz vs 44kHz) Perspectives on a Hybrid Model Conclusion and Based on the Complexity of the Histogram Discussion Simple texture image: Gradient of the image intensity Complex texture image: Isocurves of constant brightness (isophotes) C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 12
  • 18. t Complex Textures Sound and Virtuality Coding the Excitation Profiles Physics-Based Synthesis Contact Modeling Audio Texture Synthesis For Complex Isophotes = Large amount of data Contacts Resonator Modeling How Can We Lighten the Info? A Robust and Multi-Scale Modal Analysis Example-Based By Coding the Excitation Profiles Synthesis = Main Features + Noise Part Perspectives on a Hybrid Model Conclusion and Discussion = + Noise Part: Statistical approximation C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 13
  • 19. t Real-Time Audio Management Sound and Virtuality A Flexible Audio Pipeline Physics-Based Synthesis Contact Modeling Audio Texture Synthesis For Complex Contacts Resonator Modeling Simulations Driven by Ageia’s PhysX (now NVIDIA) A Robust and Multi-Scale Modal Analysis Example-Based Synthesis Perspectives on a Hybrid Model Conclusion and Discussion C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 14
  • 20. t Audio Texture Synthesis Sound and Virtuality A Solution for Interactive Simulations Physics-Based Synthesis Contact Modeling Audio Texture Synthesis For Complex Contacts Resonator Modeling A Robust and Multi-Scale Modal Analysis Example-Based A Sound in Coherence with Visuals Synthesis Perspectives on a Hybrid Model Flexible Resolution Conclusion and Discussion Adapted to Procedural Generation C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 15
  • 21. t Overview Sound and Virtuality Physics-Based Synthesis 1 Sound and Virtuality Contact Modeling Audio Texture Synthesis For Complex Contacts 2 Physics-Based Sound Synthesis Resonator Modeling Contact Modeling A Robust and Multi-Scale Modal Analysis Resonator Modeling Example-Based Synthesis 3 Example-Based Synthesis Perspectives on Flexible Sound Synthesis a Hybrid Model Conclusion and 4 Perspectives on a Hybrid Model Discussion Motivation and Application 5 Conclusion and Discussion Contributions Extensions and Applications C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 16
  • 22. t Vibration Models Sound and Virtuality Modal Analysis Physics-Based Synthesis Contact Modeling Audio Texture Synthesis For Complex Contacts Generating Sounds Based on Physics Simulation Resonator Modeling A Robust and Multi-Scale Modal In computer musics Analysis [Iovino et al. 97, Cook 02] Example-Based Synthesis In computer graphics Perspectives on [Van Den Doel 01, O Brien et al. 02] a Hybrid Model Conclusion and Discussion Improvements for Interactive Sound Rendering Modal parameter tracking [Maxwell et al. 07] Frequency content sparsity [Bonneel et al. 08] C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 17
  • 23. t Vibration Models Sound and Virtuality Modal Analysis Physics-Based Synthesis Contact Modeling Audio Texture 1 Get a Sounding Object and its Geometry Synthesis For Complex Contacts Resonator Modeling A Robust and Multi-Scale Modal Analysis Example-Based Synthesis 2 Construct the FEM (ex: Tetrahedral Mesh) Perspectives on a Hybrid Model 3 Apply Newton Second Law to DOF Conclusion and Discussion ¨ ˙ M d + C d + Kd = f (1) 4 Eigendecomposition ⇒ Modal Parameters M = LL−T ; L−1 KL−T = V ΛV T (2) where V = matrix of eigenvectors Λ = diagonal matrix of eigenvalues C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 18
  • 24. t Vibration Models Sound and Virtuality Modal Analysis Physics-Based Synthesis Contact Modeling Audio Texture Synthesis For Complex Contacts In Real-time: Resonator Modeling A Robust and Modal synthesis Multi-Scale Modal Analysis Example-Based Synthesis Perspectives on a Hybrid Model Conclusion and Discussion 1 s(t) = ai sin(wi t)e −di t (3) n Control for vibration models C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 19
  • 25. t Vibration Models Sound and Virtuality Modal Analysis Physics-Based Synthesis Contact Modeling Audio Texture What Are Synthesis For Complex Contacts The Current Limitations? Resonator Modeling A Robust and Multi-Scale Modal Meshing is difficult Analysis Example-Based No real control on the FEM resolution Synthesis No clear interface between physics and audio Perspectives on a Hybrid Model Conclusion and Discussion C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 20
  • 26. t Vibration Models Sound and Virtuality Modal Analysis Physics-Based Synthesis Contact Modeling Audio Texture What Are Synthesis For Complex Contacts The Current Limitations? Resonator Modeling A Robust and Multi-Scale Modal Meshing is difficult Analysis Example-Based No real control on the FEM resolution Synthesis No clear interface between physics and audio Perspectives on a Hybrid Model Conclusion and Discussion HOW Can We Solve Them? By proposing A robust and multi-scale modal analysis which is Coherent with the physics simulation [Picard et al., DAFx 09] C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 20
  • 27. t Our Deformation Model Sound and Virtuality Physics-Based Synthesis Inspired from Work by Nesme et al. Contact Modeling Audio Texture [Nesme et al. 06] Synthesis For Complex Contacts Resonator Modeling A Robust and Technique Multi-Scale Modal Analysis Merged voxels used as Hexahedral Finite Elements Example-Based Synthesis Perspectives on a Hybrid Model Conclusion and Discussion Implementation with the Sofa Framework Validation of the Model Tests on a metal cube C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 21
  • 28. t Robustness Sound and Virtuality Physics-Based Synthesis Contact Modeling Audio Texture Robust Even for Non-Manifold Geometries Synthesis For Complex Contacts Resonator Modeling A Robust and Multi-Scale Modal Analysis Example-Based Synthesis Perspectives on a Hybrid Model Conclusion and Discussion Material: Aluminium C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 22
  • 29. t Multi-Scale for Efficient Memory Usage Sound and Virtuality Physics-Based A Squirrel in Pine Wood Synthesis Contact Modeling Audio Texture Synthesis For Complex Contacts Resonator Modeling A Robust and Multi-Scale Modal Analysis Example-Based Synthesis Perspectives on a Hybrid Model Conclusion and Discussion C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 23
  • 30. t Multi-Scale for Efficient Memory Usage Sound and Virtuality Physics-Based A Squirrel in Pine Wood: Different FE resolutions Synthesis Contact Modeling Audio Texture 3x3x3 4x4x4 8x8x8 9x9x9 Synthesis For Complex Contacts Resonator Modeling A Robust and Multi-Scale Modal Analysis Example-Based Synthesis Perspectives on a Hybrid Model Frequency Content = f (Hexahedral FE Resolution) Conclusion and Discussion Higher resolution models Frequency centroid shift Convergence of the frequency content C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 24
  • 31. t Comparison with Classical Approach Sound and Virtuality Physics-Based Sounding Bowl - Material: Aluminium Synthesis Contact Modeling Audio Texture Classical Approach Our Approach Synthesis For Complex Contacts (816 modes) (75 modes) Resonator Modeling A Robust and Multi-Scale Modal Analysis Example-Based Synthesis Perspectives on a Hybrid Model Conclusion and Discussion C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 25
  • 32. t A Robust and Multi-Scale Modal Analysis Sound and Virtuality A Solution for Sound Synthesis Physics-Based Synthesis Contact Modeling Audio Texture Synthesis For Complex Contacts Resonator Modeling A Robust and Multi-Scale Modal Analysis Example-Based Realistic Synthesis Perspectives on a Hybrid Model Adapted to Non-Manifold Geometries Conclusion and Discussion Resources Flexibility C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 26
  • 33. t Overview Sound and Virtuality Physics-Based Synthesis 1 Sound and Virtuality Example-Based Synthesis 2 Physics-Based Sound Synthesis Flexible Sound Synthesis Contact Modeling Retargetting Example Sounds Resonator Modeling Perspectives on a Hybrid Model 3 Example-Based Synthesis Conclusion and Discussion Flexible Sound Synthesis 4 Perspectives on a Hybrid Model Motivation and Application 5 Conclusion and Discussion Contributions Extensions and Applications C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 27
  • 34. t Implementation of Signal-Based Models Sound and Virtuality Physics-Based Synthesis Example-Based Synthesis Flexible Sound Concatenative Synthesis Synthesis Retargetting Example [Roads 91, Schwarz 06] Sounds Perspectives on Sound Textures Based on Physics a Hybrid Model [Cook 99] Conclusion and Discussion [Dobashi et al. 03, Zheng et al. 09] Dobashi et al. 03 Authoring and Interactive Control [Cook 02] Cook 99 C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 28
  • 35. t Implementation of Signal-Based Models Sound and Virtuality Physics-Based Synthesis Example-Based Synthesis What Are Flexible Sound Synthesis The Current Limitations? Retargetting Example Sounds Processing is not generic Perspectives on a Hybrid Model Parametrizing is difficult Conclusion and Discussion C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 29
  • 36. t Implementation of Signal-Based Models Sound and Virtuality Physics-Based Synthesis Example-Based Synthesis What Are Flexible Sound Synthesis The Current Limitations? Retargetting Example Sounds Processing is not generic Perspectives on a Hybrid Model Parametrizing is difficult Conclusion and Discussion HOW Can We Solve Them? By Retargetting example sounds To physics-driven animation [Picard et al., AES 09] C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 29
  • 37. t Our Approach Sound and Virtuality Physics-Based Synthesis SINUSOIDAL AUDIO RECORDING Example-Based TRANSIENT Synthesis OBJECT GEOMETRY VIRTUAL ENVIRONMENT Flexible Sound 1 DICTIONARY OF AUDIO GRAINS Synthesis Impulsive / Continuous Retargetting Example BUILD COLLISION Sounds STRUCTURES Perspectives on 2 CORRELATION PATTERNS a Hybrid Model DEFINE PROCEDURES PREPROCESSING INTERACTIVE Conclusion and RETARGETTING RIGID-BODY Discussion TO ANIMATION SIMULATION AUDIO RENDERER VIDEO RENDERER ANIMATION WITH AUDIO Amplitude Time Our Contributions C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 30
  • 38. t Preprocess: A Generic Analysis Sound and Virtuality Physics-Based Synthesis Example-Based Synthesis Flexible Sound Synthesis Impulsive and Continuous Contacts Retargetting Example Sounds Spectral Modeling Synthesis (SMS) [Serra 97] Perspectives on a Hybrid Model Conclusion and Automatic Extraction of Audio Grains Discussion Dictionary: Impulsive/Continuous Generation of Correlation Patterns between original recordings and audio grains C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 31
  • 39. t On-Line: Flexible Sound Synthesis Sound and Virtuality Physics-Based Synthesis Example-Based Synthesis Resynthesis of the Original Recordings Flexible Sound Synthesis Candidate grains: max. correlation amplitude Retargetting Example Sounds Perspectives on a Hybrid Model Conclusion and Discussion C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 32
  • 40. t On-Line: Flexible Sound Synthesis Sound and Virtuality Physics-Based Synthesis Example-Based Synthesis Resynthesis of the Original Recordings Flexible Sound Synthesis Candidate grains: max. correlation amplitude Retargetting Example Sounds Perspectives on Interactive Physics-Driven Animations a Hybrid Model Physics Info for Retargetting Conclusion and Discussion Contact type: impulsive or continuous? Penetration force and relative velocity C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 32
  • 41. t On-Line: Flexible Sound Synthesis Sound and Virtuality Physics-Based Synthesis Example-Based Synthesis Resynthesis of the Original Recordings Flexible Sound Synthesis Candidate grains: max. correlation amplitude Retargetting Example Sounds Perspectives on Interactive Physics-Driven Animations a Hybrid Model Physics Info for Retargetting Conclusion and Discussion Contact type: impulsive or continuous? Penetration force and relative velocity Flexible Audio Shading Approach Additional, User-defined Resynthesis Schemes Spectral domain adaptation/modification C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 32
  • 42. t Resynthesis of the Original Recordings Sound and Virtuality Physics-Based Synthesis Example-Based Synthesis Flexible Sound 94 recordings (14.6Mb) Synthesis ≈ 5000 grains + 94 Correlation Patterns (20% Gain) Retargetting Example Sounds Perspectives on Breaking Glass a Hybrid Model Conclusion and Shooting Gun Discussion Rolling Additional Material: http://www-sop.inria.fr/members/Cecile.Picard/ "‘Supplemental AES"’ C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 33
  • 43. t Flexible Audio Shading Approach Sound and Virtuality Physics-Based Synthesis Example-Based Synthesis Easy Implementation of Time-Scaling Flexible Sound Synthesis Retargetting Example Faster Rolling Sounds Perspectives on Slower Breaking a Hybrid Model Conclusion and Discussion Synthesis of An Infinity Similar Audio Events by varying the audio content Rythmic pattern from Breaking Stone New material content: stone and gun Rythmic pattern from Breaking Glass New material content: ceramic C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 34
  • 44. t Interactive Physics-Driven Animations Sound and Virtuality Physics-Based Synthesis Example-Based Synthesis Flexible Sound Simulations Driven by Sofa Framework Synthesis Retargetting Example Sounds Perspectives on a Hybrid Model Conclusion and Discussion C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 35
  • 45. t Retargetting Example Sounds Sound and Virtuality A Solution for Interactive Simulations Physics-Based Synthesis Example-Based Synthesis Flexible Sound Synthesis Retargetting Example Variety Sounds Perspectives on a Hybrid Model Adapted to Scenarios Conclusion and Discussion Small Memory Footprint Real-Time Rendering An attractive solution for industrial applications (Eden Games, an ATARI game studio) C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 36
  • 46. t Overview Sound and Virtuality Physics-Based Synthesis 1 Sound and Virtuality Example-Based Synthesis 2 Physics-Based Sound Synthesis Perspectives on a Hybrid Model Contact Modeling Motivation Resonator Modeling A Hybrid Model for Fracture Events Conclusion and 3 Example-Based Synthesis Discussion Flexible Sound Synthesis 4 Perspectives on a Hybrid Model Motivation and Application 5 Conclusion and Discussion Contributions Extensions and Applications C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 37
  • 47. t Sound Modeling Sound and Virtuality When Nonlinearity Occurs Physics-Based Synthesis Example-Based Synthesis Perspectives on a Hybrid Model Motivation Problems of Single Models A Hybrid Model for Fracture Events Vibration models assume linearity Conclusion and Discussion Example-based sounds are hard to parametrize C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 38
  • 48. t Sound Modeling Sound and Virtuality When Nonlinearity Occurs Physics-Based Synthesis Example-Based Synthesis Perspectives on a Hybrid Model Motivation Problems of Single Models A Hybrid Model for Fracture Events Vibration models assume linearity Conclusion and Discussion Example-based sounds are hard to parametrize Previous Work Modeling nonlinearities [O Brien et al. 01, Chadwick et al. 09] [Cook 02] C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 38
  • 49. t Fracture Events Sound and Virtuality Physics-Based Synthesis Example-Based Background Synthesis Frequently occur in virtual environments Perspectives on a Hybrid Model Motivation Visual rendering A Hybrid Model for Fracture Events [O Brien et al. 99, 02] Conclusion and [Parker and O Brien. 09] Discussion Sound rendering: Little research [Warren et al. 84] [Rath et al. 03] C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 39
  • 50. t Fracture Events Sound and Virtuality Physics-Based Synthesis Example-Based Background Synthesis Frequently occur in virtual environments Perspectives on a Hybrid Model Motivation Visual rendering A Hybrid Model for Fracture Events [O Brien et al. 99, 02] Conclusion and [Parker and O Brien. 09] Discussion Sound rendering: Little research [Warren et al. 84] [Rath et al. 03] Challenges Event depends on the material involved Differents phases emerge from fracture event C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 39
  • 51. t Parametrization of Our Hybrid Model Sound and Virtuality Physics-Based Synthesis Example-Based Synthesis Selection Criteria Perspectives on Hybrid model applied when nonlinearity occurs a Hybrid Model Motivation A Hybrid Model for Fracture Events Techniques Conclusion and Discussion FM synthesis Audio grains FM synthesis Parametrization Smooth transition with vibration model Coherence inside the hybrid model C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 40
  • 52. t Discussion Sound and Virtuality Physics-Based Synthesis Example-Based Synthesis Perspectives on a Hybrid Model Motivation Prospective model A Hybrid Model for Fracture Events Conclusion and Possible problem: report from the physics engine Discussion Simplicity of the tools allows real-time rendering C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 41
  • 53. t Overview Sound and Virtuality Physics-Based Synthesis 1 Sound and Virtuality Example-Based Synthesis 2 Physics-Based Sound Synthesis Perspectives on a Hybrid Model Contact Modeling Conclusion and Resonator Modeling Discussion Contributions 3 Example-Based Synthesis Extensions and Applications Flexible Sound Synthesis 4 Perspectives on a Hybrid Model Motivation and Application 5 Conclusion and Discussion Contributions Extensions and Applications C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 42
  • 54. t Synthesis of Sounds for Animation Sound and Virtuality Difficulties Physics-Based Synthesis Example-Based Synthesis Perspectives on a Hybrid Model Conclusion and Audio-Visual Coherence Discussion Contributions Extensions and Applications Extremely Dynamic Character Precision of Synthesis Large Variety of Objects C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 43
  • 55. t Contributions Sound and Virtuality An Overview Physics-Based Synthesis Example-Based Complex Contact Modeling Synthesis Perspectives on 2D visual textures used as roughness maps a Hybrid Model Audible and position-dependent variations Conclusion and Detail-layer mechanisms Discussion Contributions Extensions and Applications C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 44
  • 56. t Contributions Sound and Virtuality An Overview Physics-Based Synthesis Example-Based Complex Contact Modeling Synthesis Perspectives on 2D visual textures used as roughness maps a Hybrid Model Audible and position-dependent variations Conclusion and Detail-layer mechanisms Discussion Contributions Extensions and Applications Improved Modal Analysis for Resonator Modeling Complex non-manifold geometries can be handled Multi-scale resolution Coherence between simulation and audio C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 44
  • 57. t Contributions Sound and Virtuality An Overview Physics-Based Synthesis Example-Based Complex Contact Modeling Synthesis Perspectives on 2D visual textures used as roughness maps a Hybrid Model Audible and position-dependent variations Conclusion and Detail-layer mechanisms Discussion Contributions Extensions and Applications Improved Modal Analysis for Resonator Modeling Complex non-manifold geometries can be handled Multi-scale resolution Coherence between simulation and audio Flexibility of Sound Design Audio grains and correlation patterns Dynamic retargetting to events Extended sound synthesis capabilities C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 44
  • 58. t Contributions Sound and Virtuality Perspectives Physics-Based Synthesis Example-Based Synthesis Perspectives on a Hybrid Model Conclusion and Discussion A Prospective Hybrid Model Contributions for Complex Physical Phenomena Extensions and Applications Focus on Nonlinearity Combination of physically based and example-based methods Application Case: Fracture Events C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 45
  • 59. t Overview Sound and Virtuality Physics-Based Synthesis 1 Sound and Virtuality Example-Based Synthesis 2 Physics-Based Sound Synthesis Perspectives on a Hybrid Model Contact Modeling Conclusion and Resonator Modeling Discussion Contributions 3 Example-Based Synthesis Extensions and Applications Flexible Sound Synthesis 4 Perspectives on a Hybrid Model Motivation and Application 5 Conclusion and Discussion Contributions Extensions and Applications C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 46
  • 60. t Promising Directions for Future Work Sound and Virtuality Physics-Based Synthesis Complex Contact Modeling Example-Based Synthesis Two interacting textures Perspectives on Surface-based interactions a Hybrid Model Adequate perceptual experiments Conclusion and Discussion Contributions Extensions and Applications C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 47
  • 61. t Promising Directions for Future Work Sound and Virtuality Physics-Based Synthesis Complex Contact Modeling Example-Based Synthesis Two interacting textures Perspectives on Surface-based interactions a Hybrid Model Adequate perceptual experiments Conclusion and Discussion Contributions Improved Modal Analysis for Resonator Modeling Extensions and Applications Recent work from [Nesme et al. Siggraph 09] Investigations with GPU for in-line computation Complete integration in a virtual scene C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 47
  • 62. t Promising Directions for Future Work Sound and Virtuality Physics-Based Synthesis Complex Contact Modeling Example-Based Synthesis Two interacting textures Perspectives on Surface-based interactions a Hybrid Model Adequate perceptual experiments Conclusion and Discussion Contributions Improved Modal Analysis for Resonator Modeling Extensions and Applications Recent work from [Nesme et al. Siggraph 09] Investigations with GPU for in-line computation Complete integration in a virtual scene Example-Based Technique Clustering of similar grains Statistical analysis of correlation patterns Physics engine design C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 47
  • 63. t Promising Directions for Future Work Sound and Virtuality Physics-Based Synthesis Example-Based Synthesis Perspectives on a Hybrid Model Conclusion and Discussion Hybrid Model for Fracture Events Contributions Extensions and Fracture sound simulation framework Applications Tracking of relevant physical data C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 48
  • 64. t Conclusion Sound and Virtuality Physics-Based Synthesis Example-Based Synthesis Perspectives on a Hybrid Model New Physically Based Algorithms Conclusion and Discussion for Sound Rendering Contributions Extensions and Flexibility of Sound Modeling Applications Ideas on an Adequate Hybrid Sound Model Additional info: http://www-sop.inria.fr/members/Cecile.Picard/ C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 49
  • 65. t Acknowledgements Sound and Virtuality Physics-Based Synthesis Example-Based Synthesis Perspectives on a Hybrid Model George Drettakis, François Faure, Conclusion and and Nicolas Tsingos Discussion Contributions REVES Team Extensions and Applications Marie-Paule Cani and the Evasion Team Paul G. Kry at the McGill University, Montréal Eden Games, an ATARI game studio, Lyon C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 50