Dynamic Sound for iOS Apps and Games
3 likes2,953 views
This document discusses sound design for mobile apps and games. It introduces Pure Data (Pd), an open-source visual programming environment for procedural audio generation. It also introduces libpd, which allows embedding a Pd instance within iOS and Android apps. Two example apps are provided: a simple keyboard app and a space shooter game app enhanced with dynamic sound effects created using Pd patches. The document provides an overview of sound design principles and techniques for mobile as well as tutorials on basic Pd functionality and libpd integration.
![360 iDev tackmobile.com
Messages
• Used for sending values without a connection
• [send] and [receive] objects
• Message boxes](https://image.slidesharecdn.com/dynamicsoundios-130910123042-phpapp02/85/Dynamic-Sound-for-iOS-Apps-and-Games-39-320.jpg)
Dynamic Sound for iOS Apps and Games
- 1. Dynamic Audio For Apps and Games
- 2. 360 iDev tackmobile.com About Me • @thillerson, +thillerson (if you’re into that) • Developer at Tack Mobile (tackmobile.com) • iOS, Android, and Mobile Web • Rails, Node, maybe Elixir (one day)
- 3. 360 iDev tackmobile.com Rawk Star About Me
- 4. 360 iDev tackmobile.com About Me • Bass • Electronic Music • Some day I hope to play this thing --------->
- 5. 360 iDev tackmobile.com Sound Design Work: Start • Alarm App • Alarm Sounds • App Store: “Start by Tack”
- 6. 360 iDev tackmobile.com Sound Design Work: Adrift • Puzzle Game - “Adrift by Tack” in App Store • Background Music • Sound Effects
- 7. 360 iDev tackmobile.com Programming Sound • In Beta Soon* (*this week) • http://pragprog.com/ • What we’re talking about today... • ... and much more
- 8. 360 iDev tackmobile.com What We’re Going To Do Today • Answer: What Is Sound Design? • Discuss UX Principles of Sound Design • Introduce Pure Data • Introduce libpd • Tour Two iOS Apps Embedding Pd
- 9. 360 iDev tackmobile.com When We Are Done, You Will • Understand Sound Design • Understand Procedural Audio • Know what Pd is, and how libpd makes it awesome • Be excited about Dynamic Audio in your apps and games
- 10. 360 iDev tackmobile.com What is Sound Design?
- 11. 360 iDev tackmobile.com A Sound Designer Ben Burtt
- 12. 360 iDev tackmobile.com A Sound Designer... • Conceptualizes • Captures or Designs • Records • Engineers • Produces • ... the sound for an experience
- 13. 360 iDev tackmobile.com Sound Designer Skills • Understand Sound • Intuitive vs. Theoretical • Music vs. Sound • Audio Engineering
- 14. 360 iDev tackmobile.com Sound Design in Mobile Apps • Developers/Designers wear lots of hats • Control • Efficiency • Asset size
- 15. 360 iDev tackmobile.com Sound Design Principles
- 16. 360 iDev tackmobile.com A Sound Designer’s Goals • Audible Feedback • Fulfill Expectations • Immersion • Emotional Response
- 17. 360 iDev tackmobile.com Sound Design and Good UX • Your app doesn’t need sound • Expect to be turned off • Don’t make me listen to you • Listen on lots of devices/environments • Engineer for small speakers • Make your sound mean something!
- 18. 360 iDev tackmobile.com Always • Test with phone calls • Test while playing music • Have fallbacks - e.g. No music, just effects • Provide control to the user
- 19. 360 iDev tackmobile.com A Good Example: Clear • Realmac Software • Josh Mobley • Meaningful Sound • demo
- 21. 360 iDev tackmobile.com Longitudinal Waves
- 22. 360 iDev tackmobile.com Of Pressure
- 23. 360 iDev tackmobile.com Through A Medium
- 24. 360 iDev tackmobile.com Transverse Wave Longitudinal Wave
- 25. 360 iDev tackmobile.com • Longitudinal Waves • Of Pressure • Through a Medium Sound
- 26. 360 iDev tackmobile.com Properties of Waves • A - Wavelength (Distance), Period (Time) • B - Amplitude • C - Frequency - 1/Period (measured in Hertz) A B B C 1 2 3
- 27. 360 iDev tackmobile.com Fourier Series • Based on his work -------> • Periodic signals composed of simple oscillating functions • Sine/Cosine waves are fundamental
- 28. 360 iDev tackmobile.com Oscillators • Something that moves according to a periodic function • A physical body • An electronic device
- 29. 360 iDev tackmobile.com Basic Electronic Sound oscillator voltage magnet pressure waves speaker
- 30. 360 iDev tackmobile.com Digital Audio • Stream of Numbers • Signal • Frequency and Amplitude • DAC - Digital to Analog Converter
- 31. 360 iDev tackmobile.com Dynamic Audio
- 32. 360 iDev tackmobile.com Procedural Audio • Functions • Objects • Controls and Components • Produce a Signal • “Building Sound from First Principles”
- 33. 360 iDev tackmobile.com Pure Data • Procedural Audio • Visual Programming Environment • Free and Open Source • pure-data.info
- 34. 360 iDev tackmobile.com Pure Data 101
- 35. 360 iDev tackmobile.com Signals and Values • Signals are streams of numbers, usually audio data • Values are numbers or symbols
- 36. 360 iDev tackmobile.com • Pd’s class • or, kind of like a command line tool Objects osc~ 440 signal inlet inlet signal outlet class name arguments
- 37. 360 iDev tackmobile.com Hot vs Cold Inlets • Hot inlets make objects immediately act • Cold inlets usually store a value • Execution is right to left
- 38. 360 iDev tackmobile.com • Signal connections are thick • Others are thin Connections osc~ 440 *~ dac~ 1
- 39. 360 iDev tackmobile.com Messages • Used for sending values without a connection • [send] and [receive] objects • Message boxes
- 40. 360 iDev tackmobile.com libpd • http://libpd.cc • Peter Brinkmann - Google Engineer • Enables you to embed an instance of Pd in your app • iOS and Android
- 42. 360 iDev tackmobile.com Source Code • github.com/thillerson/dynamic-sound-ios • Examples of capabilities, not necessarily the best architecture • Goal: understand capabilities, have a place to start
- 43. 360 iDev tackmobile.com Example 1: A Simple Keyboard • Sine wave oscillator • Single voice • Note on/off
- 44. 360 iDev tackmobile.com Example 1: A Simple Keyboard Patch osc~ dac~ receive midinote mtof receive gate line~ *~ ; midinote 60; gate 1; ; gate 0$1 1 print
- 45. 360 iDev tackmobile.com Example 2: A Game • Top down perspective space shooter • Code by Lucas Jordan from iOS 5 Game Dev -> • Now with sound!
- 46. 360 iDev tackmobile.com Example 2: A Game Patch load a square wave at startup Thruster noise~ *~ samphold~ dac~ hip~ 200 phasor~ 800 line~ $1 200 *~ 0.1 receive thruster BulletShip_HitAsteroid_Hit samphold~ tabosc4~ square dac~ line~ phasor~ 8000 tabosc4~ square line~ unpack f f 0 $1 tabosc4~ square line~ unpack f f 0 $1 hip~ 1000 unpack f f 0 $1 *~ 0.5 receive asteroid receive ship receive bullet 8-bit-ify these sounds cut low frequencies comment comment Test_Messages ; bullet 25 200; ; thruster 0 ; thruster 1 pd wavesetup ; ship 1000 50; ; asteroid 200 100;
- 47. 360 iDev tackmobile.com Further Reading • puredata.info && libpd.cc • youtube.com/user/cheetomoskeeto • http://designingsound.org • mitpress.mit.edu/books/designing-sound
- 49. Thank you! Dynamic Audio • Tony Hillerson • We’re Hiring! • careers@tackmobile.com • Excellent Team • Awesome Projects • Great Office