Speech Recognition as a User Interface
Download as PPTX, PDF4 likes1,116 views
The document discusses speech recognition as a user interface. It begins with an introduction of the speaker and an overview of what will be covered, including a definition of speech recognition, examples of what it can be used for, and a demonstration. The document then discusses how speech recognition works from an technical perspective involving acoustic signal processing and various machine learning techniques. It covers aspects like speaker dependence and independence as well as continuous speech recognition. The document discusses moving beyond basic dictation to understand a user's intent in order to take appropriate actions. It provides examples of speech recognition systems and APIs from companies like Google, Apple, Amazon and Microsoft. It concludes with code examples and contact information for the speaker.
Speech Recognition as a User Interface
- 1. PAGE1 © 2015 Apio Systems, Inc. Confidential 1 Jared Sheehan @ Driversiti Speech Recognition as a User Interface
- 2. PAGE2 © 2015 Apio Systems, Inc. Confidential 2 Who am I Glass explorer, speech recognition enthusiast and big android nerd Android Lead @Driversiti - driving safety for the mobile generation Speech Recognition application for the Amazon Fire Phone Suite of applications - AIM Android, Engadget Android, Distro Android, TechCrunch Android, AOL HD, AIM Blackberry Meetup evangelist – “DC Android Meetup Group” – Join today!
- 3. PAGE3 © 2015 Apio Systems, Inc. Confidential 3 Overview What is voice/speech recognition? What awesome stuff you can do with it? How it works… Demo! Question and Answer
- 4. PAGE4 © 2015 Apio Systems, Inc. Confidential 4 Hello Computer…
- 5. PAGE5 © 2015 Apio Systems, Inc. Confidential 5 Definition
- 6. PAGE6 © 2015 Apio Systems, Inc. Confidential 6 What can you do with SR? Technology that allows spoken input into software systems. You speak to your computer, tablet, phone or device and it uses what you said as input to trigger some sort of action. Replace other methods of input like clicking, swiping, typing or selecting in other ways. It is a means to make devices and software more user-friendly and to increase productivity. It is used extensively as a form of accessibility assistance.
- 7. PAGE7 © 2015 Apio Systems, Inc. Confidential 7 ASR - Dictation Automatic speech recognition (ASR) also called Dictation Translates speech input into words, sentences and punctuation. Audio is input through a microphone and streamed somewhere The result is usually returned as a string with a confidence level Very easy integration with Android – 2 ways to do it.
- 8. PAGE8 © 2015 Apio Systems, Inc. Confidential 8 How does it work? A user speaks into a recording device of some sort Speech recognition begins with the digital sampling of speech and then acoustic signal processing of the audio. Several processes including DTW (Dynamic time warping), HMM (Hidden Markov models) and NN’s (Neural Networks) can achieve the desired results Most systems use language specific knowledge to tune the models. Next is the actual recognition of phonemes, groups of phonemes and words
- 9. PAGE9 © 2015 Apio Systems, Inc. Confidential 9 Speech Recognition system architecture
- 10. PAGE10 © 2015 Apio Systems, Inc. Confidential 10 Into the weeds Speaker dependence Speaker independence Continuous Speech How good is your system? Hint: Word Error Rate Isolated word Is that all it does??
- 11. PAGE11 © 2015 Apio Systems, Inc. Confidential 11 Dictation is cool, but not that cool Next step is understanding what the user wants to do Then act on it Generally, the ASR results are passed into an Intent recognition system with additional information Contextual information can be, where the utterance is coming from (mobile phone, computer), what app they are using, location etc. That information is used to determine the user’s intent and execute the request.
- 12. PAGE12 © 2015 Apio Systems, Inc. Confidential 12 Intent recognition Recognizing speech is only part of the process. How does Google Now know that I want to send an SMS message to a friend? How does Siri know when I want to know how tall Kobe Bryant is? ASR is only the first step in true Speech as a user interface. To successfully help users perform useful actions we must understand their intent. How to do this? Three systems; ASR, Intent Recognition and a Dialog Engine The Dialog engine takes the output from the IR system and sends responses and actionable information to the caller.
- 13. PAGE13 © 2015 Apio Systems, Inc. Confidential 13 Android Speech APIs
- 14. PAGE14 © 2015 Apio Systems, Inc. Confidential 14 Android Speech APIs http://developer.android.com/reference/android/speech/package-summary.html Relatively easy implementation <uses-permission android:name="android.permission.RECORD_AUDIO" /> A UI and no UI API InputMethodServices use the no UI version - Keyboards
- 15. PAGE15 © 2015 Apio Systems, Inc. Confidential 15 Recognizer Intent UI is supplied for you Fire the intent and get a result Again very easy to use
- 16. PAGE16 © 2015 Apio Systems, Inc. Confidential 16 SpeechRecognizer UI is not supplied for you Results are streamed directly to the EditText Still “fairly” easy to use
- 17. PAGE17 © 2015 Apio Systems, Inc. Confidential 17 Google Now – Onto Intent recognition systems…
- 18. PAGE18 © 2015 Apio Systems, Inc. Confidential 18 Google Now – On tap
- 19. PAGE19 © 2015 Apio Systems, Inc. Confidential 19 Apple – Siri
- 20. PAGE20 © 2015 Apio Systems, Inc. Confidential 20 Amazon – Fire phone, Fire Tv and Echo
- 21. PAGE21 © 2015 Apio Systems, Inc. Confidential 21 Microsoft – Cortana
- 22. PAGE22 © 2015 Apio Systems, Inc. Confidential 22 Speech providers – Google, Nuance, IBM Watson
- 23. PAGE23 © 2015 Apio Systems, Inc. Confidential 23 Google Voice Interaction API
- 24. PAGE24 © 2015 Apio Systems, Inc. Confidential 24 Nuance Speech SDK Dragon Mobile – SDK – Free up to 20k transactions per/month Upload custom vocabularies Developer: Uploads a new song and music vocabulary Utterance: “Eminem” higher probability then “M&M”
- 25. PAGE25 © 2015 Apio Systems, Inc. Confidential 25 User Interface examples - Google Glass
- 26. PAGE26 © 2015 Apio Systems, Inc. Confidential 26 User Interface examples - Google Glass continued…
- 27. PAGE27 © 2015 Apio Systems, Inc. Confidential 27 User Interface examples - Google Glass continued…
- 28. PAGE28 © 2015 Apio Systems, Inc. Confidential Enough talk!
- 29. PAGE29 © 2015 Apio Systems, Inc. Confidential Show me code!
- 30. PAGE30 © 2015 Apio Systems, Inc. Confidential jared.sheehan@driversiti.com http://www.meetup.com/DCAndroid/ Tweet: @jayroo5245 THANK YOU
Editor's Notes
- #17: What else is there?