PhoneGap Day US 2013 - Simon MacDonald: Speech Recognition
2 likes2,926 views
1. The document discusses adding speech recognition and speech synthesis capabilities to web applications using the Web Speech API specifications. 2. It provides code examples for using the SpeechRecognition interface to translate spoken words to text and the SpeechSynthesis interface to output text as synthesized speech. 3. The document demonstrates how to build a simple speech-enabled question and answer application that allows the user to ask questions out loud and receives the answers as synthesized speech.
![Let's recognize some
speech
Click to Speak
hello world
var recognition = new SpeechRecognition();
recognition.onresult = function(event) {
if (event.results.length > 0) {
var test1 = document.getElementById("test1");
test1.innerHTML = event.results[0][0].transcript;
}
};
recognition.start();](https://image.slidesharecdn.com/phonegap-day-speechrecognition-130801035807-phpapp01/85/PhoneGap-Day-US-2013-Simon-MacDonald-Speech-Recognition-17-320.jpg)
![Let's do something a little
less trivial
Click to Speak
recognition.onresult = function(event) {
var result = event.results[0][0].transcript;
var music = document.getElementById("music");
switch(result) {
case "jazz":
music.src="jazz.mp3";
music.play();
break;
case "rock":
music.src="rock.mp3";
music.play();
break;
case "stop":
default:
music.pause();
}
};](https://image.slidesharecdn.com/phonegap-day-speechrecognition-130801035807-phpapp01/85/PhoneGap-Day-US-2013-Simon-MacDonald-Speech-Recognition-23-320.jpg)
PhoneGap Day US 2013 - Simon MacDonald: Speech Recognition
- 1. JavaScript Speech Recognition Applicationsand maybe some other HTML5 goodness
- 2. Who is this guy? core contributor nutty about IBM'er @macdonst macdonst on Github simonmacdonald.com PhoneGap speech recognition
- 3. Why do I care about speech rec? Here's a conversation between two Cape Bretoners P1: jeet? P2: naw, jew? P1: naw, t'rly t'eet bye.
- 4. And here's the translation P1: jeet? P1: Did you eat? P2: naw, jew? P2: No, did you? P1: naw, t'rly t'eet bye. P1: No, it's too early to eat buddy.
- 6. Speech recognition is the process of translating
- 7. the spoken word into text.
- 8. The process of speech rec includes 1. Record and digitize the audio data 2. Split data into phonemes 3. Apply the phonemes to the recognition model 4. Analyze the results against the grammar 5. Return a confidence weighted result
- 9. Basically...
- 11. So how do we add speech rec to our app?
- 12. You may look at the W3C Speech API Specification
- 13. but only Chrome on the desktop has implemented that spec
- 14. But that's okay!
- 15. The spec looks like this: interface SpeechRecognition : EventTarget { // recognition parameters attribute SpeechGrammarList grammars; attribute DOMString lang; attribute boolean continuous; attribute boolean interimResults; attribute unsigned long maxAlternatives; attribute DOMString serviceURI; // methods to drive the speech interaction void start(); void stop(); void abort(); };
- 16. With additional event methods to control behaviour: attribute EventHandler onaudiostart; attribute EventHandler onsoundstart; attribute EventHandler onspeechstart; attribute EventHandler onspeechend; attribute EventHandler onsoundend; attribute EventHandler onaudioend; attribute EventHandler onresult; attribute EventHandler onnomatch; attribute EventHandler onerror; attribute EventHandler onstart; attribute EventHandler onend;
- 17. Let's recognize some speech Click to Speak hello world var recognition = new SpeechRecognition(); recognition.onresult = function(event) { if (event.results.length > 0) { var test1 = document.getElementById("test1"); test1.innerHTML = event.results[0][0].transcript; } }; recognition.start();
- 19. ...if taking dictation gets you going
- 21. But I want to do something more exciting with the
- 22. result
- 23. Let's do something a little less trivial Click to Speak recognition.onresult = function(event) { var result = event.results[0][0].transcript; var music = document.getElementById("music"); switch(result) { case "jazz": music.src="jazz.mp3"; music.play(); break; case "rock": music.src="rock.mp3"; music.play(); break; case "stop": default: music.pause(); } };
- 25. Which seems much cooler to me
- 26. Let's ask the web a question Click to Speak Q: what day is it today A: Friday July 19th, 2013
- 28. Let's style our button with some CSS
- 29. + = <a class="speechinput"> <img src="images/mic.png"> </a> #speechinput input { cursor:pointer; margin:auto; margin:15px; color:transparent; background-color:transparent; border:5px; width:15px; -webkit-transform: scale(3.0, 3.0); }
- 30. And we'll add some color using by Nicholas Gallagher Speech Bubbles Pure-CSS-Speech-Bubbles
- 31. Then pull it all together!
- 32. what is steve jobs middle name Steven Paul Jobs
- 33. But wait, why am I using my eyes like a sucker
- 34. We'll output the answer using SpeechSynthesis
- 35. The SpeechSynthesis spec looks like this: interface SpeechSynthesis { readonly attribute boolean pending; readonly attribute boolean speaking; readonly attribute boolean paused; void speak(SpeechSynthesisUtterance utterance); void cancel(); void pause(); void resume(); SpeechSynthesisVoiceList getVoices(); };
- 36. The SpeechSynthesisUtterance spec looks like this: interface SpeechSynthesisUtterance : EventTarget { attribute DOMString text; attribute DOMString lang; attribute DOMString voiceURI; attribute float volume; attribute float rate; attribute float pitch; };
- 37. With additional event methods to control behaviour: attribute EventHandler onstart; attribute EventHandler onend; attribute EventHandler onerror; attribute EventHandler onpause; attribute EventHandler onresume; attribute EventHandler onmark; attribute EventHandler onboundary;
- 38. who won the stanley cup this year Chicago Blackhawks
- 39. Plugin repo's SpeechRecognitionPlugin - https://github.com/macdonst/SpeechRecognitionPlugin SpeechSynthesisPlugin - https://github.com/macdonst/SpeechSynthesisPlugin
- 40. THE END