Control your Voice like a Bene Gesserit
1 like579 views
The document discusses the development and architecture of voice user interfaces (VUIs), highlighting the transformation of an existing app into an advanced architecture that supports VUI applications. It covers key concepts such as natural language processing and domain routing while providing implementation examples in node.js. Additionally, it emphasizes the importance of a solid architecture for effective VUI design and shares lessons learned from the process.
![@jdortiz
protocol AlexaUIController {
func process(serviceRequest: ASServiceRequest) throws !-> ASServiceResponse
}
class AlexaSkill { !//âŠ
func process(serviceRequest: ASServiceRequest) throws !-> ASServiceResponse {
guard serviceRequest.safetyCheck(id: id) else {
throw SkillError.wrongSkillId
}
switch serviceRequest.request.type {
case .launch:
return serviceRequest.reply(message: welcomeMessage)
case let ASRequestType.intent(data: intentData):
if let controller = controllerMap[intentData.name] {
return try controller.process(serviceRequest: serviceRequest)
} else {
throw SkillError.undefinedIntent
}
case .sessionEnd:
return serviceRequest.end()
}
}
}](https://image.slidesharecdn.com/20180316controlyourvoicelikeabenegesserit-appdevcon-180402190020/85/Control-your-Voice-like-a-Bene-Gesserit-35-320.jpg)
Control your Voice like a Bene Gesserit
- 1. Control your Voice like a Bene Gesserit Jorge D. Ortiz Fuentes @jdortiz
- 3. @jdortiz Agenda â Context â Voice User Interfaces 101 â Advanced Architecture Brief â VUI + Adv. Architecture â Lessons Learned
- 4. Warning
- 5. Sorry!
- 6. Put all your devices in non listening mode
- 7. Some Context
- 8. Living like a 3rd Stage Guild Navigator
- 9. @jdortiz The Request â Existing App â Refactored to an Advanced Architecture â Wanted to explore VUI â Prototype with minimum cost
- 12. @jdortiz Maybe not as good as we wished (YET) â âSorry. I donât know that oneâ âSorry about that. Iâm still learningâ â English, German, French, Japanese (Voice interaction) â Private actions
- 13. The Contenders Alexa Google Home SiriCortana
- 14. â Bene Genesserit âI must not fear. Fear is the mind-killer. Fear is the little-death that brings total obliteration. I will face my fear.â
- 15. The Guts Domain 1 Domain 2 Domain 3 Domain n Routing Natural Language Processing Automatic Speech Recognition Text to Speech Voice ⥠Voice Text Domain & Intent & Parameters Text
- 16. Domain nDomain n Extend the System JSON ⥠JSON HTTPS Parse Request Do Your Stuff Build Response
- 17. @jdortiz Development â Node JS library (Alexa Skills & Google Actions) â Also Alexa SDK for Java(/Kotlin) â Several unofficial ones
- 18. @jdortiz var routes = Routes() routes.add(method: .post, uri: â/") { request, response in defer { response.completed() } if let postString = request.postBodyString, let data = postString.data(using:.utf8) { let reply = AlexaServiceResponse(version: "1.0", response: AlexaResponse(outputSpeech: OutputSpeech(text: "Hello. I am your Genie. How can I help you?", type: .plaintext))) let encoder = JSONEncoder() let responseData = try! encoder.encode(reply) let responseString = String(bytes: responseData, encoding: .utf8) !?? "" response.setHeader(.contentType, value: "application/json") response.setBody(string: responseString) } else { response.badRequest(message: "No request data found.") } }
- 19. @jdortiz var routes = Routes() routes.add(method: .post, uri: â/") { request, response in defer { response.completed() } if let postString = request.postBodyString, let data = postString.data(using:.utf8) { let reply = GAV1ServiceResponse(speech: "Hello. I am your Genie. How can I help you?", data: gaServiceRequest.originalRequest.data) let encoder = JSONEncoder() let responseData = try! encoder.encode(reply) let responseString = String(bytes: responseData, encoding: .utf8) !?? "" response.setHeader(.contentType, value: "application/json") response.setBody(string: responseString) } else { response.badRequest(message: "No request data found.") } }
- 20. OK But⊠Why do I need a good architecture?
- 22. @jdortiz Persistance FW View Network LocationFW Presenter Entity Gateway Clean Architecture Interactor Entity
- 23. Advanced Architecture: Mobile App Delegate / Application View (VC) / Activity Presenter Use Case Entity Gateway Connector Tap on phone button User wants to talk to contact Start call if⊠Get contact details
- 26. Kwisatz Haderach
- 27. Advanced Architecture VUI Server UI Presenter Use Case Entity Gateway Connector Reuse Domain Logic Intent StartCall User wants to talk to contact Start call if⊠Get contact details
- 28. Lessons Learned
- 29. â Dune, Frank Herbert âMuad'Dib learned rapidly because his first training was in how to learn. And the first lesson of all was the basic trust that he could learn⊠Muad'Dib knew that every experience carries its lesson.â
- 31. @jdortiz Implementation Details â Rebuild your Model (Alexa Skills) â Lots of corner cases in Codable â Documentation of the JSON requests / responses could be better
- 32. @jdortiz var routes = Routes() routes.add(method: .post, uri: â/alexa/â, handler: alexaRequestHandler) routes.add(method: .post, uri: â/gaction/â, handler: gactionRequestHandler) let server = HTTPServer() server.addRoutes(routes) server.serverPort = UInt16(httpPort) do { try server.start() } catch PerfectError.networkError(let err, let msg) { LogFile.debug("Network error thrown: (err) (msg)") }
- 33. @jdortiz func gactionRequestHander(request: HTTPRequest, response: HTTPResponse) { defer { response.completed() } LogFile.info("!>> Google Action Service Request received !>>") if let postString = request.postBodyString, let data = postString.data(using:.utf8) { do { let decoder = JSONDecoder() let gaRequest = try decoder.decode(GAV1ServiceRequest.self, from: data) let reply = try gaction.process(serviceRequest: gaRequest) let encoder = JSONEncoder() let responseData = try! encoder.encode(reply) let responseString = String(bytes: responseData, encoding: .utf8) !?? "" LogFile.debug("Response: (responseString)") response.appendBody(string: responseString) response.setHeader(.contentType, value: "application/json") } catch let error as GActionError { response.append(error: error) LogFile.error("Google Action error: (error)â) } catch let error { response.badRequest(message: "Request data is wrong.") LogFile.error("Error decoding request: (error)") } } else { response.badRequest(message: "No request data found.") LogFile.error("No request data found.") } }
- 34. @jdortiz let skill = AlexaSkill(id: appId, welcomeMessage: "Welcome to Family Chores. How can I help you?", logger: GenericLogger()) skill.register(controller: RecordTaskAlexaUIController(), forIntent: "RecordTask") let connector = PersonReportAlexaConnector() skill.register(controller: PersonReportAlexaController(), forIntent: "PersonReport")
- 35. @jdortiz protocol AlexaUIController { func process(serviceRequest: ASServiceRequest) throws !-> ASServiceResponse } class AlexaSkill { !//⊠func process(serviceRequest: ASServiceRequest) throws !-> ASServiceResponse { guard serviceRequest.safetyCheck(id: id) else { throw SkillError.wrongSkillId } switch serviceRequest.request.type { case .launch: return serviceRequest.reply(message: welcomeMessage) case let ASRequestType.intent(data: intentData): if let controller = controllerMap[intentData.name] { return try controller.process(serviceRequest: serviceRequest) } else { throw SkillError.undefinedIntent } case .sessionEnd: return serviceRequest.end() } } }
- 36. Identity & Flow
- 37. @jdortiz Flow â Donât make your users talk their way through â Straight from A -> B
- 38. But wait! There is concurrency
- 40. @jdortiz skill.register(connector = PersonReportAlexaConnector(), forIntent: "PersonReport")
- 41. Fast Prototyping
- 42. @jdortiz Quality Time â Avoid duplication to address both platforms â Ignore the DB â Reuse and extend use cases
- 43. Recap
- 44. @jdortiz Key Takeaways â Having (/Going to) an Advanced Architecture is key â Pros & Cons of own API interface â Easy Prototyping â Speak what you wish /Use what you wish
- 45. Bedankt!
- 46. Thank You!
- 47. @jdortiz