QA Fest 2018. Александр Хотемский. Использование голосовых помощников для разработки и тестирования
- 1. Voice assistants What it might help us with testing and development?
- 2. Oleksandr Khotemskyi Independent Contractor Software Developer Engineer in Test xotabu4.github.io
- 3. What is voice assistants? And how do they work?
- 4. Voice assistants in nutshell • Always listening microphone connected to voice recognition service • Service understands different languages, and context • Lots of ready to use reactions to user input, and can be extended by custom, own reactions • Made possible due to great progress of voice understanding algorithms
- 5. TL DR : just another interface to communicate with computer
- 7. Why only now? • First prototypes had already appeared in the late sixties • Unrealistic expectations from users lead to disappointment and AI winter of the 70s- 80s • Huge performance of modern computers allows to apply machine learning algorithms for natural language processing • Biggest companies in the world started to invest money, and prepared consumer- ready solutions
- 8. Usage examples How sales see use cases
- 12. Extending with own responses How developers might extend Google Assistant
- 14. Components • Google Assistant - virtual assistant powered by artificial intelligence. Can engage in two-way conversations • Actions on Google is platform to extend Google Assistant with own reactions • DialogFlow - an end-to-end development suite for conversational interfaces (e.g., chatbots, voice- powered apps and devices)
- 18. DialogFlow • Works as middleware between user intents and your responses to them • Parses inputs, extracts parameters, prepares entities with needed data • Integrates to different services, has API, and SDKs • Has training mechanism • Supports context, fallbacks, events
- 21. Webhook that calls your server
- 22. Limitations • The communication is one direction at a time. Kind of “push-to-talk”. Interface limitation • “Push notifications aren't currently supported on voice-activated speakers” (September 2018). google-home-notifier package can be used, but requires listening server in local network • Hard to pass complex parameters. Works best with single words and numbers • For best results - visual representation will be needed (smartphone, tv, smart display)
- 23. Usage Examples in development How it might be useful in everyday work
- 24. Launching CI/CD jobs/builds • Voice commands can invoke with HTTP API (tested with Jenkins/DroneCI) • Usually client libraries exist for most popular CI/CD systems • You will need API key. Be sure you are using it securely • You can pass parameters for jobs using dialogflow
- 25. Working with JIRA tickets • Reading status of JIRA tickets • Updating tickets • Leaving comments • Getting statuses of sprints, boards
- 26. Health/status checks • Notifying on service death • Notifying on resources overconsumption • Custom metrics change notifying • Asking to check server status • Should be working nice with Google Cloud Platform + Google Cloud Functions
- 27. Notifying • Uses push notifications, but requires server in local network • Fresh build released • Branch merged • Tests failed • Code freeze started • Meeting is in 5 minutes! • Whatever notification you want
- 28. Is there advantages? Why this might be useful?
- 29. Improving chatbots • Voice assistant can greatly improve your chatbots • Can replace or cooperate with chatbot • Response to voice request can be sent via chatbot response
- 30. New communication channel • Voice communication might be easier in specific cases • No context switching (No need to switch windows, write commands with hands) • Faster - no need to open laptop, login … • My mom liked it
- 31. Doubts • Might be time-wasting • Additional support for code that not directly used in project • Might be hard to provide wide range of useful commands
- 32. Materials • Human-Computer interactions: https://en.wikipedia.org/wiki/Human%E2%80%93computer_interaction • What is Google Actions: https://developers.google.com/actions/extending-the-assistant • Starting with Google Actions: https://codelabs.developers.google.com/codelabs/actions-1/#0 • Using in office: https://www.zdnet.com/article/five-ways-voice-assistants-are-going-to-change-the-o • Intonations in voice: https://developers.google.com/actions/reference/ssml
- 33. Ask your questions now xotabu4.github.io