Kranky Geek WebRTC 2015 - A closer look at the WebRTC UX/UI API

Editor's Notes

• #2: Overall Outline: User interaction for real time communications is way different than dealing with typical web content and streaming media. Arin will cover best practices for incorporating WebRTC into your application for a smooth user experience. * Experiences WebRTC provides out of the box (WebRTC provides several key interfaces you can't control: allow/disallow buttons for video/audio/screenshare, icons to show the camera is on) * To distract or not to distract? Seeing your own video (Users need to see how others see them, but ideally can also move/hide themselves) * Micro-interactions for controlling the video/audio (Little details like placement of mute/pause/record buttons matter) * Incorporating text messages into the video on small screens (overlaying text chat messages on mobile devices) * Extra considerations for multiparty calls (placement of icons, showing the dominant speakers, speaker names/profile pics) * Providing feedback on the call quality (both before and during the call) * Information hierarchy: Stay focused on what's important (relative sizes of video and other screen elements depend on the situation) * Conclusion - it's not just a video tag, it's a complete experience
• #3: To us, WebRTC is about the technology P2P, encrypted, STUN/TURN, Signaling, Reliable and Unreliable Data Channels As technologists and entrepreneurs, We love this stuff! But users don’t care!
• #4: So we try to explain WebRTC using cliches, and that’s alright, but the end users still don’t care! Metaphors like “Skype in your browser” help explain the business value of WebRTC, which is incredibly important, but we still shouldn’t jump from business plans right to the technology, without thinking about the user interaction first.
• #5: The end users care about one thing. They have a problem to solve, and hopefully we’ve already established that WebRTC is going to solve that problem. So once they are using our tool, the only thing they care about is Experience. User Experience design is incredibly important, because without a good user experience, your great idea and cool technology will falter. So let’s talk briefly about how to make WebRTC sing visually!
• #6: Right out of the box, WebRTC provides some visual elements that we cannot control, but are important. However, they are not enough to determine a good user experience. The allow/disallow buttons for video and audio are controlled by the browser – but what will your app do when someone doesn’t allow you to use their camera? There’s also that handy “on air” type of icon on the browser tab, to help users know they are live. But users might not notice this, especially when they are under SSL and have previously given you video permissions. So what will your app do to help them remember that they are on camera? Screen sharing is a very useful feature to build, and has that handy permissions bar at the bottom of the screen that you can’t control, but how will you make the most of screen sharing in your app, especially if combined with video?
• #7: So let’s talk about User Experience and Web RTC. There are a few key UX themes in the following recommendations based on our experience. Micro interactions are the “spit and polish” of design, the little things users may not notice but make their life better Information hierarchy refers to the placement and implicit ranking of features in a design. What is most important to users? What do they need immediately available to them? Feedback in this case is not referring to audio echo or feedback noises, but giving the user feedback about how things are going, and what you might need from them. Ok, let’s get to details!
• #8: Before the call even starts, there are things that you can do to improve the experience. Hangouts does a nice job of warning you are about to join the call, letting you see the video you will be sharing, and giving you a chance to mute yourself, pause video, or adjust other settings before joining the call. Once you are in hangouts (ADD SCREENSHOT OF THIS TOO), Hangouts will give you feedback on the volume of your voice – which is a great reminder once you’re in the call if you’re on mute or not.
• #9: The most common video placement for 1-1 video chat is familiar to us all. In this case, you see yourself in a smaller video – it’s important that users can see how they appear to others. This serves to remind them that they are on camera and should stop picking their nose, and it also helps them to make sure the correct camera is being used and that it’s adjust properly. However, keep in mind that this Picture in Picture style can be distracting. Ideally you should be able to move your video or hide it in case it’s interfering with the video of the other party.
• #10: When you’re in a desktop based conversation, the audio and video controls can be subtle. A typical pattern is to put them on top of the video as overlays. Users should be able to find them easily, but the focus should be on the size and quality of the video since this is typically richer for desktop users. Once you are muted, however, it’s a good idea to make that more obvious to the muted user so they don’t talk to themselves
• #11: On mobile, remember finger size, and keep the controls prominent, well separated, and easy to touch, overlaid on top of the video. Screen shot credit: ViTAM on the iOS Store https://itunes.apple.com/us/app/vitam/id1000773687?mt=8
• #12: On phones, you don’t have much space to work with. So keep the text chat on the same screen as the video, and overlay it on top. Keep in mind that this will cover up the video as the messages move up the screen, so they should only stay on the page briefly.
• #13: For multi party calls, remember that the dynamics are different. Video pausing may be more important to help performance. Perhaps you want to highlight the dominant speaker with a larger video area. Something I think is particularly important is showing a caller’s name or profile picture when their video is paused. If you’ve ever been on a multiparty call where everyone paused their video, it can be hard to tell who is speaking.
• #14: It’s important that you provide feedback to users about the call both before and during the call. If someone’s internet is weak, don’t let people even start a call, or at least warn them about the quality. During a call, if the connection goes bad, pause video and tell users why, as Facetime does in this example.
• #15: Here is another example of a demo app we built using the Call Quality API of OpenTok. In this case, before the user gets into the call we test their connection, and the video will be disabled by default for the user if the connection strength is not sufficient.
• #16: What is the most important thing going on right now? In this example, it’s screen sharing. So the screen share gets the most prominent placement on the page, but all other elements are visible too. However, for a use case like this, we should allow the user to go full screen on the screen share so they can entirely focus on that.
• #17: For your mobile applications, make sure you spend some extra time on the UX. Always remember the most important thing to users on the mobile app, and that probably has to do with seeing the video. In this app GeeVee, it allows me to preview my video while I’m waiting for the other party to connect. It’s using overlays very nicely to show me the profile photo of who I’m calling, give me status on the connection, and allow me to quickly hang up with a standard mobile hangup button, in the place I would normally expect it to be. Remember that mobile users expect certain patterns like swipes and common button placements.
• #18: WebRTC is much more than just a video tag thrown on the page. Remember these key points to ensure your users have the best experience possible!