![Here’s foobot, an air quality service.
[I also like how one of its implicit service promises is to identify when your kids are
smoking pot.]
29](https://image.slidesharecdn.com/348066350-the-ux-of-predictive-ai-in-the-iot-rosenfeld-to-be-designed-2017-190624232342/85/The-UX-of-Predictive-AI-in-the-IoT-Rosenfeld-To-Be-Designed-29-320.jpg)
The UX of Predictive AI in the IoT (Rosenfeld To Be Designed)
- 1. Good afternoon and thanks for having me here. In this talk I want to look at the design challenges of systems that anticipate users’ needs and then act on them. That means it sits at the intersection of the internet of things, user experience design and machine learning, and although people have dealt with one of those disciplines before, I don’t think they’ve ever been combined in quite the ways they are now, or with the current enthusiasm. The talk is divided into several parts: it starts with an overview of how I think Internet of Things devices are primarily components of services, rather than being self-contained experiences, how predictive behavior enables key components of those services, and then I finish by trying to to identify user experience issues around predictive behavior and suggestions for patterns to ameliorate those issues. A couple of caveats: - My current work in this field focuses almost exclusively on the consumer internet of things and a limited set of neural network-based AI technologies, so I see most things through those lenses. Predictive AI has a long history in industrial applications, and there are many kinds of AI, but I think it’s in the consumer space that we really the the UX issues. - I want to point out that few if any of the issues I raise are new. Though the terms “internet of things” and “machine learning” are hot right now, the ideas have been discussed in research circles for decades. Search for “ubiquitous computing,” “ambient intelligence,” and “pervasive computing” and you’ll see a lot of great thought in the space. If you’re really ambitious, you can read the Artificial Intelligence and Cybernetics works of the 50s and 60s and you’ll be surprised by the prescience of the people working in this space when the entire world’s compute power was about as much as my key fob. - There are a lot of ideas here, and I will almost certainly under-explain something. For that I apologize in advance. My goal here is to give you a general sense of how these the pieces connect, rather than an in-depth explanation of any one of the pieces. - Finally, most of my slides don’t have words on them, so I’ll make the complete deck with a transcript available as soon I’m done. 1
- 2. Let me begin by telling you a bit about my background. I m a user experience designer. I was one of the first professional Web designers. This is the navigation for a hot sauce shopping site I designed in the spring of 1994. 2
- 9. When you have a multitude of connected devices and apps, value shifts to services and the devices, software applications and websites used to access it—its avatars— become secondary. A camera becomes a really good appliance for taking photos for Instagram, while a TV becomes a nice Instagram display that you don’t have to log into every time, and a phone becomes a convenient way to check your friends’ pictures on the road. Hardware, physical things, become simultaneously more specialized and devalued as users see “through” each device to the service it represents. The avatars exist to get better value out of the service. 9
- 10. Amazon really gets this. Here s a telling older ad from Amazon for the Kindle. It’s saying Look, use whatever device you want. We don t care, as long you stay loyal to our service. You can buy our specialized devices, but you don t have to. 10
- 11. Most large-scale IoT products are service avatars. They use specialized sensors and actuators to support a service, but have little value—or don’t work at all—without the supporting service. Smart Things, which was acquired by Samsung, clearly states its service offering right up front on their site. The first thing they say about their product line is not what the functionality is, but what effect their service will achieve for their customers. Their hardware products’ functionality, how they will technically satisfy the service promise, is almost an afterthought. 11
- 17. One approach is to change from an ownership model to a subscription model. Now the device gives access to a desired end result, without the burdens of ownership or maintenance. The IoT technology is what gives an efficient way to track and charge for assets. Car sharing, bike sharing, Uber and AirBNB follow this model. You don’t use it every day, so why own it? High-end clothing is going this way. Do you really need to own that Prada handbag so you can use it twice a year? 17
- 33. 33
- 37. 37
- 40. Computers acting autonomously erode this simple tool/user relationship. Predictive IoT is more than just recommending a new song, it’s acting on your behalf on the basis of its assumption about what you want, and what’s best for you. At the dawn of computing in the late 1940s cyberneticists like Norbert Wiener philosophized about the increasingly complex relationship between people and computers, and how it was fundamentally different than the way we interact with other kinds of machines. Developers working in supervisory control of manufacturing machines and robotics have had to deal with these questions pragmatically for about 30 years, but thanks to the Internet of Things, this is now a problem that everyone will have to grapple with going forward. Here’s a diagram by the greats Tom Sheridan and Bill Verplank from 1978, in which they illustrate four ways that semi-autonomous computers and humans can work together to solve a problem. 40
- 41. By 2000 Sheridan expanded these ideas with Parasuraman and Wickens to define a spectrum of responsibility between people and computers. It ranges from humans doing all the work (this is you writing an essay) to computers doing all the work completely autonomously (this is your car’s fuel injection controller). Of course the goal is to get a system to level 9 or 10. That’s the maximum reduction in cognitive load. However, for a system to qualify for that, it has to be very stable, its effects need to be highly predictable and, equally importantly, it’s role needs to be adequately embedded in society. It needs to be OK for a computer to take on that level of responsibility. At the airport we trust the monorail computers to work without human intervention, but we don’t trust the plane autopilot to do that, even though-–as I understand it—planes can basically fly themselves these days. Predictive IoT devices generally fall between 5 and 7 on this scale right now. The problem is that this is the exact range where you’re maximizing someone’s cognitive load, but not necessarily doing all the work for them, so the result of the automation had better be worth it. This fundamentally undermines what we expect from our tools, and when that tool is trying to anticipate what we’re trying to do, it fundamentally changes our working relationship with it. 41
- 44. The irony in predictive systems is that they’re pretty unpredictable, at least at first. When machine learning systems are new, they’re often inaccurate and unpredictable, which is not what we expect from our digital devices. 60%-70% accuracy is typical for a first pass, but even 90% accuracy isn’t enough for a predictive system to feel right, since if it’s making decisions all the time, it’s going to be making mistakes all the time, too. It’s fine if your house is a couple of degrees cooler than you’d like, but what if your wheelchair refuses to go to a drinking fountain next to a door because it’s been trained on doors and it can’t tell that’s not what you mean in this one instance? For all the times a system gets it right, it’s on the mistakes that we judge it and a couple such instances can shatter people’s confidence. Anxiety is a kind of cognitive load, and a little doubt about whether a system is going to do the right thing is enough to turn a UX that’s right most of the time into one that’s more trouble than it’s worth. When that happens, you’ve more than likely lost your customer. Unfortunately, sooner than we think, such inaccurate predictive behavior isn’t going to be an isolated incident. Soon we’re going to have 100 connected devices simultaneously acting on predictions about us. If each is 99% accurate, then one is always wrong. So the problem is: How can you design a user experience to make a device still functional, still valuable, still fun, even when it’s spewing junk behavior? How can you design for uncertainty? Photo CC BY 2.0 photo 2011 Pop Culture Geek taken by Doug Kline: https://www.flickr.com/photos/popculturegeek/6300931073/ 44
- 50. Because machine means your tools adapt to you and learns from you, adaptive tools are more like apprentices, rather than implements and our use of them is more like a conversation rather than than linear tool use. In fact, I heard one of Nest’s UX designers say that he considered users’ evolving relationships to the Nest as a conversation. This is about more than just chatbots and voice UI. If you listen to a human conversation, it’s almost never a linear, straightforward, well-structured process. We stop, we rephrase, we ask for corrections, we talk past each other, we interrupt. More likely than not, this is how a predictive machine learning system will interact with people, from whom it will want guidance, confirmation, and who will ask it for recommendations or changes to its behavior. Ethnomethdologists and conversation analysts have been modeling how people talk to each other for about 40 years, so I’m going to borrow some of their concepts. • Sequence organization is about organizing action in time. What happens first, what happens next? How do the two parties expand on ambiguity? For example, if a home security system decides you’re not home, it can tell you “I see you’re driving away from home. I’m going to turn all the alarms on.” You can then say “All of the except for the back yard.” • Turn-taking is critical. We don’t just simply take turns when talking, we continuously provide feedback and correct. We have expectations for whose turn is next and what they’re supposed to do. “Ok, chair, I’m sitting here, now it’s your turn. Confirm you know I’m here. Warn me if you’re going to adjust.” • Repair is backtracking, clarifying, continuing after an interruption, etc. What happens when the expected sequence, either from the perspective of the person or the service, is broken and needs to be reconstructed? 50
- 51. In addition to teaching apprentices about our needs, we also learn from apprentices what their capabilities are and why they made certain decisions, rather than others, when doing the things we taught them to do. This is both a part of how they learn about us and how we learn to work with them effectively. The BMW iDrive system was notorious for its UI, which didn’t tell you what it could or couldn’t do, and how to do it. You had a knob and that was basically it. How do I interrogate an adaptive system to understand what it can do, and to ask it to explain what it just did. How do you know what Siri or Google Now have learned to do? Well, you use the app. But what about services for which you don’t have a display? Chatbots today are essentially command line interfaces. They know specific words and sequences, but what if those commands change over time? What if the device learns new things over time? 51
- 53. Since predictive systems are neither consistent, nor are the reasons for their behavior clear, this can be really confusing. The same thing can behave differently in what appear to be similar circumstances. If we undermine people’s confidence in a system by violating their expectations, they’re likely to be disappointed and stop using it. When we’re dealing with a human or an animal, unpredictable behaviors are expected and tolerated, but that’s not the case with computers. A predictive UX needs to do is to set people’s expectations appropriately. It needs to explain the nature of the device, to describe it is trying to predict, that it’s trying to adapt, that it’s going to sometimes be wrong, to explain how it’s learning, and how long it’ll take before it crosses over from creating more trouble than benefit. Recommender systems, such as Google Now, describe why a certain kind of content was selected, and that sets the expectation that in the future the system will recommend other things based on other kinds of content you’ve requested. Nest’s FAQ kind of buries the information, but it does explain that you shouldn’t expect your thermostat to make a model of when you’re home or not until it’s been operating for a week or so. 53
- 54. About ten years ago Timo Arnall and his students tried to address a similar set of questions around interactions with RFID-enabled devices by creating an iconography system that communicated to potential users that these devices had functionality that was invisible from the outside. Perhaps we need something like this for behavior created by predictive behavior? 54
- 56. You have to give people a clear way to teach the system and tell it when its model is wrong. Statistical systems, by definition, don’t have simple rules that can be changed. There aren’t obvious handles to turn or dials to adjust, because everything is probabilistic. If the model is made from data collected by several devices, which device should I interact with to get it to change its behavior? Google Now asks whether I want more information from a site I visited, Amazon shows a explanation of why it gave me a suggestion. Mapping this to the consumer IoT means way more explanation than we’re currently getting, which is either that a thing has happened, or it hasn’t. 56
- 57. Finally, don’t expect to automate, other than in narrow circumstances. In the vast majority of situations, these systems shouldn’t try to replace people, but to support them, to augment and extent their capabilities, to help them be better at what they want to do, not to replace them. For example, Ember from Meshfire, is a machine learning assistant for social media management. It doesn’t try to replace the social media manager. Instead it manages the media manager’s todo list. It adds things that it thinks are going to be interesting, deletes old things, and reprioritizes the manager’s list based on what it thinks is important. I think this is a good model for how such systems can add value to a person’s experience without creating a situation where random, unexplained behaviors confuse people, frustrate them and make them feel powerless. Ember is an augmentation to the social media manager, it helps that person focus on what’s important so that they can be smarter about their decisions. It doesn’t try to be smarter than they are. How can our devices HELP us, rather than trying to replace us? 57
- 59. Finally, for me the IoT is not about the things, but the experience created by the services for which the things are avatars. 59
- 61. Thank you. 61