Augmented UX: Creating Alternate Realities for Real Humans

Editor's Notes

• #2: Tonight I want to talk about AR and UX. AR, or Augmented Reality, is the integration of digital experiences with the physical world in real time. And as you can imagine this poses some interesting challenges for UX.
• #3: I’m Christina. I’ve been in UX for 14 years. This year I left my job as UX manager and started my own tech company that does AR for children’s books. I’ve never done AR before. And I needed to learn quickly. Here are my learnings.
• #4: For an AR experience you need a device that can “see” or detect, like a smartphone with a camera. The device recognized an object and presents the digital assets associated with that object. People can interact with both physical and digital objects in AR.
• #5: AR is becoming more pervasive. There are mobile apps out there that can show you what’s in a building, provide pricetags for windowshoppers, or flesh out a dinosaur skeleton. (Thanks for the nightmares, Smithsonian!)
• #6: You can also experience alternate realities through headsets, like Google Glass. Other headsets you may have heard of like Oculus and Hololens, are virtual reality devices, and I’m going to talk VR. I’m also not going to talk about that guy who scotch-taped an iPhone to his face.
• #7: AR is prominent in may visions of the “screenless” future. Today, alternate reality companies like Hololens and Magic Leap want to obliterate the idea of the traditional screen interface. So just in case Magic Leap achieves world domination…
• #8: …we should learn more about immersive experiences from other industries, such as the movies, gaming, theatre, choreography. We can learn from people who plan for lighting and atmosphere.
• #9: The major considerations for a good AR experience are much the same as for any interface. Two key things to remember though: You are working with both digital and physical objects and environments You are working in 3D with an infinite canvas
• #10: Let’s take a look at each of these components through an AR lens. Because an AR experience can mean whole body, the WHOLE person needs to be accounted for: their size, shape, coordination, and physical endurance.
• #11: I made a mistake early on with our children’s app where I didn’t consider proportion Kids have short stubby arms, therefore they are much closer to the target object and device than say a 5’5” woman might be…this meant the app didn’t work properly for our target audience.
• #12: As a UX designer I’ve always had to consider device. But specs become even more crucial when you’re asking people to hold up a device for a long time or download large assets, or detect words or sound.
• #13: Which brings me to my next fail. Devices created for kids often have crappy processors and even crappier cameras. Because our app is for kids we have had to solve these tech shortcomings with design.
• #14: Knowing and designing for probably location of use is invaluable. At home with wifi On a long car trip with no cellular data In a fluorescent-lit classroom Location affects quality of experience.
• #15: This year we exhibited at Augmented World Expo. We were in a large hall with massive fluorescent lighting directly above. This created severe shadows on the pages of the book which would randomly trigger interactions, making our app seem all Poltergeist-y.
• #16: It is important to think about how the human-device interaction changes behaviour. A group of kids using Kinect. A person sitting at a desktop at home. Using a smartphone in public. A wearable. Not spoken: Shared: The whole body is involved, as are large screens and full movement of limbs and torso; think Nintendo Wii or Microsoft Kinect. Intimate: Think of the user in front of their desktop computer with a webcam, generally sitting down with their body an average of 2-3 feet from the camera. Private: The newest category, including wearable tech: glasses like Google Glass or headgear like the Oculus Rift. This will be a personal and intimate experience that will be completely sensory and visual. In the case of the Oculus, it may completely remove the real-world stimulus from the user. Personal: Using a smartphone in a public 360º space, like recent ARGs such as Layar and Junaio. Here the user will be standing up and/or walking and interacting with the environment around them
• #17: The role of the device is also important. A parent who’s phone is primarily for capturing photos of children and posting them to social media might not have the capacity to download large bundles of digital assets.
• #18: And we all know that behaviour varies with context. A person shopping, carrying bags, and using a smartphone with one hand will have a limited range of motion and may not want to gesticulate wildly in public.
• #19: How the device interacts with location also matters. Real-time interaction with the environment is difficult with poor connectivity and poor visibility scenarios. And these situations can result in no AR experience being triggered at all.
• #20: Despite these challenges, alternate reality technology is becoming increasingly popular. As more and more people seek immersive environments to work and play in, the challenge will be for UX to make these experiences good enough to lose yourself in.
• #21: The future of AR is to create context-driven, personalized experiences in REAL time and to do this seamlessly across devices and environments. When I say it that way it sounds hard. But this is the most fun (professionally) I’ve had in 15 years. Thanks!