Stefano Baldassi (Meta): Designing for the Future of Spatial Computing
- 1. Designing (with Neuroscience) for the Future of Spatial Computing Stefano Baldassi, PhD Sr. Director | Analytics & Neuroscience
- 3. VISION segregation integration bias correction figure-ground textures contours MULTISENSORY audiovisual localization timing haptic ATTENTION cue modulation fixation control clutter analysis saliency analysis distraction analysis NATURAL COMPUTER COGNITION task analysis A/D task integration attention aware design targeted +R targeted -R predictive design
- 7. A UI is a toolset to interact with digital content
- 8. What is the lesson from neuroscience when it comes to Spatial UIs?
- 11. 1. Think Spatial: Place Tools and Content in Space Replace flat layouts of windows, menus and buttons, which generally favor only half of the visual system, with a Spatial Interface that arranges tools and content in 3D space around the user. This leverages the user’s full perception of form and depth.
- 12. 2. Minimize Abstractions: Design Tools with Volume and Affordances Now that we understand how to arrange objects in space, how should we design the objects themselves? Replace abstract representations (like flat icons) with volumetric tools featuring physical characteristics that suggest their use without explicit instruction, leveraging the Neuroscience of Affordance.
- 13. 3. You are the OS: Organize holographic files and tools in the user’s environment Replace the traditional file system—which demands continual decoding of abstract symbols and hierarchies—with spatial organization of files in the world that leverages the brain’s spatial memory, naturally tracking the contents of our personal space.
- 14. 4. Touch to See: Use the hands to interact directly Replace remote gestures with direct manipulation, leveraging the neuroscience of interaction—namely, the brain’s naturally heightened understanding of objects near the hands
- 15. 5. Do Not Disturb: Protect the user’s workflow from disruption Spatial computing demands spatial notifications. Rather than interrupting the user’s workflow with pop-ups, allow them to designate a separate container to passively collect incoming notifications. This allows the user to remain focused on their task—only stopping to check for updates when desired—and leverages the Neuroscience of Attention.
- 16. 6. Avoid Surprises and Magic Tricks: Pair actions with intuitive outcomes Avoid “magical” events that appear unrelated to user behavior or prior causes, or violate laws of physics to the point of confusion, as they confound the user’s expectations and trigger Error Neurons
- 17. 7. The Holographic Campfire: Don’t obscure hands and faces with the UI Replace UIs that block eye contact with a shared space that promotes it, and ensure hands are visible while collaborating. Doing so leverages the power of mirror neurons and face coding.
- 18. 8. Public by Default: Shared understanding reduces anxiety among users Replace private UIs—which separate users and cause ambiguity by obscuring their intentions—with a common digital environment for all Meta users that leverages the neuroscience of theory of mind and ensures comfortable awareness for all participants.
- 19. 9. Augmented, not Mixed Reality: Enhance the user’s perception with relevant information Rather than block out users’ reality (as in virtual reality) or distort the user’s reality (as in mixed reality), provide Meta UIs—metadata about the world, near but not occluding it—that adds something useful to their understanding of their environment. This is the ultimate aim of this document: to create an informative, powerful and unobtrusive layer of digital information on top of the real world.
- 20. This is just the beginning!