![Airplane
Telephone
Television
Computer
electricity
Wheel
(directed energy) (primitive telepathy) (primitive clairvoyance)
(primitive telekinesis) (primitive teleportation) (enhanced intelligence)
(primitive omniscience)
or
(the power to know
every dumb fact in existence)
(primitive omnipotence)
or
(the power to believe
you’re important)
(primitive omnipresence)
or
(the power to keep up with friends you
couldn’t be bothered to call)
Every game-changing technology
can be seen as a human superpower
For each major technology:
Describe it like a human superpower
Find the intrinsic human need
Project it forward into new tech
[XR advances all of these superpowers]](https://image.slidesharecdn.com/mit-230224190153-1653722e/85/mit-pdf-13-320.jpg)
![The Adverse (noun) : a "seemingly free” reality funded by advertising [*which you
actually pay for via higher product prices] in which advertisers place bets about your
wants and needs, violating your privacy for pro
fi
t.
The Dejaverse (noun): 3D virtual worlds like we’ve seen since 2001 but with better
graphics. Typically ignores dystopian red
fl
ags because “this time it’s di
ff
erent.”
The Meterverse (noun) : a virtual world, potentially overlapping our real lives, in
which everything we do costs us time, money, or privacy.
To Metaverse (verb) : an act of punditry, in which one takes any cool idea from
the last 30 years and calls it “The Metaverse.”
Some Practical Definitions
credits: Matt Miesnieks, @imaginaryZina](https://image.slidesharecdn.com/mit-230224190153-1653722e/85/mit-pdf-48-320.jpg)
mit.pdf
- 1. Avi Bar-Zeev, Feb 23, 2023 MIT Media Lab MAS.S61 AR, the Metaverse and Beyond…
- 2. 30+ Years HoloLens (2010) Second Life (2001) DisneyQuest (1997) Disney VR (1993) Keyhole (1999) -> Google Earth Apple XR (2016) Amazon (2015)
- 3. Experience Prototyping What, When, Why • Designers and free-range programmers need to map the solution space • Before spending billions of dollars on any products • XP is about e ffi cient exploration (breadth fi rst) of the solution space • Answering key questions aligned to the organization’s priorities • Is it di ff erent from academic explorations? • Less about inspiration, originality and more about building con fi dence • XP teams are responsible for guiding very expensive decisions
- 5. Tom F fi ske Prof. Bob Stone
- 6. Licking the Cookie What in the World Wide World is going on with Metaverse hype? Underpromise and Overdeliver
- 7. Source: Avi Bar-Zeev / RealityPrime The VR / Metaverse / AI Hype Cycle The Future of Work The Future of Information The Next Internet 2021 Facebook Rebrands ~2024 Chick- fi l-A Chick-averse 1992 Snow Crash 2008 Second Life 2022 NFTs ~2026 Interoperability & Standards Metaverse Cyberspace Next Name Virtual Reality Information Superhighway Arti fi cial Intelligence 1960s-1980 Ussing, Gibson The Future of Everything Above/Below the Radar
- 8. South Park’s Gnomes had a really simple idea to make money. But without understanding “Phase 2,” it can’t work It’s the Gnomes
- 10. Timeline of Common XR Terminology * The 1968 Sutherland demo is often incorrectly remembered as “The Sword of Damocles” — that was the name of the large overhead mechanical tracker Copyright 2022 Avi Bar-Zeev, Licensed under Creative Commons CC BY-ND 2.0. https://creativecommons.org/licenses/by-nd/2.0/ Metaverse3,4,5 (various) Memex (Bush, 1945) The Metaverse1 (Stephenson, 1992) The Metaverse2 (Roadmap, 2006) The Veldt (Bradbury, 1950) The Web web5 The Other Plane (Vinge, 1981) Telesphere (HMD) & Sensorama (Heilig, 1956) The Ultimate Display* (Sutherland, 1965) Virtual Reality (Lanier, 1987) XR (Mann/Wyco ff , 1991) Virtual Reality (Artaud, 1938) Mixed Reality1 Spectrum (MIlgram/Kishino, 1994) Arti fi cial Reality (Krueger, 1983) Real-World Metaverse (Hanke, 2021) Spectacles (Weinbaum, 1935) CAVE (Cruz-Neira et al, 1992) Wearable Computing (Mann, 1980) Video-See-Through (Tamura, 1997) Waldo (Heinlein, 1942) Spatial Computing (Jacobson, 1993) Cyberspace1 (Gibson, 1982) 1. A vast 3D virtual world laid out along a linear central axis, populated by alternate- identity 3D avatars. Devices support AR, but no one used that feature, because reality was too dystopian. 2. Separated out 4 major components of The Metaverse future: a. Virtual Worlds (see Stephenson), b. Mirror Worlds (see Google Earth) c. Augmented Reality (see HoloLens) d. Life-logging (see Gordon Bell) 3. The Next Web, sometimes with 3D & Avatars, but requiring decentralized content, monetization and identity 4. The Fusion of all things Digital and Physical (a.k.a. AR) 5. Any 3D virtual world Holographic Telepresence (various) Telexistence (Tachi, 1980s) The Matrix (Wachowski, 1999) The Oasis (Cline, 2011) The Internet web3 Web 2.0 2025 2015 2020 2010 2005 1995 2000 1990 1985 1975 1980 1970 1940 1945 1935 1960 1955 1965 Hyperreality (Baudrillard, 1981) Information Superhighway v5 AR Cloud (Inbar, 2017) PolySocial Reality (Applin/Fischer, 2011) Cyberspace2,3 Cyberspace2,3. The de fi nition morphed from 3D virtual worlds to The Internet as is, and “cyber” remains common short-hand for network security. Virtual Reality (Broderick, 1982) Multi-User Dungeon (Bartle, 1978) Virtual Worlds Stereopsis Camera Obscura Telepresence (Gunkel, 1970s) Virtual Environments (Fisher et al, 1980s) Synthetic Environments (Various, 1990s) Cyberspace (Ussing/Ho ff , 1960s) Digital Twin (Hayashi, 1998) Holodeck (Roddenberry et al, 1988) Viewpoint Dependent Imaging (Fisher, 1981) 1950 The Days of Perky Pat (Dick, 1963) Avatar+ (Farmer/Morningstar, 1985) + The term is from Hinduism and many hundreds of years old, but this is the fi rst known use for agents on some display Augmented Reality (Caudell, 1990) Mirror Worlds (Gelertner, 1991)
- 11. A Framework for Understanding XR’s Effects on People
- 12. Virtual Worlds 101 Typical Human Communication “Car” Mediated Communication The Realities of XR
- 13. Airplane Telephone Television Computer electricity Wheel (directed energy) (primitive telepathy) (primitive clairvoyance) (primitive telekinesis) (primitive teleportation) (enhanced intelligence) (primitive omniscience) or (the power to know every dumb fact in existence) (primitive omnipotence) or (the power to believe you’re important) (primitive omnipresence) or (the power to keep up with friends you couldn’t be bothered to call) Every game-changing technology can be seen as a human superpower For each major technology: Describe it like a human superpower Find the intrinsic human need Project it forward into new tech [XR advances all of these superpowers]
- 14. Superpowers for Everyone? Super Unique We each have deeply personal ideas of what is super Super Normal As ‘superpower’ becomes common, we may take it for granted Super Advantaged Some people seek to overcome natural limits. Some seek power over others… Super Skewed Power inequalities create anxiety, anger, instability, and blowback Super Bene fi cent Personal tech should enable everyone equitably (not identically) Superpowers, coming of age
- 15. Super Backlash
- 17. Super Power Spectrum Collision of Intents Socially Destructive Neutral Joint Attention Mutual Bene fi t Socially Constructive Socially Destabilizing
- 18. Most Literal XR Superpowers Virtual Spectrum Superpower Example Flying (+clairvoyance) Google Earth VR Telekenisis Hand Tracking Teleportation Locomotion UX Super Strength Manipulation UX Invisibility Self-Protection Directed Energy Harassment leap motion blog https://www.nature.com/articles/srep09831, @2015 James Liu, Hirav Parekh, Majed Al-Zayer, Eelke Folmer
- 19. Most Literal XR Superpowers Augmented Spectrum Superpower Example Telepathy Telepresence Time Control History Sim Emotional Intel. Sensor+AI X-Ray Vision Surgery Aid Night Vision Military IVAS Mind Control Ad-Tech Stanford XR and Law
- 20. Holographic Telepresence (ala Qualcomm)
- 22. Perceptual Intelligence aka Situational Awareness Not Literal
- 23. The World-Wide-World AR Cloud Not Literal Read/Write World (2011)
- 24. Augmented Emotional Intelligence Biometric Feedback Glasses Pupil dilation Eye gaze patterns Blood fl ow Voice analysis Feeling Anxious, Nervous, Frustrated, Angry Colorize lenses, fl ashing LEDs Private feedback
- 25. Zeitgeist
- 26. To the Future
- 27. How did The Web Evolve? T W E F G I There is no map (but knock yourself out if you want to build one)
- 28. How will The Metaverse Evolve? There is no map (but knock yourself out if you want to build one)
- 29. Screenshot from Twitter Interoperability is Essential
- 30. Links, References and Portals Anything new here for XR? • Browsing across links is like teleporting from one page to another • Embedded references can pull remote content into a page • iFrames are (theoretically safe) sandboxed pages-within-pages (in practice, not so safe) Linked Worlds Composite Worlds Portals
- 31. Not XR XR Consumer XR Business/Industrial XR On “The iPhone Moment” 2020 2030 Critical Mass 100% 0% 15-20% App Store 2004 2014 Con fi dence Point 2025 ± 36 months
- 32. Why is AR Special?
- 33. Is the Metaverse the Sum of all Virtual Worlds? Native Apps, Zoom, and Talking Face-to-Face use Virtual Worlds Must everything be part of a globally shared space? Hell No!
- 34. AR<->VR Spectrum More Present Here Present Anywhere Else vs.
- 35. AR comes in many Coordinate Systems Experiences are all relative. Some are relative to the planet Geo Body Head (HUD) 2:05 pm Local Object
- 36. AR should be Additive But adding too much is subtractive
- 37. Keiichi Matsuda’s HyperReality, plus/minus
- 38. Separating “Verses” and “Purses” • Contextual Discovery Engine — fi nds the most relevant content (anonymously) • Personal AI Shield — keeps our private data safe and defends vs. all exploits • Holographic Telepresence — brings remote people into your local world Personal Data World Data Verses Purses AR is Mainly Personal Versional Personal
- 39. AR is Mainly Personal If it’s just for you, it’s not The Metaverse Suggested paths and 1:1 hologram convos are usually visible only to you. Mapbox Rémi Rousseau
- 40. AR is Mainly Personal Replacing Ads with Personal Photos
- 41. Do you want your most personal data being addressable in the Metaverse? Even assuming competent encryption and permissions Hell No!
- 42. AI is Mostly About People Too Augmented Human Intelligence Model Prompt Learning
- 43. AR is ‘Bigger’ than The Metaverse Not by Volume, but by Ubiquity and UX Message from Anne Message from Anne If The Metaverse is the Internet, and XR hardware+software is the Browser then AR+AI UX may be the common User Interface
- 44. Is There Something Beyond The Meta? Rather than Adding more Fiction, Can we Pierce the Veil? Understanding Reality Exploring Inner Worlds
- 45. Enlightenment Are we Living in a Simulation? Finding Meaning in XR Remember Plato’s Cave? Is There Something Beyond The Meta?
- 46. The Levels of Versality
- 47. Let’s Define “Levels of Versality”
- 48. The Adverse (noun) : a "seemingly free” reality funded by advertising [*which you actually pay for via higher product prices] in which advertisers place bets about your wants and needs, violating your privacy for pro fi t. The Dejaverse (noun): 3D virtual worlds like we’ve seen since 2001 but with better graphics. Typically ignores dystopian red fl ags because “this time it’s di ff erent.” The Meterverse (noun) : a virtual world, potentially overlapping our real lives, in which everything we do costs us time, money, or privacy. To Metaverse (verb) : an act of punditry, in which one takes any cool idea from the last 30 years and calls it “The Metaverse.” Some Practical Definitions credits: Matt Miesnieks, @imaginaryZina
- 49. The Seven Major Verses Adverse (directed for advertisers) Dejaverse (revisited directions) Transverse (directed across) Metaverse or Meterverse Exoverse (directed outside) Esoverse (directed within) Holiverse (directed to wholeness) We Are Here Facebook time
- 50. Path to and Beyond Meta Level Level 0 Adverse (directed by advertisers) Level 1 Dejaverse (revisited directions) Level 2 Transverse (directed across) Level 3 Meta/Meterverse (directed beyond) Level 4 Exoverse (directed outside) Level 5 Esoverse (directed within) Level 99 Holiverse (directed to wholeness) Spatial Metaphors 2D Pages Web of Links Parallax 2.5D Silo 3D Worlds Virtual Earths Connected Virtual Worlds Virtual Universe Real World AR+IOT AR Interface for any Reality Omnipresence People & Avatars Cursors, Names, Icons Cartoon Avatars Video Avatars Virtual Actors Portable ID OK Telepresence Full Avatar Telepresence Holographic Telepresence AI Twins Async Comms Omniscience Immortality Artifacts & Interaction Text, Photos, Videos Virtual Clothing & Accessories Portable Inventory (NFT, DAO) Mashable Assets Intuitive Search AI Creation Assistants Holodeck Post Symbolic Communication Omnipotence Context & Privacy Cookies Web Bugs Global Trackers Do Not Track VPNs Cloaked IDs Portable Permissions Consensual Reality Contextual Discovery Engines Personal AI Shield Solipsism or Hive Mind Monetization Targeted Ads Creator Economies Distributed Ownership Distributed Authorship Chains >50% Creatorship >90% Creatorship Universal Creatorship The Levels of Versality article
- 51. The Whateverse is a shared reality that we don't care what it's called, just tell us what it does or does not do. The Internet of People keeps the proper focus on people, connected Please also consider: Virtual Worlds are various spaces we can virtually inhabit Presence is the feeling of “being” in a particular place (also not in your head) Co-Presence is the feeling that other people are also present and truly human Co-Reality is any web of worlds where we can be present/co-present Better terms? Whateverse: Rebecca Evans
- 52. Conclusions On XR vs. Metaverse Futures XR Metaverse It’s About… People, Perception, Communication, Augmented Humanity Spaces, Commerce, Analysis, Network E ff ects, Escapism, Transhumanism Dependencies… Better Hardware & Algorithms (lower power, cost, weight) Better Communication Better Regulation More Democracy Public View… Limited Personal Footprint (e.g., avatar, virtual clothing) Anything Addressable (like the internet) Privacy is Supported by… Contextual Discovery Engine Personal AI Shield Veri fi able Credentials, DIDs, (and god forbid, blockchain) Augmented Human Fostering Abundance
- 53. Q & A
- 54. Bonus
- 55. Avi Bar-Zeev President: xrguild.org Consulting: www.realityprime.com The Eyes Have It Deep Dive into Eye Tracking and Mental Privacy in XR Stanford Existing Law and Extended Reality Conference Jan 6th, 2023
- 57. Your Eyes visual targets Eye Tracking Cameras Scene Understanding & Tracking Cameras Add Facial Computing AR Glasses Scene Understanding & Tracking Cameras Eye Tracking Cameras
- 58. Bene fi ts of Eye Tracking • Foveated Rendering and Compression • Avatar Eyes / Emotional Fidelity • Spatial User Interfaces for AR & VR • Re-directed Walking for VR • Marketing Research and Lab-Testing Designs • Diagnosis of Autism, Multiple Sclerosis, Alzheimer’s, ALS, Schizophrenia, Parkinson’s, ADHD, concussions…
- 59. Heat Map “Look At" 1 person everyone
- 63. Human Perception Takeaways Our brains continually reconstruct our perception of the world in small sections (fovea, saccades) Our eyes can be easily distracted, tricked, and lead We are frequently blind, allowing the world to be changed without us noticing Our eye-gaze interaction can reveal how we feel about anything we see If we allow this biometric information to leave our devices, even anonymously, we can later be re-identi fi ed This forms a robust model of our individual personality and innate reactions that can be used to manipulate us
- 64. The Darker Side • Security of Biometric Signatures • Emotion-Mapping Personality Pro fi les • Medical-Grade Invasions of Privacy • Non-consensual Experimentation • Real-Time Emotional Manipulation
- 65. The Guardian
- 66. Identity You can change your password… Unique to You: Sclera (blood vessels) Retina (vessels, topology) Lens (distortion) Iris Pattern (color+IR) Eye Motion, Shape, Position The high- fi delity info should never leave the device, even anonymized (because you can easily be re-identi fi ed). infrared
- 67. Raw and Derived Eye Tracking Data Multiple Streams of Concern Raw 3D eye rotations plus 6D head position -> what did you look at? Time/Motion analysis -> did you ignore, notice or focus on it? Pupil analysis -> how excited or motivated were you? (involuntary reflex) ML classification -> how did you feel? (confused, upset; involuntary) Eyebrows / eyelids -> how did you respond? (happy, angry; voluntary) This data also supports medical diagnosis of Autism, Multiple Sclerosis, Alzheimer’s, ALS, Schizophrenia, Parkinson’s, ADHD, concussions…
- 68. “Half the money I spend on advertising is wasted The trouble is, I don’t know which half.” – John Wanamaker
- 69. Feedback Loops The subtle cycle of behavioral manipulation Improve Model Predicting Behavior (given stimuli) Change Environment to Trigger or Test New Stimuli Collect Cognitive & Emotional Responses to Stimuli In the Lab In the Lab In the Lab
- 71. Does “Opt-In” Cover Privacy Concerns? One step beyond contracts of adhesion Consumers will almost certainly opt-into eye tracking for the obvious benefits without understanding the risks. Just like “click-wrap!” First and third party apps could theoretically off-load this data for almost any other use. Permission per app helps a bit. But how do we know which apps to trust or not trust?
- 72. Does “Opt-In” Cover Privacy Concerns? It’s several steps too short 1. Ask consumers for permission based on clearly explained use cases, not just by app name. Apps must declare ‘intents’ to use sensitive data for specific use cases too. 2. Require AAA security and E2E encryption for all data sharing, even first-party. 3. The personal device holds the keys that can audit or invalidate remote copies of this data. Permission to use my Eye Tracking for: Animating my avatar’s eyes Improving 3D rendering performance Marketing Studies (anon, off-device) Diagnostic Health Studies (for ALS) VR-chat (avatar eyes, E2E encrypted) AAA = Authentication, Authorization and Accounting E2E = End to End Encryption
- 73. "The Game Changers” (Net fl ix) Duracell Battery Commercial “How Juul made nicotine go viral” (Vox, Youtube) Have you ever been Manipulated?