Tool Support for Prototyping Interfaces

Distributed Multimodal Information Processing Group
Prof. Dr. Matthias Kranz Technische Universität München

Tool Support for Prototyping Interfaces
for Vision-Based Indoor Navigation

Andreas Möller1, Christian Kray2, Luis Roalter1,
Robert Huitl1, Stefan Diewald1, Matthias Kranz3

1Technische
Universität München, Germany
2University of Münster, Germany

3Luleå University of Technology, Department of Computer Science,

Electrical and Space Engineering, Luleå, Sweden

1st Workshop on Mobile Vision and HCI held at MobileHCI 2012, San Francisco


Background and Motivation

•  Location information still the
most important contextual information
•  Indoor localization is a hot topic and
useful for a lot of scenarios:
–  Airports
–  Hospitals
–  Conference venues
–  Large environments
•  Various indoor localization methods
possible:
–  WLAN/cell based localization
–  Sensor based localization
–  Beacon based localization
–  Vision based localization

21.09.2012 Luis Roalter 2


Why use vision based localization?



Why vision based localization?

•  Use existing features in the environment

•  Modern devices are equipped with a camera and powerful processor

•  No additional augmentation needed



Which challenges will appear?



Challenges of Vision-Based Localization

•  No fixed device needed
–  Use your own smartphone without any optional accessories
–  Hold the device as you like (landscape, portrait, to the ground, to the wall)

•  Quality of visual features
–  Repeating furniture in one hallway
–  Wall of the same paint

•  Orientation of the user’s smartphone
–  Features are typically found in eye height (windows, adverts, posters)

•  In a typical pose, user holds the device in a way that it points at the floor



Challenges in HCI

•  User interfaces need to support/address those problems:
–  Deal with accuracy
–  Corrective actions if not enough features are available

•  Hard to evaluate (would require already functional localization, but interfaces
goes hand in hand with algorithms)

•  Little/no experience with developing user interfaces for vision-based systems

•  There is a need for adequate prototyping tools!



Prototyping. What do we need?



Prototyping Indoor Navigation Interfaces

•  Requirements:

–  Prototyping in early development

–  Ease change of the location (e.g. choose a new environment)

–  Evaluation of usability, efficiency and effectivity

–  Controlled conditions of localization conditions (e.g. location and
orientation accuracy)

–  Evaluation of individual features in real-world scenarios



The Wizard of Oz Approach

Send image directly OR

send image ID of local database Local
Image DB

Trigger UI events

Experimenter‘s Phone Subject‘s Phone
(Server) (Client)



User Interface Concept

•  Use the camera of the smartphone

•  Enable the usage of:
–  Virtual Reality (VR) with panorama images (compare to
Google StreetView) taken in advance
–  Augmented Reality (AR) with a live view of the environment

•  In both cases the user needs to hold the camera device towards interesting
regions.

•  How to motivate the user to hold up the smartphone?



Augmented Reality

•  Display the live camera-image

•  Draw information within the
image

•  Hold the smartphone in eye-
height

•  Live view of the environment

•  Ease match from to the real
environment



Virtual Reality

•  Preloaded panorama
images

•  Similar images to the real
environment

•  Holding the smartphone to
the ground

•  No orientation change
needed

•  User has to match real and
virtual reality



Mockup

•  Testing the augmentation content



What if the algorithm needs a feature-rich region?

•  Motivate the user to look at the phone to follow the instructions
(in both AR and VR conditions!)



Theoretical Evaluation

•  Evaluation based on the initial requirements
–  Use the prototype at early project stage (even before localization is
implemented)

–  Testing AR and VR interface concepts on a real smartphone
–  Inaccuracy possible due to manual transmission of images

–  Different motivation screens to point upon a feature-rich region

•  All initial requirements are met.



Practical Evaluation

•  5 subjects evaluating the mockup system
•  Predefined route through the building:
–  About 100m
–  Dataset recorded with LadyBug
Camera system
–  Corridors and a hall including doors



Findings of the first evaluation

•  AR considered as nice and convenient
•  AR Overlay was not always accurate, felt disturbing
(in case of orientation error)

•  VR had offset in location
•  But user can deal better with inaccuracy, matching panoramas to the real
environment



Interface Design and Usability

•  Future research questions:

–  Which visualization deals better with varying conditions?

–  Automatic vs. manual adjustments by the user?

–  Preferences for proactive behavior?

–  Are VR and AR sufficient for navigation tasks?



Ongoing related work

•  Test the application within a large environment (a larger route with multiple
turns)

•  Use dataset of a complete building recorded in advance (for VR)

•  Evaluate the user‘s experience with the application user-interface

•  Survey with 30 participants



Current Implementation

Subject’s Screen (top)
Drawing of images in AR or VR mode,
connected to the server device

Experimenter’s Screen (right)
Send specific images to the client



Path trough the building

•  Including hallways, corridors and
different turns

•  Evaluate the user’s experience in
case of errors

•  Current research topic and
related user study will be
published soon



Summary

•  All initial requirements are met
–  Testing of the prototype without having an actual localization
–  Different views and augmentations are possible

•  A larger study with a longer way is currently evaluated with the same
prototype

•  Full functional prototype framework available (for public on request)

•  Wizard-of-Oz approach can be replaced step by step



Map



Thank you for your attention!
Questions?

?
?
roalter@tum.de
www.vmi.ei.tum.de/team/luis-roalter.html



Paper Reference

•  Please find the associated paper at:
https://vmi.lmt.ei.tum.de/publications/2012/mobivis_v02_preprint.pdf

•  Please cite this work as follows:
•  Andreas Möller, Christian Kray, Luis Roalter, Stefan Diewald, Rorbert Huitl,
Matthias Kranz. 2012.
Tool Support for Prototyping Interfaces for Vision-Based Indoor Navigation
In: Workshop on Mobile Vision and HCI (MobiVis) on MobileHCI 2012, San
Francisco, USA, September 2012



If you use BibTex, please use the following entry
to cite this work:

@INPROCEEDINGS{Mobivis12moeller,
author={Andreas M"{o}ller and Christian Kray and Luis Roalter and Stefan Diewald
and Robert Huitl and Matthias Kranz},
title={{Tool Support for Prototyping Interfaces for Vision-Based Indoor Navigation}},
booktitle={{Proceedings of the Workshop on Mobile Vision and HCI (MobiVis).
Held in Conjunction with Mobile HCI}},
year={2012},
month=sep,
location={San Francisco, USA},
abstract={Vision-based approaches are a promising method for indoor navigation,
but prototyping and evaluating them poses several challenges. These include the
effort of realizing the localization component, difficulties in simulating real-world behavior
and the interaction between vision-based localization and the user interface. In this paper,
we report on initial findings from the development of a tool to support this process. We
identify key requirements for such a tool and use an example vision- based system to evaluate
a first prototype of the tool.},
}


Tool Support for Prototyping Interfaces

More Related Content

Similar to Tool Support for Prototyping Interfaces (20)

More from Distributed Multimodal Information Processing Group (9)

Tool Support for Prototyping Interfaces