Information Architectures - Lecture 04 - Next Generation User Interfaces (4018166FNR)

2 December 2005
Next Generation User Interfaces
Information Architectures
Prof. Beat Signer
Department of Computer Science
Vrije Universiteit Brussel
http://www.beatsigner.com

Beat Signer - Department of Computer Science - bsigner@vub.ac.be 2October 16, 2017
Information Architecture
 Information architecture (IA)
addresses the organisation,
structuring and labelling of
content
 Modelling of information
spaces
 maintenance
 linking
 navigation
 search
 presentation
 Personal Information Management (PIM)
IA
Content
Users Context

Personal Information Management (PIM)
 PIM is about keeping, organising and re-finding information
Personal information management or PIM is both the
practice and the study of the activities people perform to
acquire, organise, maintain, retrieve, use, and control the
distribution of information items such as documents
(paper-based and digital), web pages, and email
messages for everyday use to complete tasks (work-
related and not) and to fulfil a person's various roles (as
parent, employee, friend, member of a community, etc.).
William Jones & Jaime Teevan, Personal Information Management,
University of Washington Press, 2007

Filers and Pilers
 Study by Malone (1983) revealed two strategies of
information organisation in offices
 Filers: Instances are explicitly titled and arranged in some
systematic order and these structures may themselves be
explicitly titled and systematically organised
 Pilers: Piles tend not to have internal structure, other than access
frequency; their spatial location is often the key to finding them
 Both files and piles are ways of collecting groups of
elements into larger units
 These strategies are also applied in the digital world,
e.g. to organise emails, bookmarks, files, etc.
Thomas Malone, How do People Organize their Desks? Implications for the Design of Office
Information Systems, ACM Transactions on Information Systems (TOIS) 1(1) (1983): 99-112

Long Term Memory
 Permanent memory store intended for the long-term
storage of information (everything that we know including
experiential knowledge, procedural skills, etc.)
 huge (if not unlimited) capacity
 relatively slow access time
 forgetting (when it occurs) happens much more slowly than with
short-term memory
 Three long-term memory subsystems
 episodic memory: memory of events and experiences in a serial
form (chronology)
 semantic memory: structured record of facts and concepts that we
have acquired
 procedural skills: "know-how" memory (skills, procedures)

"As We May Think" (1945)
When data of any sort are placed in storage,
they are filed alphabetically or numerically,
and information is found (when it is) by
tracing it down from subclass to subclass.
It can be in only one place, unless duplicates
are used; one has to have rules as to which
path will locate it, and the rules are cumbers-
ome. Having found one item, moreover, one
has to emerge from the system and re-enter
on a new path. The human mind does not
work that way. It operates by association. ...
Vannevar Bush

"As We May Think" (1945) …
... It affords an immediate step, however, to
associative indexing, the basic idea of which
is a provision whereby any item may be
caused at will to select immediately and
automatically another. This is the essential
feature of the memex. The process of tying
two items together is the important thing. ...
Vannevar Bush, As We May Think,
Atlanic Monthly, July 1945
Vannevar Bush

"As We May Think" (1945) …
 Bush's article As We May
Think (1945) is often seen
as the "origin" of hypertext
 The article introduces
the Memex
 memory extender
 store and access information
 follow cross-references in the form
of associative trails between pieces
of information (microfilms)
 prototypical hypertext machine
 trail blazers are those who find delight in
the task of establishing useful trails
Memex

Beyond the Desktop Metaphor
 Desktop metaphor
 WIMP interaction
 "filing cabinets" with
hierarchical folders
 Paper paradigm
 "What You See Is What
You Get" (WYSIWYG)
 what about richer
document formats that
existed at that time?
Xerox Star 8010 [http://www.digibarn.com/collections/systems/xerox-8010/]

Digital Documents as a Paper Simulator?
Most people don't understand the logic of the
concept: "What You See Is What You Get" is based
on printing the document out ("get" means "get
WHEN YOU PRINT IT OUT"). And that means a
metaphysical shift: a document can only consist of
what can be printed! This re-froze the computer
document into a closed rectangular object which
cannot be penetrated by outside markings (curtailing
what you could do with paper). No marginal notes,
no sticky notes, no crossouts, no insertions, no
overlays, no highlighting - PAPER UNDER GLASS.
Ted Nelson, Geeks Bearing Gifts: How the Computer
World Got This Way, Mindful Press 2009
Ted Nelson

What is Wrong with Digital Documents?
 Existing document formats are based on the simulation
of paper affordances on desktop computers
 How to manage mixed-media "documents" in open and
fluid cross-media information spaces?
 on the data level
- context-sensitive adapatation
- cross-media transclusion
 on the visualisation and navigation level
- zoomable user interfaces
 on the cross-media interaction level
- fluid multimodal cross-media interfaces
 Remediation of the "paper simulation" approach
 WYSIWYG is only one out of many options!
CISA
Human-Machine &
Human-Information
Interaction
Information
Systems &
Management
Information
Visualisation
& Navigation

Lifestreams
 Avoid naming and
classification efforts
 time-ordered stream as
storage model
 motivated by studies of
Malone
 Stream
 represents the past, present
and future (e.g. reminders)
 stream filters (virtual
directories) to further classify
documents
Freeman & Gelernter, 1996

MyLifeBits
 Lifelogging research
project by Microsoft
 inspired by Bush's Memex
 towards "total recall"
 Database of resources
and links
 multiple classification via
collections
 dynamic collections (queries)
 annotations
 full-text search
 multiple visualisations
Gemmell et al., 2002

Microsoft SenseCam
 Take a picture when one
of the sensors is active
 light, motion or temperature
 around 4000 pictures a day
 Studies have shown that
rich image sets do not
promote "total recall"
 Commercial products
Narrative Clip

Presto
 Placeless Documents
 more natural and fluid forms
of interaction with a
document space
 Semantic information no
longer encoded in a file's
path
 separation of document
content, storage and
properties
 Single document model
 integrate content from
various sources
Dourish et al., 1999

Presto …
 User-level properties
 key/value pairs added to document "proxies"
 Documents can be grouped in collections
 multiple classification
 dynamic collections ("fluid" collections)
- membership defined based on predicates (presence and value of document
properties)
- furthermore, there exists an inclusion and exclusion list
 static collections
- implemented as dynamic collections where only inclusion list is non-empty
 Lessons learned
 "One of the primary experiences of property-based document
interaction is uniformity. […] This uniformity has proven invaluable."

Haystack
 PIM solution using RDF as
data model
 predefined basic ontology
 personalised ontologies
 Most data imported
 special extractors
 Collections
 multiple classification and
query-based collections
 User interfaces also
modelled in RDF
 highly customisable UI
Adar et al., 1999

Semantic Desktops
 Apply Semantic Web tech-
nologies to personal infor-
mation management (PIM)
 inter-application data sharing
 enhancement of limited
filesystem functionality
- add document metadata
 machine-readable data
 Nepomuk is a semantic
desktop solution
integrated in KDE 4.0
 semantic search
Nepomuk Integration with Dolphin (KDE 4.0)

Stuff I've Seen
 Index over multiple
repositories
 email, file system, …
 Different forms of search
 full-text search
 contextual cues
- time, user, …
 Platform for different
visualisations
 "Predecessor" of Windows
Search and Spotlight
Dumais et al., 2003

RSL Hypermedia Metamodel and iServer
Global Information Systems Group, ETH Zurich

Links
Users
Layers
Plug-ins
Structure
Active
Content
Global Information Systems Group, ETH Zurich

Bridging the Paper-Digital Divide

RSL-based Associative File System
signer
My Pictures
Vancouver ER 2010
Granville
Island
Vancouver
Harbour
Public
Market
Keynote
J. Thorp
Olga and
Lamia
Dinner
ER 2010
multiple classification
structural link
navigational link
My Videos
Vancouver
Aquarium
ER 2010
Keynote
cross-media association
My Presentations
ER 2010 WISE 2009
cross-media
transclusion
single slide
transclusion
(via OOXML
RSL selector)
Gregory Cardone

RSL-based Associative File System ...
Gregory Cardone

PIM@WISE
 Keeping, organising and
re-finding information
 digital and physical
 Study of human-
information interaction
 files, piles, mixtures, …
 OC2 PIM model
 based on RSL hypermedia
metamodel
 Cross-media PIM system
 explicit as well as implicit
associations between entities
Sandra Trullemans

Object-Concept-Context (OC2) Framework
object layer
structural link
concept layer
context layer
navigational link
associative link
weighted link
Object 1 Object n
Object 2
Context 1 Context n
Concept 1
Concept 2 Concept n
extent link
entity
link
Links
selector resource
(1,*)(1,*)
(1,1) (0,*)
(0,*) (0,*)
RefersTo
HasTargetHasSource
partition
Entities
(0,*)
link
Navigational
Links
link
Extent
Links
partition
ResourcesSelectors
context
Contexts
link
Structural
Links
link
Associative
Links
resource
Physical
Objects
resource
Digital
Objects
resource
Concepts
resource
Objects
HasAssoc
Target
(0,*)
(1,*)
HasExt
Target
HasExt
Source
HasAssoc
Source
(1,1)
(1,*)
(0,*)
(0,*)
(0,*)
(1,1)
disjoint
partition
translation
OC2 conceptual framework RSL-based metamodel

Cross-Media PIM (PimVis)
Sandra Trullemans

PimVis Setup
Sandra Trullemans

PimVis Document View
Audrey Sanctorum

PimVis Context View
Audrey Sanctorum

Open Cross-Media Linking
 Integration of a new resource type includes
 development of a data plug-in
 implementation of a visual plug-in
 registration with the resource plug-in repository
 Link browser loads plug-ins on demand
 internal or external (third party application) visualisation
 Third-party applications use the link browser for
visualisation
 similar to the integration of third-party applications with web
browsers
 Non-monolithic annotation model and authoring tool
 resource-specific plug-ins

Open Cross-Document Linking
local visual plug-ins
external visual plug-ins
communicate
Link Browser

What is Wrong with Slideware?
 Simulation of physical slides
 limited space due to the slide concept
 linear navigation from slide to slide
 difficult to reuse content and embed rich media types
 MindXpres addresses these issues
 unlimited canvas with zoomable user interface
 non-linear navigation and associative linking
 content-based approach with automatic visualisation

MindXpres Presentation Platform
 Extensible prototyping
platform for novel presen-
tation concepts
 content-based approach
- separation of content and pre-
sentation (automatic visualisation)
 cross-media content reuse
 non-linear navigation via
zoomable user interface
 connectivity and interactivity

MindXpres Platform
 MindXpres presentations are currently represented in the
XML-based MindXpres document format
 Compiler (node.js application) translates XML to HTML
 Presentation engine based on HTML5 and related APIs
 e.g. WebSockets for connectivity

MindXpres Communication Platform

Polls and Quizzes Plug-in

Interactive Source Code Plug-in
Paul Mestereaga

Interactive Data Visualisation Plug-in

Interactive Data Visualisation Plug-in ...

References
 V. Bush, As We May Think, Atlantic Monthly
176(1), July 1945
 http://dx.doi.org/10.1145/227181.227186 (reprint)
 T. Malone, How do People Organize their Desks? Impli-
cations for the Design of Office Information Systems,
ACM Transactions on Information Systems (TOIS) 1(1),
January 1983
 http://dx.doi.org/10.1145/357423.357430
 T. Nelson, Geeks Bearing Gifts: How the Computer
World Got This Way, Mindful Press 2009

References …
 B. Signer, What is Wrong with Digital Docu-
ments? A Conceptual Model for Structural Cross-Media
Content Composition and Reuse, Proceedings of
ER 2010, 29th International Conference on Conceptual
Modeling, Vancouver, Canada, November 2010
 http://beatsigner.com/publications/signer_ER2010.pdf
 E. Freeman and D. Gelernter, Lifestreams:
A Storage Model for Personal Data, ACM SIGMOD
Record 25(1), March 1996

References …
 J. Gemmell et al., MyLifeBits: Fulfilling the
Memex Vision, Proceedings of Multimedia 2002,
Juan-les-Pins, December 2002
 P. Dourish et al., Presto: An Experimental Architecture
for Fluid Interactive Document Spaces, ACM Trans-
actions on Computer-Human Interaction 6(2), June 1999

References …
 E. Adar, D. Karger and L.A. Stein, Haystack:
Per-User Information Environments, Proceedings of
CIKM 1999, 8th International Conference on Information
and Knowledge Management, Kansas, USA,
November 1999
 S. Dumais et al., Stuff I’ve Seen: A System for
Personal Information Retrieval and Re-Use, Proceedings
of SIGIR 2003, 26th Annual International ACM SIGIR
Conference on Research and Development in Informa-
tion Retrieval, Toronto, Canada, August 2003

References …
 B. Signer and M.C. Norrie, As We May Link:
A General Metamodel for Hypermedia Systems, Pro-
ceedings of ER 2007, 26th International Conference on
Conceptual Modeling, Auckland, New Zealand,
November 2007
 http://beatsigner.com/publications/signer_ER2007.pdf
 S. Trullemans and B. Signer, Towards a Conceptual
Framework and Metamodel for Context-Aware Personal
Cross-Media Information Management Systems,
Proceedings of ER 2014, 33rd International Conference
on Conceptual Modelling, Atlanta, USA, October, 2014
 http://beatsigner.com/publications/trullemans_ER2014.pdf

References …
 S. Trullemans, A. Sanctorum and B. Signer,
PimVis: Exploring and Re-finding Documents in Cross-
Media Information Spaces, Proceedings of AVI 2016,
International Working Conference on Advanced Visual
Interfaces, Bari, Italy, June 2016
 http://beatsigner.com/publications/trullemans_AVI2016.pdf
 B. Signer and M.C. Norrie, A Model and Architecture for
Open Cross-Media Annotation and Link Services,
Information Systems 36(3), Elsevier, May 2011
 http://beatsigner.com/publications/signer_IS2011.pdf

References …
 A.A.O Tayeh and B. Signer, Open Cross-Docu-
ment Linking and Browsing Based on a Visual Plug-in
Architecture, Proceedings of WISE 2014,15th Interna-
tional Conference on Web Information System
Engineering, Thessaloniki, Greece, October, 2014
 http://beatsigner.com/publications/tayeh_WISE2014.pdf
 R. Roels and B. Signer, MindXpres: An Extensible
Content-driven Cross-Media Presentation Platform,
Proceedings of WISE 2014, 15th International Confer-
ence on Web Information System Engineering,
Thessaloniki, Greece, October, 2014
 http://beatsigner.com/publications/roels_WISE2014.pdf

References …
 R. Roels, P. Mestereaga and B. Signer, An
Interactive Source Code Visualisation Plug-in for the
MindXpres Presentation Platform, Communications in
Computer and Information Science (CCIS), 583, 2016
 http://beatsigner.com/publications/roels_CCIS2016.pdf
 R. Roels, Y. Baeten and B. Signer, Interactive and
Narrative Data Visualisation for Presentation-based
Knowledge Transfer, Communications in Computer and
Information Science (CCIS), 739, 2017
 http://beatsigner.com/publications/roels_CCIS2017.pdf

2 December 2005
Next Lecture
Multimodal Interaction

Information Architectures - Lecture 04 - Next Generation User Interfaces (4018166FNR)

More Related Content

What's hot (20)

Similar to Information Architectures - Lecture 04 - Next Generation User Interfaces (4018166FNR) (20)

More from Beat Signer (20)

Recently uploaded (20)

Information Architectures - Lecture 04 - Next Generation User Interfaces (4018166FNR)