A Distributed Staged Architecture for Multimodal Applications
Download as PPT, PDF1 like831 views
This document describes a distributed staged architecture for enabling multimodal applications on mobile devices. The architecture aims to provide device and domain independence. It consists of a client-server model with a staged pipeline of XML transformations between sub-architectures for adaptation, dialog management, and the domain-specific documentation application. The architecture was implemented as an XML transformation pipeline and evaluated for mobile maintenance documentation applications, demonstrating device and domain independence. Key research questions remaining are how to better apply the multimodal interaction framework to concrete architectures.
![Enabling Multimodal Access
6/13
W3C Multimodal Interaction Framework (MMI-F)
The multimodal interaction framework is not an
architecture .
The multimodal interaction framework is a level of
abstraction above an architecture. […]
High-Level View
OutputComponents
Input Components
© SAP AG 2007, A Distributed Staged Architecture for Multimodal Applications](https://image.slidesharecdn.com/falk-hartmann-20070925-ecsa-2007-131108133142-phpapp02/85/A-Distributed-Staged-Architecture-for-Multimodal-Applications-6-320.jpg)
A Distributed Staged Architecture for Multimodal Applications
- 1. A Distributed Staged Architecture for Multimodal Applications Alessandro Costa Pereira, Falk Hartmann, Kay Kadner SAP Research/TU Dresden
- 2. Agenda The Project SNOW Prerequisites Enabling Multimodal Access (MMI-F) The SNOW Architecture The Implementation Lessons Learned/Research Questions © SAP AG 2007, A Distributed Staged Architecture for Multimodal Applications 2/13
- 3. Project SNOW 3/13 Services for Nomadic Workers EU FP6 Project, 2004-2006, 6 Partners www.snow-project.org A large-scale industrial diffusion of multimodal mobile documentation for maintenance operations. How to author multimodal mobile maintenance documentation? How to exploit this documentation through robust interaction modalities? © SAP AG 2007, A Distributed Staged Architecture for Multimodal Applications
- 4. 4/13 Project SNOW From paper … a ob C J rd Maintenance procedures and related information …to wearable electronic documentation Airplane maintenance worker uses paper-based documentation Unanticipated problems cause high expenses Multimodality essential Increased productivity for future maintenance © SAP AG 2007, A Distributed Staged Architecture for Multimodal Applications
- 5. Prerequisites 5/13 Multimodality is the use of multiple types of interaction between a user and a computer. W3C Activity "Multimodal Interaction" "Multimodal Interaction Framework" (MMI-F) "Extensible MultiModal Annotation markup language" (EMMA) A role model is "the description of a (possibly) infinite set of object collaborations using role types". Riehle/Gross "Role model based framework design and integration" A staged architecture is "a sequence of n subarchitectures, in which stage produces the (data for the) next stage" Aßmann "Invasive Software Composition" © SAP AG 2007, A Distributed Staged Architecture for Multimodal Applications
- 6. Enabling Multimodal Access 6/13 W3C Multimodal Interaction Framework (MMI-F) The multimodal interaction framework is not an architecture . The multimodal interaction framework is a level of abstraction above an architecture. […] High-Level View OutputComponents Input Components © SAP AG 2007, A Distributed Staged Architecture for Multimodal Applications
- 7. Enabling Multimodal Access MMI-F is a role model Short illustration of the core of the MMI-F Applicable to the software architecture of your choice © SAP AG 2007, A Distributed Staged Architecture for Multimodal Applications 7/13
- 8. The SNOW Architecture Architectural Requirements Client-/Server architecture Domain independence Device independence (⇐ multimodality) (Smallest) End-user device: PDA No speech-/gesture-recognition, no speech synthesis Available modalities depend on network connection Reuse standard software on PDA (Browser) © SAP AG 2007, A Distributed Staged Architecture for Multimodal Applications 8/13
- 9. 9/13 The SNOW Architecture Staged Architecture Browser Client Server Device-specific Adaptation Manager Dialog Manager Device-independent, Domain-independent Domain-independent Documentation Application Domain-specific Procedures © SAP AG 2007, A Distributed Staged Architecture for Multimodal Applications
- 10. 10/13 The SNOW Architecture Keyboard Control Integration Manager Speaker Control Camera Control Microphone Control Text to Speech Gesture Recognition Speech Recognition Browser Client Server Adaptation Manager Dialog Manager Procedures © SAP AG 2007, A Distributed Staged Architecture for Multimodal Applications Rendering Interpretation Styling Integration Documentation Application Recognition Generation Interaction Manager
- 11. The Implementation 11/13 Staged Architecture → XML transformation pipeline Problem 1: Implementation effort More components Use of a common base technology in the components Special template engine with bypassing (XTL Engine) Problem 2: Performance penalty Several parse/serialize processes Mitigated using StAX for XML processing Compensated by deployment options (esp. within development) © SAP AG 2007, A Distributed Staged Architecture for Multimodal Applications
- 12. Lessons Learned/Research Questions 12/13 Lessons Learned Mapping the MMI-F to a concrete architecture can be difficult Not very suitable for client/server architectures Improvement of the MMI-F (fission role) Staged Architecture can enable domain independence Evaluated the architecture in a medical scenario (WearIT@Work) Only the Documentation Application needs to be substituted But: there is no best-suited architecture for multimodality Research Questions How to generically apply the MMI-F role model? © SAP AG 2007, A Distributed Staged Architecture for Multimodal Applications
- 13. 13/13 Questions WLAN Access Point “Server” (Text-to-speech, Speech Recognition, Gesture recognition) Mobile Client (PDA w/ camera and headset) …Tablet PC based demonstrator now also available © SAP AG 2007, A Distributed Staged Architecture for Multimodal Applications
Editor's Notes
- #2: No process Reverse approach: re-interpretation and refinement of architecture after new findings
- #5: Anders erzählen (heute, dann, Vorteile handsfree, kein Büro…)
- #8: role nicht component
- #10: Überleitung zum nächsten: DialogManager & AdaptationManager not merged map to different roles in the MMI-F
- #11: Überleitung zum nächsten: DialogManager & AdaptationManager not merged Adaptation Manager is not bound to this application map to different roles in the MMI-F (indication)