Towards a Reference Ontology on Mulsemedia Systems
0 likes43 views
This document presents a research paper on developing a reference ontology for mulsemedia systems, which integrate traditional media with non-traditional sensory experiences. The authors aim to create a comprehensive vocabulary and structure to reduce ambiguity and improve communication about these complex digital ecosystems. The paper discusses the ontology's scope, requirements, evaluation methods, and proposes future works to enhance the framework further.
Towards a Reference Ontology on Mulsemedia Systems
- 1. Towards a Reference Ontology on Mulsemedia Systems SALEME1, E. B., SANTOS1, C. A. S., FALBO1, R. A., GHINEA2, G., ANDRES 3, F. 1Federal University of Espírito Santo, Brazil 2Brunel University London, UK 3National Institute of Informatics, Japan The 10th International Conference on Management of Emergent Digital EcoSystems (MEDES'18)
- 2. 2MEDES 2018 - Towards a Reference Ontology on Mulsemedia Systems Outline ▪ Introduction and motivation ▪ Our Research ▪ Brief Background on Mulsemedia ▪ MulseOnto ▪ Scope and Requirements ▪ Mulsemedia System sub-ontology ▪ Medium sub-ontology ▪ Sensory Effects sub-ontology ▪ Evaluation and results ▪ Conclusion and Future Works
- 3. 3MEDES 2018 - Towards a Reference Ontology on Mulsemedia Systems Intro and Motivation ▪ Multimedia -> requirements stems from the continuous, temporal nature of audio and video ▪ Mulsemedia -> integrate these classic media types with non- traditional ones such as olfactory, haptic, and gustatory ▪ Heterogeneous digital ecosystem ▪ Lack of formal and explicit representation of what mulsemedia is ▪ Potential issues: ▪ Misconception -> lame (weak) solutions ▪ Overlook standardization, reuse, compatibility, and so on
- 4. 4MEDES 2018 - Towards a Reference Ontology on Mulsemedia Systems Our research ▪ Capture the common conceptualization of Mulsemedia systems and what it entails to ▪ Reduce ambiguity and inaccuracy by creating an ontology ▪ Make the best possible description of the domain in reality and then improve communication, learning and problem-solving ▪ What did we do? ▪ Developed a domain reference ontology (≠ operational ontology) - > common vocabulary ▪ Big picture of mulsemedia (entities, interactions, kinds of sensory effects, physical realizations) ▪ Evaluation: assessment by humans and a data-driven approach
- 5. 5MEDES 2018 - Towards a Reference Ontology on Mulsemedia Systems Brief Background on Mulsemedia ▪ Common concepts: ▪ Mulsemedia system -> a multimedia system (constituted of programs) + at least one non-traditional media (embed in mulsemedia programs or descriptions) ▪ Media types and creation -> text, graphics, sound, and motion created by humans, acquired through sensors, or synthesized ▪ Sensory Effects -> contain light, vibration, spraying, wind, scent, taste, temperature descriptions ▪ Mulsemedia Presentation -> it coordinates the integration of outputs emanating from the media and sensory-effect specific devices and considers resource limitations, user preferences, and other presentation features
- 6. 6MEDES 2018 - Towards a Reference Ontology on Mulsemedia Systems MulseOnto - Scope ▪ What mulsemedia systems and their small parts are ▪ Which entities they interact with ▪ Where the sensory effects come from ▪ What physical realizations different kinds of media do in this context ▪ The current MulseOnto does not intend to answer: ▪ QoE related aspects, how stimuli are perceived, how media are loaded, composed and presented, timing aspects management, set up of systems and devices, neither attribute details (computer media extensions, encoding methods for computer medium, kinds of flavors or scent, the intensity of effects, temporality constraints, colors, etc)
- 7. 7MEDES 2018 - Towards a Reference Ontology on Mulsemedia Systems MulseOnto - Requirements ▪ Functional requirements specified as CQs
- 8. 8MEDES 2018 - Towards a Reference Ontology on Mulsemedia Systems MulseOnto - Requirements ▪ Non-functional requirements: ▪ SEON is organized in layers: ▪ Top level: Unifed Foundational Ontology (UFO), a foundational ontology that is based on a number of theories from Formal Ontology, Philosophical Logics, Philosophy of Language, Linguistics and Cognitive Psychology ▪ Core level: on software (Software Ontology - SwO) and software process (Software Process Ontology - SPO)
- 9. 9MEDES 2018 - Towards a Reference Ontology on Mulsemedia Systems MulseOnto – sub-ontologies
- 10. 10MEDES 2018 - Towards a Reference Ontology on Mulsemedia Systems MulseOnto – Mulsemedia sub- ontology 1 2 3
- 11. 11MEDES 2018 - Towards a Reference Ontology on Mulsemedia Systems MulseOnto – Medium sub-ontology
- 12. 12MEDES 2018 - Towards a Reference Ontology on Mulsemedia Systems MulseOnto – Sensory Effects sub- ontology
- 13. 13MEDES 2018 - Towards a Reference Ontology on Mulsemedia Systems Evaluation and Results ▪ First: expert judgments to analyse whether concepts and relations defined in the ontology are able to answer their competency questions ▪ Are there elements besides the point? If so, they are not part of this current ontology ▪ Second: data-driven approach -> mulsemedia system instance taking into account concepts and relations ▪ Validated whether MulseOnto is capable of representing a real-world case through an elastic hypothetical mulsemedia scenario
- 14. 14MEDES 2018 - Towards a Reference Ontology on Mulsemedia Systems Hypothetical mulsemedia scenario ▪ It involves all of the concepts presented in the ontology in terms of instances A recently created science museum in Tokyo will provide a multi-sensory experience to its visitors. It is elaborating an astonishing exhibition on exploring the earth through places like volcanoes, glaciers, earthquakes, aurora borealis, waterfalls, and locations with exotic food. The goal is that visitors not only see videos and pictures of those places, but also feel immersed in each environment they are experiencing. To do so, the museum will be using a mulsemedia platform called PlaySEM, which has a video player and a renderer that support sensory effects. The video-clips are annotated with sensory effect descriptions in MPEG-V (with the help of SEVino), that include light, vibration, spraying, kinesthetic, tactile, wind, scent, taste, fog, and temperature effects. Visual media will be displayed on screens, projectors, head-mounted displays. Some visual media objects will also be read from a behavior presentation specification and be presented in holographic devices. LED strips and lightbulbs will come up with lighting effects. Eventually, a live event from a real place can be directly transmitted to the museum. The behavior presentation specification will cater to mulsemedia program with audio media sources. To provide auditory experiences, the museum will arrange box speakers, headphones, earbuds, and rumble devices. To deliver haptic experiences, there will be a motion chair, local air conditioners, spray machines, wind turbines and vibration vests. Olfactory and gustatory effects will be conveyed in two ways. Scent diffusers and electric smell devices to feel the smell of the object media presented to the visitors. A digital lollipop and a beverage electronic device will be set for taste.
- 15. 15MEDES 2018 - Towards a Reference Ontology on Mulsemedia Systems Evaluation and Results Excerpt of Competency Questions Verification Excerpt of MulseOnto Instantiation
- 16. 16MEDES 2018 - Towards a Reference Ontology on Mulsemedia Systems Conclusion ▪ Mulsemedia is not easy to understand at a glance -> complex digital ecosystem and different views ▪ We established a common conceptualization of mulsemedia systems, described a common vocabulary to discuss mulsemedia solutions ▪ We captured and adapted the ontology to reflect concepts sometimes neglected when reading a plain English text by ▪ Evaluating the reference ontology against the competency questions ▪ Instantiating the concepts to see if a hypothetical mulsemedia system adheres to it ▪ MulseOnto -> initial step to discuss the mulsemedia ecosystem that involves capturing, distributing, rendering and perceiving sensory effects
- 17. 17MEDES 2018 - Towards a Reference Ontology on Mulsemedia Systems Future Works ▪ Besides our elastic hypothetical mulsemedia system, further scenarios from real-world environments must be subject of study to strengthen the accuracy of our ontology ▪ Create a mulsemedia ontology network to cope with the process of capturing, distributing, rendering and perceiving sensory effects in detail ▪ Materialization of this conceptual model to an operational ontology to exchange sensory effects metadata ▪ Model to integrate different mulsemedia standards enabling interoperability between different solutions by mapping them to this reference ontology
- 18. 18MEDES 2018 - Towards a Reference Ontology on Mulsemedia Systems Thanks for your attention! Questions? Frederic Andres andres at nii.ac.jp Estêvão B. Saleme, Celso A. S. Santos, Ricardo A. Falbo, Gheorghita Ghinea, and Frederic Andres. 2018. Towards a Reference Ontology on Mulsemedia Systems. In Proceedings of International Conference on Management of Emergent Digital EcoSystems, Tokyo, Japan, September, 2018 (MEDES’18), 8 pages. https://doi.org/10.1145/3281375.3281378