SlideShare a Scribd company logo
Quality of Experience in Smart
Environments



Pedro Maurício Costa               DEGI Club, FEUP
Intelligent Systems and Networks      Porto, 8 June
Introduction

Background
Context
   • Smart Environments
   • Human Computer Interaction
   • Quality of Experience
Proposed Framework
   • Cloud-based software platform
   • Device integration
Domain of Application: Public Transport
Field Experiment
Conclusions
Background

2007
   • Licenciatura em Engenharia Informática e Computação


2007 - 2009
   • Software Engineer at Critical Software


2009 - ?
   • PhD, Mobile Human Computer Interaction
      • Jeremy Pitt
      • João Falcão e Cunha
      • Teresa Galvão
Context

Pervasive Computing
   • miniaturisation of devices
   • modern communication networks
   • Internet of Things

Human Computer Interaction
   • shift in paradigm from PCs to multiple devices
   • Affective Computing

Smart Environments
   • making human interaction with the system a
     pleasant experience


Quality of Experience
   • “the degree to which a system meets users’
     expectations”
Data Flows

Data streams
   • user and environment generated
   • app and service offering: excessive choice may be detrimental and frustrating




                       Collection of Data



                        Service Offering




                                                               1M+ apps available
                                                                (iOS + Android)
Siri

Intelligent personal assistant
   •   natural language interface
   •   personalises results
   •   adapts to user’s individual preferences
   •   curator of things that are relevant

Concerns developing Siri include:
   •   gender neutral?
   •   being friendly and humble
   •   occasional light attitude
   •   “How would we want a person to respond?”
Affective Computing

A novel dimension of interaction
   • to make systems more emphatic
   • implicit interactions with the system
       » camera
       » microphone
       » patterns of usage                                Activation
       » physiology measurement
                                               Nervous                  Excited

Model for emotion representation
   • Russell’s circumplex
   • valence (x): cognitive
                                      Unpleaseant                           Pleaseant
   • arousal (y): physiological



                                             Depressed                  Relaxed

                                                         Deactivation
Framework

Addresses the disconnection between two main data streams
Continuous loop of interaction between user and system
   • leverages existing technology and context
   • integrates emotion

Collection of user and environment data
   • world and user representation

Delivery of personalised services
   • enhancing Quality of Experience
   • influence user behaviour

User-centric and systemic
Cloud2Bubble




                           Cloud     2   Bubble
abstract computing infrastructure        user-centric, local
                  *-as-a-Service         composed of devices and services
         aggregates data streams         collects user and environment data
     generates individual profiles       delivers relevant services
  action generation and selection
Cloud2Bubble

Smart System
   •   collects user and environment data
   •   assesses its current state
   •   generates and selects actions
   •   acts on the environment


World Modelling
   • high-level events                                    Decision Making
   • environment model
   • user profiling
                                                          World Modelling

Decision Making
   • imprecise and incomplete data                           Hardware
   • fuzzy reasoning
                                            Environment                      Action
                                             Perception                     Execution
Public Transport

Domain of Application
   • Public Transport: Intelligent Traveller
     Information Systems
   • poor Quality of Experience
   • equipped modern networks (sensors, info)
   • large user base with idle time
   • benefits for both users and providers
Mobile Device App for Public Transport

Establish connection between users and the system
   •   collects user and environment data
   •   pre-processes data
   •   requests feedback
   •   receives notifications
   •   integrates with the environment

MSc project: João Vieira
PTSense prototype

Non-functional prototype, high-fidelity
   • based on the main features identified
       » collect data & deliver service
   • Android platform
       » large user base
       » ease of development
   • mobile sensing platform
   • usability testing in London


Functional prototype
   • available on Google Play
   • sub-set of features: purely participatory; public
     transport independent
   • in use for the on-going experiment
Field Experiment

Deploy the system on a real-world environment: Porto, Portugal

stage 1: assess Quality of Experience
   •   Experience Sampling Method-based
   •   collect user and environment data
   •   correlation between environment and user feedback
   •   little intervention required from PT providers


stage 2: enhance Quality of Experience
   • delivery of services, identified previously
   • requires tighter integration with PT providers
Collection of Environment and User Data




                                          •personal
                                          •environment
                                          •feedback
Assessing Quality of Experience

Acquiring users, Porto Portugal
   •   survey for selecting users: 170 replies
   •   demographics
   •   travelling habits
   •   smartphone OS and usage (limited to Android users)
   •   “I'm willing to actively contribute with daily, anonymous reports on my
       Public Transport experience through my smartphone during the course
       of this experiment”
“How can your journey be improved?”



                              Comfort
                               16%

             Real-time info             Trip planning
                 18%                         8%
                                               Price
                                                5%

                                              Internet
                                                 9%

                                     Entertainment
                                           5%
                    Service         Nothing
                     32%              8%
Other replies


“Eu não me importo com os
atrasos, desde que seja avisado”       “Quero saber se posso ficar mais
                                       5 minutos a tomar o meu café
                                       ou se tenho de me apressar”



   “Temperatura mais agradável no
   metro, é preciso levar casaco”


                                    “Viagens grátis com bebidas
                                    grátis a caminho do piolho (:D)”

“O ar é abafado no autocarro
quando está demasiada gente”
On-going in Porto, Portugal

User Sample
   •   10 regular travellers
   •   18 - 35 years old
   •   students and professionals
   •   higher education


Goal
   • collect environment data and user feedback:
       » 2 weeks, 10 working days
       » 2 journeys / day
       » total of 200 journeys


Running until mid June
   • incidents related with heterogenous version and devices
   • some of the participants quit after confirmation
   • so far: 110 journeys, 25+ hours
Contribution

Forward thinking, user-centric and systemic
Generic smart framework
Body of knowledge: Quality of Experience
Usage of emotion
Instantiation in public transport
Opens up a number of opportunities
Workshop on Quality of Experience in Mobility Contexts

           10 September @ hci2012, Birmingham UK
                submission deadline: 15 June
                       www.mqoe.org
References
   • Cloud2Bubble: a context-aware platform for enhancing Quality of
     Experience, in awareness magazine, 2012;
   • Cloud2Bubble: Enhancing Quality of Experience in Mobile Cloud Computing
     Settings, in The Third ACM Workshop on Mobile Cloud Computing and
     Services, 2012
   • Smart Mobile Sensing for Measuring Quality of Experience in Urban Public
     Transports, in The Second International Workshop on Smart Mobile
     Applications, 2012

Cloud2Bubble
www.cloud2bubble.com

Pedro Maurício Costa
pm.costa@imperial.ac.uk
www.pedromauricio.com

More Related Content

PDF
Smart Built-Environments for People Living with Dementia_Crimson Publishers
PDF
Model executability within the GEMOC Studio
PDF
Smart Modeling: On the Convergence of Scientific and Engineering Models
PDF
Towards Language-Oriented Modeling (HDR Defense)
PDF
Towards Smart Modeling (Environments)
PDF
Modeling For Sustainability: Or How to Make Smart CPS Smarter?
PDF
Modeling for Sustainability
PDF
Model Execution: Past, Present and Future
Smart Built-Environments for People Living with Dementia_Crimson Publishers
Model executability within the GEMOC Studio
Smart Modeling: On the Convergence of Scientific and Engineering Models
Towards Language-Oriented Modeling (HDR Defense)
Towards Smart Modeling (Environments)
Modeling For Sustainability: Or How to Make Smart CPS Smarter?
Modeling for Sustainability
Model Execution: Past, Present and Future

Viewers also liked (7)

PDF
5 Trends To Redefine Human Capital
PDF
Navigazione e Acquisti - 10 Regole per difendersi dal malware-Silvia Belli
PPT
The Damned Legacy, Chapter 1.1
PPTX
Apply your knowledge root word.for project
KEY
A Framework for Quality of Experience enhancing Systems
PDF
Gli Otto Principi Della Qualità Totale-Questionmark-Silvia Belli
DOCX
Analisis tendencias pedagogicas caso personal
5 Trends To Redefine Human Capital
Navigazione e Acquisti - 10 Regole per difendersi dal malware-Silvia Belli
The Damned Legacy, Chapter 1.1
Apply your knowledge root word.for project
A Framework for Quality of Experience enhancing Systems
Gli Otto Principi Della Qualità Totale-Questionmark-Silvia Belli
Analisis tendencias pedagogicas caso personal
Ad

Similar to Quality of Experience in Smart Environments (20)

PDF
Cloud2Bubble: Enhancing Quality of Experience in Mobile Cloud Computing Settings
PDF
Vanderbeeken_EXPERIENTIA
PDF
Tamara Chahine Honours Proposal
PDF
Designing Mobile Personal Informatics Applications for Urban Settings
PDF
LBi Digital Academy | Class #2
PDF
Qo E E2 E5 User Centric Approach Katrien De Moor
PDF
Customer experience 2.0
PPT
3D context-aware mobile maps for tourism - ENTER2011 PhD Workshop
KEY
From Napkin Sketch to Product
PDF
Public transports in data age
PDF
iDiff 2008 conference #09 IP-Racine FP7 Call3 presentation
PDF
Session UXnUI SJCIT-1.0.1-1apr2019-desops.io
PDF
Design activity framework for visualization design
PDF
Mobility&Udi 2011
PDF
[Nux]12 nux
PDF
Technology Trends, Consumer Experience @MICA 2016
PDF
User Engagement: from Sites to a Network of Sites or The Network Effect Matters!
PDF
Designing Interactions / Experiences: Lecture #05
PDF
We've Never Had It So Good
PPTX
Exploring “live” Social and Networked Interaction with the Future Media Inter...
Cloud2Bubble: Enhancing Quality of Experience in Mobile Cloud Computing Settings
Vanderbeeken_EXPERIENTIA
Tamara Chahine Honours Proposal
Designing Mobile Personal Informatics Applications for Urban Settings
LBi Digital Academy | Class #2
Qo E E2 E5 User Centric Approach Katrien De Moor
Customer experience 2.0
3D context-aware mobile maps for tourism - ENTER2011 PhD Workshop
From Napkin Sketch to Product
Public transports in data age
iDiff 2008 conference #09 IP-Racine FP7 Call3 presentation
Session UXnUI SJCIT-1.0.1-1apr2019-desops.io
Design activity framework for visualization design
Mobility&Udi 2011
[Nux]12 nux
Technology Trends, Consumer Experience @MICA 2016
User Engagement: from Sites to a Network of Sites or The Network Effect Matters!
Designing Interactions / Experiences: Lecture #05
We've Never Had It So Good
Exploring “live” Social and Networked Interaction with the Future Media Inter...
Ad

Recently uploaded (20)

PDF
Profit Center Accounting in SAP S/4HANA, S4F28 Col11
PPTX
Big Data Technologies - Introduction.pptx
PPTX
sap open course for s4hana steps from ECC to s4
PDF
Spectral efficient network and resource selection model in 5G networks
PPT
Teaching material agriculture food technology
PDF
Advanced methodologies resolving dimensionality complications for autism neur...
PDF
Reach Out and Touch Someone: Haptics and Empathic Computing
PDF
Machine learning based COVID-19 study performance prediction
PDF
Architecting across the Boundaries of two Complex Domains - Healthcare & Tech...
PPTX
Programs and apps: productivity, graphics, security and other tools
PPTX
Cloud computing and distributed systems.
DOCX
The AUB Centre for AI in Media Proposal.docx
PDF
Encapsulation_ Review paper, used for researhc scholars
PDF
Agricultural_Statistics_at_a_Glance_2022_0.pdf
PDF
Build a system with the filesystem maintained by OSTree @ COSCUP 2025
PDF
Assigned Numbers - 2025 - Bluetooth® Document
PDF
Mobile App Security Testing_ A Comprehensive Guide.pdf
PPTX
ACSFv1EN-58255 AWS Academy Cloud Security Foundations.pptx
PPTX
Machine Learning_overview_presentation.pptx
PDF
Chapter 3 Spatial Domain Image Processing.pdf
Profit Center Accounting in SAP S/4HANA, S4F28 Col11
Big Data Technologies - Introduction.pptx
sap open course for s4hana steps from ECC to s4
Spectral efficient network and resource selection model in 5G networks
Teaching material agriculture food technology
Advanced methodologies resolving dimensionality complications for autism neur...
Reach Out and Touch Someone: Haptics and Empathic Computing
Machine learning based COVID-19 study performance prediction
Architecting across the Boundaries of two Complex Domains - Healthcare & Tech...
Programs and apps: productivity, graphics, security and other tools
Cloud computing and distributed systems.
The AUB Centre for AI in Media Proposal.docx
Encapsulation_ Review paper, used for researhc scholars
Agricultural_Statistics_at_a_Glance_2022_0.pdf
Build a system with the filesystem maintained by OSTree @ COSCUP 2025
Assigned Numbers - 2025 - Bluetooth® Document
Mobile App Security Testing_ A Comprehensive Guide.pdf
ACSFv1EN-58255 AWS Academy Cloud Security Foundations.pptx
Machine Learning_overview_presentation.pptx
Chapter 3 Spatial Domain Image Processing.pdf

Quality of Experience in Smart Environments

  • 1. Quality of Experience in Smart Environments Pedro Maurício Costa DEGI Club, FEUP Intelligent Systems and Networks Porto, 8 June
  • 2. Introduction Background Context • Smart Environments • Human Computer Interaction • Quality of Experience Proposed Framework • Cloud-based software platform • Device integration Domain of Application: Public Transport Field Experiment Conclusions
  • 3. Background 2007 • Licenciatura em Engenharia Informática e Computação 2007 - 2009 • Software Engineer at Critical Software 2009 - ? • PhD, Mobile Human Computer Interaction • Jeremy Pitt • João Falcão e Cunha • Teresa Galvão
  • 4. Context Pervasive Computing • miniaturisation of devices • modern communication networks • Internet of Things Human Computer Interaction • shift in paradigm from PCs to multiple devices • Affective Computing Smart Environments • making human interaction with the system a pleasant experience Quality of Experience • “the degree to which a system meets users’ expectations”
  • 5. Data Flows Data streams • user and environment generated • app and service offering: excessive choice may be detrimental and frustrating Collection of Data Service Offering 1M+ apps available (iOS + Android)
  • 6. Siri Intelligent personal assistant • natural language interface • personalises results • adapts to user’s individual preferences • curator of things that are relevant Concerns developing Siri include: • gender neutral? • being friendly and humble • occasional light attitude • “How would we want a person to respond?”
  • 7. Affective Computing A novel dimension of interaction • to make systems more emphatic • implicit interactions with the system » camera » microphone » patterns of usage Activation » physiology measurement Nervous Excited Model for emotion representation • Russell’s circumplex • valence (x): cognitive Unpleaseant Pleaseant • arousal (y): physiological Depressed Relaxed Deactivation
  • 8. Framework Addresses the disconnection between two main data streams Continuous loop of interaction between user and system • leverages existing technology and context • integrates emotion Collection of user and environment data • world and user representation Delivery of personalised services • enhancing Quality of Experience • influence user behaviour User-centric and systemic
  • 9. Cloud2Bubble Cloud 2 Bubble abstract computing infrastructure user-centric, local *-as-a-Service composed of devices and services aggregates data streams collects user and environment data generates individual profiles delivers relevant services action generation and selection
  • 10. Cloud2Bubble Smart System • collects user and environment data • assesses its current state • generates and selects actions • acts on the environment World Modelling • high-level events Decision Making • environment model • user profiling World Modelling Decision Making • imprecise and incomplete data Hardware • fuzzy reasoning Environment Action Perception Execution
  • 11. Public Transport Domain of Application • Public Transport: Intelligent Traveller Information Systems • poor Quality of Experience • equipped modern networks (sensors, info) • large user base with idle time • benefits for both users and providers
  • 12. Mobile Device App for Public Transport Establish connection between users and the system • collects user and environment data • pre-processes data • requests feedback • receives notifications • integrates with the environment MSc project: João Vieira
  • 13. PTSense prototype Non-functional prototype, high-fidelity • based on the main features identified » collect data & deliver service • Android platform » large user base » ease of development • mobile sensing platform • usability testing in London Functional prototype • available on Google Play • sub-set of features: purely participatory; public transport independent • in use for the on-going experiment
  • 14. Field Experiment Deploy the system on a real-world environment: Porto, Portugal stage 1: assess Quality of Experience • Experience Sampling Method-based • collect user and environment data • correlation between environment and user feedback • little intervention required from PT providers stage 2: enhance Quality of Experience • delivery of services, identified previously • requires tighter integration with PT providers
  • 15. Collection of Environment and User Data •personal •environment •feedback
  • 16. Assessing Quality of Experience Acquiring users, Porto Portugal • survey for selecting users: 170 replies • demographics • travelling habits • smartphone OS and usage (limited to Android users) • “I'm willing to actively contribute with daily, anonymous reports on my Public Transport experience through my smartphone during the course of this experiment”
  • 17. “How can your journey be improved?” Comfort 16% Real-time info Trip planning 18% 8% Price 5% Internet 9% Entertainment 5% Service Nothing 32% 8%
  • 18. Other replies “Eu não me importo com os atrasos, desde que seja avisado” “Quero saber se posso ficar mais 5 minutos a tomar o meu café ou se tenho de me apressar” “Temperatura mais agradável no metro, é preciso levar casaco” “Viagens grátis com bebidas grátis a caminho do piolho (:D)” “O ar é abafado no autocarro quando está demasiada gente”
  • 19. On-going in Porto, Portugal User Sample • 10 regular travellers • 18 - 35 years old • students and professionals • higher education Goal • collect environment data and user feedback: » 2 weeks, 10 working days » 2 journeys / day » total of 200 journeys Running until mid June • incidents related with heterogenous version and devices • some of the participants quit after confirmation • so far: 110 journeys, 25+ hours
  • 20. Contribution Forward thinking, user-centric and systemic Generic smart framework Body of knowledge: Quality of Experience Usage of emotion Instantiation in public transport Opens up a number of opportunities
  • 21. Workshop on Quality of Experience in Mobility Contexts 10 September @ hci2012, Birmingham UK submission deadline: 15 June www.mqoe.org
  • 22. References • Cloud2Bubble: a context-aware platform for enhancing Quality of Experience, in awareness magazine, 2012; • Cloud2Bubble: Enhancing Quality of Experience in Mobile Cloud Computing Settings, in The Third ACM Workshop on Mobile Cloud Computing and Services, 2012 • Smart Mobile Sensing for Measuring Quality of Experience in Urban Public Transports, in The Second International Workshop on Smart Mobile Applications, 2012 Cloud2Bubble www.cloud2bubble.com Pedro Maurício Costa pm.costa@imperial.ac.uk www.pedromauricio.com