Bug or Feature? Covert Impairments to Human Computer Interaction
- 1. Bug or Feature? Covert Impairments to Human Computer Interaction John V. Monaco Naval Postgraduate School, Monterey, CA CHI 2020 Kim SeongOuk 2021.08.13.
- 2. Contents •Introduction •Definition and Background •Kind of Impairments •Covert Impairments •Discussion •Related Works •Conclusion and Future Work
- 3. 2 Introduction • Motivation Users have acclimated to subtle HCI anomalies Intentional anomalies(covert impairments) Appeared Widespread, but little is known • Goal Provide a coherent framework for covert impairments Highlight usage of covert impairments in “wild”
- 4. 3 Definition and Background • Deceive - User’s belief about a system is “demonstrably false” • Modify - “Intentionally” adjust peripheral input • Influence - within cognitive level(10−1 s~102 s) of the Newell’s time scale
- 5. 4 Kind of Impairments • Interaction, Impairment Interaction – Mechanical determination Impairment – Breakdown between input and invoked state • Unintentional Impairments Not intentionally introduced, but attributed to a legitimate sources Main sources • Hardware defects • Software bugs • Resource constraints • Covert Impairments
- 6. 5 Covert Impairments • Use inductive coding procedure Start with set of exemplars encountered by authors and colleagues • Extract 4 canonical impairments(IDSL) Search other applications with 3 Conditions 1. Modify user input 2. Don’t reveal to the user those modification 3. Decrease task performance Characterize in 5 dimensions
- 7. 6 Covert Impairments • Modality – peripheral sensor through which the impairment occurs • Impairment – modification to the input event provided by the sensor Insertion Deletion Substitution Lag
- 8. 7 Covert Impairments • Intent – which party will benefit from the impairment • Malevolent Designer > User Risk to harm to the user without benefit • Benevolent Designer ≤ User Doesn’t mean no risk, but minimized or balanced with other needs
- 9. 8 Covert Impairments • Reaction – Whether designer rely on individual or group • Individual Between-subject differences due to Unique and identifiable behavior of individuals • Group Between-subject similarities due to same set of physiological and cognitive constraints
- 10. 9 Covert Impairments • Goal – Outcome from interaction • Elicit Influence user’s behavior to reveal information User identification (Cognitive biometrics) Bot/fraud detection(CAPTCHA)
- 11. 10 Covert Impairments • Coerce Influence users to perform a chosen action (click, cursor, stroke)-jacking Electronic voting
- 12. 11 Covert Impairments • Induce Alter the user’s affective state and reveal limits of human psychomotor system Affective computing Cognitive science Entertainment
- 14. 13 Discussion • Ethical and practical consideration Involuntary actions Resentment Privacy Cost vs return Timing Sustainability Three impact to balance these consideration • Task impact – Event sequence difference • User impact – Cost to the user • Population impact – Proportion of users vulnerable to impairment
- 15. 14 Discussion • Detection and countermeasures Defenses Range from detection to prevention • Detection – Establish the presence and can avoid • Prevention – removed or neutralized Malevolent impairments • Detection and prevention for users • Detection – closed loop measurements • Prevention – using concepts in W3C UI safety specification Benevolent impairments • Need to be effective and not circumvented
- 16. 15 Related Works • Concealed information tests Reveal deception through mouse pointer motion • Reflexive eye movements Induce Reflexive eye movements for biometric authentication • Subliminal cueing Through UI design • Difficult interfaces Show users’ reaction and adaption to impairments • Dark patterns Similar with Covert impairments, but have key differences • All of these are at interface level
- 17. 16 Conclusion and Future Work • This paper Systemized impairments into 4 kinds (IDSL) Use of Covert impairments is largely concentrated in cybersecurity Mitigation Opinion Detailed description with examples Not enough/appropriate figures Covert impairment for preventing over-immersion? Other unsystematized things like this?
- 18. 17 Else – Doing regretful behavior • Distraction How blocking distractions affects workplace focus and productivity (Gloria Mark, et al. UbiComp ‘17) • Drunk Drunk User Interfaces: Determining Blood Alcohol Level through Everyday Smartphone Tasks (Alex Mariakakis, et al. CHI ’18) • Mistakes in social media Eye Tracking Analysis of User Behavior in Online Social Networks (Wan Adilah Wan Adnan, et al. HCII 2013)
Editor's Notes
- #5: Deceive – Similar with behavior deception in the work Adar et al, but have key different
- #6: Interaction – mechanical determination Impairments – breakdown that occurs somewhere between user input and system state. In
- #7: Condition 3 ? – To exclude assistive technology such as hardware acceleration
- #11: 유저 식별 사진 출처: https://patents.google.com/patent/US20140078061A1/en 봇 탐지 사진 출처: https://reader.elsevier.com/reader/sd/pii/S0969476517301145?token=8AECE520914B7DA352CA52FB9336B29275041D0D2117FD4FA1B656152B73DC591C13BA9A8A05CB228082BEEA911D4E16&originRegion=us-east-1&originCreation=20210811013156 Invisible challenges: the next step in behavioural biometrics?
- #13: 팩맨 사진 출처 - Affective Pacman: A Frustrating Game for Brain-Computer Interface Experiments in Experimental Approach to Affective Interaction in Games – frustratio이 BCI에 주는 영향을 확인하기 위해 Covert impairment를 활용 왼쪽 사진 출처 - Fast corrections of movements with a computer mouse -
- #15: Cost to the user(e.g. increased cognitive load, negative effect)
- #17: A subliminal cue is defined as any cue whose intensity is below the level of consciousness but receives perceptual and semantic processing Dark patterns VS Covert impairments DP: all malevolent, not all deceptive Covert impairments: not all malevolent, all deceptive
- #18: Mitigation and detection of Covert impairments can be used to ensure HCI integrity