Breaking the Boundaries of Human-in-the-Loop Optimization

Saarland University, July 6th, 2023
Breaking the Boundaries of
Human-in-the-Loop Optimization
Yi-Chi Liao. Ph.D. Candidate, UI Group @ Aalto University
Website: http://yichiliao.com | Twitter: @yichiliao | Email: yi-chi.liao@aalto.
f
1

• Ph.D. dissertation:
• Human-in-the-Loop Design Optimization
• Current research group:
• User Interfaces Group at Aalto University, led by Prof Antti Oulasvirta
• Research interests:
• Optimization (inference-free); human-in-the-loop optimization, simulation-based optimization
• User modelling via reinforcement learning
• My tools:
• Bayesian optimization, reinforcement learning, meta-learning, and physics simulation.
• Am joining in November!
Yi-Chi Liao
2

Three goals
• Introduce human-in-the-loop optimization (HILO)
• Introduce the recent advancements in HILO
• Turn you into a Bayesian optimization user
3

Let's play a game! Here's how it works:
You can choose any floating-point number between 0 and 1.2.
Each number corresponds to a specific amount of money in return.
The challenge is to find the number that yields the highest amount of money!
4

Design optimization
0 1
0
3
What is the best location to place the optical sensor?
x ~ [0,1]
y ~ [0,3]
[Optimal Sensor Position for a Computer Mouse, CHI ’20]
5

Design optimization
0 1
0
3
X := design space
x := a design instance
7
x ~ [0,1]
y ~ [0,3]

Design optimization
0 1
0
3
X := design space
x := a design instance
Optimal design
8

Design optimization
0 1
0
3
9
f is unknown function!
f is expensive to evaluate!

[Caeses (https://www.caeses.com/)]
Design optimization
[Human-in-the-loop optimization of exoskeleton assistance during walking, 2017]
10

Design space (X) Objective-function space (Y)
x1
x2
x3
x4
x5
y1
y2
y3
y4
y5
Black-box function,
y = f(x)
Design instance, x Objective function value, y
Design optimization
11

Manual
optimization
Human-in-the-loop optimization
(HILO)
Beyond current
HILO
12

Manual
optimization
(HILO)
Beyond current
HILO
13

Manual optimization
Design and
prototype
Analysis User testing
14

Manual optimization
[ThirdHand: Wearing a Robotic Arm to Experience Rich Force Feedback, Liao et al., 2015]
What is the optimal torque value(s) that maximizes the gaming experience?
15

Manual optimization
What is the optimal way to split the strokes?
[EdgeVib: E
ff
ective Alphanumeric Character Output Using a Wrist-Worn Tactile Display, Liao et al., 2015]
16

Manual optimization
[Outside-In: Visualizing Out-of-Sight Regions-of-Interest in a 360 Video Using Spatial Picture-in-Picture Preview, Lin et al., 2017]
What is the optimal position, size, and tilt angle of the window?
17

Manual optimization
Design and
prototype
Analysis User testing
• Pros:
• It is easy and feasible
• Cons:
• It does not guarantee the result
• The developers may introduce bias
• Can not adapt instantly
18

“Don’t ever make the mistake [of thinking] that you can
design something better than what you get from ruthless
massively parallel trial-and-error with a feedback cycle.”
— Linus Torvalds
19

Manual
optimization
(HILO)
Beyond current
HILO
20

Human-in-the-loop optimization (HILO)
Interactive system
paremeterized by x
Computational
optimizer
User
Design candidate,
x ∈ X
Objective-function
value, y ∈ Y
Interaction
Designer
Design space, X
Objective space, Y
Optimal design, x
21

Bayesian Optimization (BO)!
23

Bayesian optimization —
1.Surrogate model of the target
problem: a Gaussian Process
regression
2.Acquisition function: a
function estimates the utility
of selecting a design choice
25

26
1.Able to (roughly) predict the
outcome of a design choice
2.Able to identify the design
choice that may yield the most
improvements within a huge
design space
Bayesian optimization —
1.Surrogate model of the target
problem: a Gaussian Process
regression
2.Acquisition function: a
function estimates the utility
of selecting a design choice

27
True function
Surrogate model

28
Observations
(Evaluations)
Surrogate model

29
Predicted mean value
Surrogate model
Predicted con
fi
dence interval
i.e., uncertainty

30
Acquisition function
Acquisition value

Surrogate model: Gaussian Process
• GP is a generalized form of multivariate Gaussian (normal)
distribution.
• Covariance function (kernel):
• White noise kernel
• Exponentiated quadratic kernel
• Periodic kernel
37

Selection of kernels
Exponentiated quadratic kernel Periodic kernel
38

• AF quantifies the utility, potential, or worth of different x points based
on the GP surrogate model.
• Computing AF is way cheaper than actual evaluation!
39

• AF quantifies the utility, potential, or worth of different x points based
on the GP surrogate model.
• Computing AF is way cheaper than actual evaluation!
• Common acquisition functions:
• Expected Improvements (EI)
• Probability of Improvements (PI)
• Upper Confidence Bound (UCB)
40

Why is BO so good (mathematically)?
41
• BO is a general tool with the least assumption of the problem.
• BO is clearly explainable and transparent.
• BO is a computational framework allows for easy customization & extension.

Applications of Bayesian Optimization
42

43
arxiv.org/abs/2303.14291
BO in engineering
BO in natural science
BO in social/
economical/political
decision-making

[Bayesian optimization explains human active search, Borji and Itti, 2013]
[Generalization guides human exploration in vast decision spaces, Wu et al., 2017]
44
BO is the best model/framework to explain human’s
parametric decision-making!

When Bayesian optimization performs poorly?
45
• GP will not work well when dealing with
non-continuous function
• More sample points will be needed in high-
dimensional (input/output) function
• Gets too expensive when handling with huge
amount of data: GP
fi
tting time ~ O(m^3)
where m is number of observations

Bayesian optimization for human-involved
interactive systems
[Human-in-the-loop optimization of exoskeleton assistance during walking, 2017]
46

[A Bayesian Interactive Optimization Approach to Procedural Animation Design, Siggraph SCA ’10]
47
interactive systems

48
[Designing Engaging Games Using Bayesian Optimization, Khajah et al., 2016]
interactive systems

Interested in BO for HCI? Also check out Yuki’s works!!!
49
https://koyama.xyz/

Is HILO / BO perfect?
There are boundaries!
• HILO only addresses single-objective problems
• HILO reduces the designer’s agency and ownership in the process
• HILO only optimizes for one user/task at a time
• HILO only works for prototype-free interactions (software interfaces)
• HILO is still not efficient enough!
50

Manual
optimization
(HILO)
Beyond current
HILO
51

From a single objective to multiple objectives
52

53
Can we come up with a new
acquisition function?

54
[Single- and Multiobjective Evolutionary Design Optimization Using
Gaussian Random Field Metamodels, Emmerich, 2005]
[Pareto frontier learning with expensive correlated objectives,
Shah & Ghahramani, 2016]

Short duration,
high amplitude
Long duration,
high amplitude
Long duration,
low amplitude
Short duration,
low amplitude
Short duration,
high amplitude
Long duration,
high amplitude
Long duration,
low amplitude
Short duration,
low amplitude
What is the optimal set of tacton (tactile icon)?
55
[Interaction Design With Multi-Objective Bayesian Optimization, Liao et al., 2023]

Short duration,
high amplitude
Long duration,
high amplitude
Long duration,
low amplitude
Short duration,
low amplitude
Short duration,
high amplitude
Long duration,
high amplitude
Long duration,
low amplitude
Short duration,
low amplitude
Length indicates duration
Set 1
56

Short duration,
high amplitude
Long duration,
high amplitude
Long duration,
low amplitude
Short duration,
low amplitude
Short duration,
high amplitude
Long duration,
high amplitude
Long duration,
low amplitude
Short duration,
low amplitude
Color indicates vibration amplitude
Set 1
Set 2
57

Short duration,
high amplitude
Long duration,
high amplitude
Long duration,
low amplitude
Short duration,
low amplitude
Short duration,
high amplitude
Long duration,
high amplitude
Long duration,
low amplitude
Short duration,
low amplitude
Color indicates vibration amplitude
Set 1
Set 2
58
Set 3

Short duration,
high amplitude
Long duration,
high amplitude
Long duration,
low amplitude
Short duration,
low amplitude
Short duration,
high amplitude
Long duration,
high amplitude
Long duration,
low amplitude
Short duration,
low amplitude
What are the design objectives?
59

a b
60

[Investigating Positive and Negative Qualities of Human-in-the-Loop Optimization for Designing Interaction Techniques, Chan et al., 2022]
61

What is the optimal design?
Imagine there are just 10 discrete levels in each parameter,
the number of possible combinations is already 10^4!
62

63

E
ffi
ciency & accuracy
64

65

Does multi-objective BO really perform better
than human designers?
66

How do designers perceive BO as a tool?
67

Shortcomings of HILO?
68

Collaborative Bayesian optimization
69
Human’s strength
Quickly determine whether a
design is good or bad
AI (BO)’s strength
More principled exploration
+

A cooperative system where designers can steer the
design when needed
70
[Cooperative Multi-Objective Bayesian Design Optimization, Mo et al., submitted to TiiS 2023]

A cooperative system where designers can steer the
design when needed
71
[Cooperative Multi-Objective Bayesian Design Optimization, Mo et al., submitted to TiiS 2023]
Results:
The Cooperative BO reaches
comparable user performances
with less full evaluations (more
e
ffi
ciently).
The designers felt more engaging
in the process than fully relying on
BO.

Boundaries of current HILO
72

73

From a single user to a population of users
74
[Practical Approaches to Group-Level Multi-Objective Bayesian Optimization in Interaction Technique Design, Liao et al., submitted 2023]
Aggregate
Sample over
design space
Global Pareto-optimal
performances
y2
y1
Mean prediction of y1
x
y
P1
y
x
x
y
x
y
Global GP
y
x
P2
P3 P4

75

From prototyping to physical emulation
76
[Button Simulation and Design via FDVV Models, Liao et al., 2020]

From prototyping to physical emulation
77
[Button Simulation and Design via FDVV Models, Liao et al., 2020]

78

Time-efficiency issue
• Bayesian optimization is mainly seen as a design tool. That is, it is
used to derive the optimal design parameter prior to deploying the
interface/interaction on end-users.
79

Time-efficiency issue
• Bayesian optimization is mainly seen as a design tool. That is, it is
used to derive the optimal design parameter prior to deploying the
interface/interaction on end-users.
• Why?
• It takes ~ 10 x “NO of design parameters” iterations to converge.
This takes 40 - 60 mins
80

But why? Isn’t BO the best optimization tool?
• Yes! It is the upper bound of optimization methods!
• But it suffers from the cold-start problem
81

Can we boost the efficiency of BO?
82

Can we boost the efficiency of BO?
Yes! By augmenting BO with prior information!
83

84
[Practical Approaches to Group-Level Multi-Objective Bayesian Optimization in Interaction Technique Design, Liao et al., submitted 2023]
Extract
representative
points
x
y
All observations across users
Initialize
BO
Observed y1
Observed y2
x
y
Warm-Start GP BO with a new user
x
y
New samples
Warm-start Bayesian optimization

86
Current HILO
Single-
objective
a b
Pareto frontier
learning

87
Current HILO
Single-
objective Lack agency
Cooperative
AI
Pareto frontier
learning

88
Current HILO
Single-
Cooperative
AI
Only for
1 user/task
Population
model
Aggregate
Sam
desig
x
y
P1
y
x
x
y
x
y
Global GP
y
x
P2
P3 P4
Pareto frontier
learning

89
Current HILO
Single-
Cooperative
AI
Only for
1 user/task
Population
model
Only GUI
Physical
emulation
Pareto frontier
learning

90
Current HILO
Single-
objective
Pareto frontier
learning
Lack agency
Cooperative
AI
Only for
1 user/task
Population
model
Only GUI
Physical
emulation
Low e
ffi
ciency
Constructing
prior
Extract
representative
points
x
y
All observations across users
Initialize
BO
Observed y1
Observed y2
x
y
Warm-Start GP BO with a new user
x
y
New samples

My advisor -
Prof. Antti Oulasvirta
Prof. Sunjun Kim
Dr. Kashyap Todi Dr. John Dudley Prof. Per Ola Kristensson Prof. Byungjoo Lee
Prof. Liwei Chan Dr. Aditya Acharya Antti Keurulainen Prof. Andrew Howes
Dr. Aakar Gupta Dr. Ruta Desai
George B. Mo
Dr. Tanya Jonker Dr. Hrvoje Benko Alec Pierce
Aini Putkonen Dr. Hee-Seung Moon Aleksi Ikkala
Amazing people who
made my Ph.D. studies
a wonderful journey!
91

92
How to get Yi-Chi
http://yichiliao.com
Twitter: @yichiliao
Breaking the Boundaries of Human-in-the-Loop Optimization

Breaking the Boundaries of Human-in-the-Loop Optimization

More Related Content

Similar to Breaking the Boundaries of Human-in-the-Loop Optimization (20)

Recently uploaded (20)

Breaking the Boundaries of Human-in-the-Loop Optimization