Exploiting Structure and Behavior of Highly Configurable Systems to Measure Performance

Exploiting Structure and
Behavior of Highly
Conﬁgurable Systems to
Measure Performance
Miguel Velez
Software Engineering Ph.D. student
Carnegie Mellon University
Joint work with: Christian Kästner, Pooyan Jamshidi, and
Norbert Siegmund
1

Most software is highly conﬁgurable
2

For developers and users
Overwhelmed by choices
Lose track of interactions
Measuring performance in HCS is difﬁcult
3

“For each valid conﬁguration, x is the y you will get”
y = quality attribute x = actual value
In an ideal world
4
Conﬁguration Execution Time
A=0 B=0 C=0 D=0 6s
A=0 B=0 C=1 D=1 6s
A=1 B=0 C=0 D=1 9s
… …
T = 3A + 1AB + 6

Black-Box Approach
5
A B C D T
1 0 0 0 0 6
2 0 0 0 1 6
3 0 0 1 0 6
4 0 0 1 1 6
5 0 1 0 0 6
6 0 1 0 1 6
7 0 1 1 0 6
8 0 1 1 1 6
9 1 0 0 0 9
10 1 0 0 1 9
11 1 0 1 0 9
12 1 0 1 1 9
13 1 1 0 0 10
14 1 1 0 1 10
15 1 1 1 0 10
16 1 1 1 1 10

Black-Box Approach
6
A B C D T
1 0 0 0 0 6
2 0 0 0 1 6
3 0 0 1 0 6
4 0 0 1 1 6
5 0 1 0 0 6
6 0 1 0 1 6
7 0 1 1 0 6
8 0 1 1 1 6
9 1 0 0 0 9
10 1 0 0 1 9
11 1 0 1 0 9
12 1 0 1 1 9
13 1 1 0 0 10
14 1 1 0 1 10
15 1 1 1 0 10
16 1 1 1 1 10

Black-Box Approach
7
A B C D T
1 0 0 0 0 6
2 0 0 0 1 6
3 0 0 1 0 6
4 0 0 1 1 6
5 0 1 0 0 6
6 0 1 0 1 6
7 0 1 1 0 6
8 0 1 1 1 6
9 1 0 0 0 9
10 1 0 0 1 9
11 1 0 1 0 9
12 1 0 1 1 9
13 1 1 0 0 10
14 1 1 0 1 10
15 1 1 1 0 10
16 1 1 1 1 10

Black-Box in practice
8
A B C D T
1 0 0 0 0 6
2 0 0 0 1 6
3 0 0 1 0 6
4 0 0 1 1 6
5 0 1 0 0 6
6 0 1 0 1 6
7 0 1 1 0 6
8 0 1 1 1 6
9 1 0 0 0 9
10 1 0 0 1 9
11 1 0 1 0 9
12 1 0 1 1 9
13 1 1 0 0 10
14 1 1 0 1 10
15 1 1 1 0 10
16 1 1 1 1 10

9
A B C D T
1 0 0 0 0 6
2 0 0 0 1 6
3 0 0 1 0 6
4 0 0 1 1 6
5 0 1 0 0 6
6 0 1 0 1 6
7 0 1 1 0 6
8 0 1 1 1 6
9 1 0 0 0 9
10 1 0 0 1 9
11 1 0 1 0 9
12 1 0 1 1 9
13
1 1 0 0 10
14 1 1 0 1 10
15 1 1 1 0 10
16 1 1 1 1 10
Black-Box in practice

Want to ﬁnd interactions
Identify what we need to measure
Determine if we do not need to measure all
conﬁgurations
Ideally
10

Our Proposal: Exploit Structure and
Behavior of the Program to Measure
Performance
11

Determine how options are used
Identify interactions
Compare execution traces based on
conﬁgurations
Know if we need to execute all conﬁgurations or
subset
What We Can Do With Structure and Behavior
13

Approach: Combine Static Taint Analysis
with Dynamic Analysis
14

Relevant Statements and Options
27

28
Relevant Statements and Options

C
Java
RQ: How often can we exploit lack of
interactions to measure performance?
Apply in Real Programs
38

Augment Tracking Information
39

40

41

• From Black-Box to White-Box
• Static Taint Analysis with Dynamic Analysis
• New pruning strategies in real programs
Summary
42
Miguel Velez
mvelezce@cs.cmu.edu

Exploiting Structure and Behavior of Highly Configurable Systems to Measure Performance

More Related Content

Similar to Exploiting Structure and Behavior of Highly Configurable Systems to Measure Performance (20)

Recently uploaded (20)

Exploiting Structure and Behavior of Highly Configurable Systems to Measure Performance