Pattern-Based Debugging of Declarative Models

Pattern-Based
Debugging of
Declarative Models
Vajih Montaghami and Derek Rayside
{vmontagh,drayside}@uwaterloo.ca
Electrical and Computer Engineering
University of Waterloo Waterloo, Ontario, Canada
Fri 2 Oct 2015

Pattern-Based Debugging of Declarative Models {vmontagh,drayside}@uwaterloo.ca
Say what you mean
Adapted from: http://cloudcomputingcell.com/computer-programmer-images/
sig Node{next:
Node} 
one sig Head
extends Node{}
pred
StructuralConstr
aints{
all n: Node |
head
2

Say what you mean
sig Node{next:
Node} 
one sig Head
extends Node{}
pred
StructuralConstr
aints{
all n: Node |
head
φintended
2

Say what you mean
sig Node{next:
Node} 
one sig Head
extends Node{}
pred
StructuralConstr
aints{
all n: Node |
head
φexpressed φintended
2

Say what you mean
sig Node{next:
Node} 
one sig Head
extends Node{}
pred
StructuralConstr
aints{
all n: Node |
head
⇎φexpressed φintended
2

Engineer intends to model a linear linked list
sig Node{next:
Node} 
one sig Head
extends Node{}
pred
StructuralConstra
ints{
all n: Node | one
n.next}
head
3

Engineer intends to model a linear linked list
sig Node{next:
Node} 
one sig Head
extends Node{}
pred
StructuralConstra
ints{
all n: Node | one
n.next}
head
head
3

sig Node{next:
Node} 
one sig Head
extends Node{}
pred
StructuralConstra
ints{
all n: Node | one
n.next}
head
head
✗Underconstraint
But, what engineer expresses is not what he intended
3

sig Node{next:
Node} 
one sig Head
extends Node{}
pred
StructuralConstra
ints{
all n: Node | one
n.next}
head
∅
head
✗Underconstraint
But, what engineer expresses is not what he intended
3

sig Node{next:
Node} 
one sig Head
extends Node{}
pred
StructuralConstra
ints{
all n: Node | one
n.next}
head
∅
head
✗Underconstraint
Engineer Strengthens the constraints, but gets nothing!
✗Full-Overconstraint
3

sig Node{next:
Node} 
one sig Head
extends Node{}
pred
StructuralConstra
ints{
all n: Node | one
n.next}
head
∅
head
head
✗Underconstraint
Engineer Strengthens the constraints, but gets nothing!
3

sig Node{next:
Node} 
one sig Head
extends Node{}
pred
StructuralConstra
ints{
all n: Node | one
n.next}
head
∅
head
head
✗Underconstraint
Engineer Weakens the constraints, but excludes some examples
✗Partial-Overconstraint
3

sig Node{next:
Node} 
one sig Head
extends Node{}
pred
StructuralConstra
ints{
all n: Node | one
n.next}
head
∅
head
head
✗Underconstraint
Engineer Weakens the constraints, and gets intended examples
✗Partial-Overconstraint
✓
head
headhead
3

Pattern-Based Debugging of Declarative Models {vmontagh,drayside}@uwaterloo.ca4
Can we invent a tool to
guide this strengthening
and weakening of the
model?

Idea: Discriminating Examples
5
φintended
φexpressed

5
φintended
φexpressed example

5
φintended
φexpressed example discriminating
example

Mean what you say
No bug φintended
⇔ φexpressed
Underconstraint φintended
⇒ φexpressed
Partial-overconstraint φintended
⇐ φexpressed
Fully-overconstraint (︎∄i: I| i ⊨ φexpressed) ∧ (∃j: I| j ⊨ φintended)
Loosely Intersecting
Intention
∃i: I | (i ⊨ φintended ⋀ φexpressed) ⋀
(i ⊭ φintended ⇒ φexpressed) ⋀ (i ⊭ φexpressed
⇒ φintended)
6

Mean what you say
No bug φintended
⇔ φexpressed
Underconstraint φintended
⇒ φexpressed
Partial-overconstraint φintended
⇐ φexpressed
Fully-overconstraint (︎∄i: I| i ⊨ φexpressed) ∧ (∃j: I| j ⊨ φintended)
Loosely Intersecting
Intention
∃i: I | (i ⊨ φintended ⋀ φexpressed) ⋀
(i ⊭ φintended ⇒ φexpressed) ⋀ (i ⊭ φexpressed
⇒ φintended)
6
φintended
φexpressed

Pattern-based debugging
Existing tools Pattern-based
Underconstraint
Random examples
Minimal examples
Discriminating examples
Help the engineer decide if certain properties
need to be strengthened or weakened
Overconstraint
UnSAT-Core Subset of UnSAT-Core
Accepting or Rejecting discriminating
examples localizes area to fix
Explanation None By Pattern Semantics
7
Better Understanding
Increase Localization
Best-effort Fix suggestion

Context
❖ Model: A text written in a logical
language
❖ Bug: a divergence between the
engineers intended logical model and
the expressed logic model
8
sig Node{next: Node} 
one sig Head extends Node{}
pred StructuralConstraints{
all n: Node | one n.next}
pred Noloop{
all n: Node | n !in n.^next}
pred showLists{
StructuralConstraints
Noloop}
run showLists
❖ Logical language: Alloy
๏ First-order relational logic with closure
๏ Alloy Analyzer: Instance finder equipped
with Inconsistent core finder in finite domain
๏ Bugs:
- Unintended instances
- Inconsistent constraints

Generating Discriminating Examples
9
Engineer: expressed
her intention φintended

9
Engineer: expressed
φexpressed
Debugger: Infer Semantic
⇒ p[R] ⋀ …
P[R]: Pattern P
applied to relation R

9
Engineer: expressed
φexpressed
⇒ p[R] ⋀ …
P[R]: Pattern P
Debugger: Take a guess
that the model
might be underconstrained
w.r.t. φintended
p’ [R]
Weaken p to p’
p[R] p’ [R]
Strengthening is in reverse

9
Engineer: expressed
φexpressed
⇒ p[R] ⋀ …
P[R]: Pattern P
that the model
w.r.t. φintended
p’ [R]
Weaken p to p’
p[R] p’ [R]
Debugger: Mutate
the semantic
p’ [R] ⋀ ¬p[R] ⋀ …

9
Engineer: expressed
φexpressed
⇒ p[R] ⋀ …
P[R]: Pattern P
that the model
w.r.t. φintended
p’ [R]
Weaken p to p’
p[R] p’ [R]
Debugger: Mutate
the semantic
Debugger: Find
discriminating example
p’ [R] ⋀ ¬p[R] ⋀ …

9
Engineer: expressed
φexpressed
⇒ p[R] ⋀ …
P[R]: Pattern P
that the model
w.r.t. φintended
p’ [R]
Weaken p to p’
p[R] p’ [R]
Debugger: Mutate
the semantic
Debugger: Find
discriminating example
p’ [R] ⋀ ¬p[R] ⋀ …
Engineer: accepts or
rejects the discriminating
example

Patterns
Binary relations
❖ Relational properties
❖ 21 are commonly known
❖ used in structural modelling
❖ e.g.: acyclic, antisymmetric,
irreflexive, …
Ternary relations
❖ Inclusion and Ordering
❖ we define 160
❖ used in behavioural modelling
❖ e.g.: ExpandHeadOfRight, … 
(ece.uwaterloo.ca/~vmontagh/alloy/debugger/)
10

Pattern-based Debugger’s components
11

Pattern-based Debugger’s components:
UI: DiscriminatingExample
12
❖ Back to the Linked list example
❖ Debugger finds: φexpressed ⇒ Total_Function[next],…
❖ Debugger generates a discriminating example, w.r.t.:
E ⊨ ¬Total_Function[next] ⋀ Partial_Function[next]
Engineer accepts E
φexpressed is overconstrained w.r.t. φintended
Fix by weakening
Engineer rejects E
Generate another example
❌✓

Knowledge-base
Consistency Graph
❖ Undirected graph
❖ Nodes: Patterns
❖ Edges: Two nodes are consistent
over the same relation
❖ Nodes: 160 Edges: 6764
Implication Lattice
❖ Directed graph
❖ Nodes: Patterns
❖ Edges: The source node implies the
sink node over the same relation
❖ Sources:12 Sinks:6 Longest Path:6
Acyclic Irreflexive
Acyclic Partial-function
13

Reasoner
Generate discriminating examples
❖ Infer Semantic
❖ Semantic Mutation
❖ Generation tactics (see paper)
❖ antecedent satisfaction,
❖ disjunction vacuity detection
❖ conjunction vacuity detection
14
Localize the issue
❖ UnSAT-Core
❖ Correspondence tactics (see paper)
❖ quantifier unrolling
❖ function and predicate inlining

A case study
❖ Dijkstra’s Dining Philosophers algorithm
❖ The bogus model shipped with Alloy for years
❖ Was fixed in 2012 by team from KIT Germany
๏ Fix is subtle and was not explained in their publications
❖ In four interactions, the pattern-based debugger assists
the engineer to
Understand, Localize, and Fix the defects
Source: wikipedia.org
15

Summary and Future work
Now
❖ Pattern-based debugging
๏ Discriminating examples
๏ Library of ternary patterns
๏ Generation tactics
๏ Correspondence tactics
❖ Dining Philosophers case study
16
Next
❖ Automating search
procedure
❖ Improve pattern library
❖ More case studies

Pattern-Based Debugging of Declarative Models

More Related Content

What's hot (6)

Viewers also liked (10)

Similar to Pattern-Based Debugging of Declarative Models (20)

Recently uploaded (20)

Pattern-Based Debugging of Declarative Models