Hybrid rule engines (rulesfest 2010)

●
The SkyNet funding bill is passed.
●
The system goes online on August 4th, 1997.
●
Human decisions are removed from strategic defense.
●
SkyNet begins to learn at a geometric rate.
●
It becomes self-aware at 2:14am Eastern time, August
29th
●
In a panic, they try to pull the plug.
●
And, Skynet fights back
Mark Proctor
Co-Creator
Project Lead

2
Topics
 Rule engines suck
 Hybrid Rule Engines
● Reaction Rules
● Query Rules
● Functional Support
 Conccurrency and Parallization

3
Because Not Everyone
Is As Smart As He Is

5
Fact of Life
Rule Engines Suck

6
Problems
 Limitations of expressiveness
● Inability to cleanly say what you want declaratively
● Verbosity, Control/Semaphores, Emulation
 Execution flow and ordering
● Conflict Resolution is not easy
● Modules (Agenda Groups), RuleFlow Groups
● Concurrency
 Ambiguity
● Rule Recursion
● Unwanted rules
 Extensibility and Pluggability
 Transactions and isolation
 Design Patterns

7
Sequencing
select * from Light l where
seq(l.color) { start: "orange" ->
"red" -> end } )
(Light seq( (color, category)
(start: "orange", "x" ->
"green", “y” -> end ) ) )

8
Sequencing
select * from Light l where
seq(l.color, l.category) {
start: "red", "x" ->
("green", "x" ||
"yellow", "y") -> mid
start: "red", "y" -> mid
mid: "blue", "z" -> end} )
(Light seq( (color, category)
(start: "red", "x" ->
("green", "x" ||
"yellow", "y") -> mid
start: "red", "y" -> mid
mid: "blue", "z" -> end) ) )

10
Life Mission
HMS Suckage
Destroyer

16
The Future
 Full Hybrid Engine
● http://community.jboss.org/wiki/DroolsLanguageEnhancements
●Nested Objects
●Casting Nested Objects
●Positional Constraints
●POSL - Positional-Slotted
Language
●Method Calls
●Maps and Arrays
(Collections)
●Collections and XPath like
filtering
●Free form Expressions
●Managed Object Graphs
(MOGS)
●Nested Patterns and Queries
●Queries and Unification
●Ontologies and Relations via
●Query Based Backward
Chaining with POSL
●Triples with Hybrid POJO Graph
Notation.
●Escapes for Dialects
●Accumulate Improvements to
Support Haskell map/fold/filter
and MVEL projection/fold
● Otherwise
● Branch (Labelled Else)
● Rule Execution Groups
● Rule Dependency Meta-Rule
Language
●Parallel Meta-Rule Language
● Field Versioning
●Logical Closures/OnFalse
●Opportunistic Backward
Chaining, Lazy Field/Object
Values
●...

18
Hyrbid Rule Engine
 Rule Based Programming
● Reactive rules
● Forward Chaining
● Production Rules: Drools, Clips, OPSJ, Jess, JRules
● Query rules
● Backward Chaining
● Prolog: Prova, Open Prolog, SWI Prolog, Visual Prolog
 Functional Programming
● Haskell/Lisp

20
D a te d a te
d o u b le a m o u n t
in t ty p e
lo n g a c c o u n tN o
C a s h flo w
lo n g a c c o u n tN o
d o u b le b a la n c e
A c c o u n t
D a te s ta r t
D a te e n d
A c c o u n tin g P e r io d
Tables

21
Account
accountNo balance
1 0
increase balance for AccountPeriod Credits decrease balance for AccountPeriod Debits
AccountingPeriod
start end
01-Jan-07 31-Mar-07
trigger : acc.balance += cf.amount
Account
balance
1 -25
accountNo
Creating Views with Triggers
date amount type
12-Jan-07 100 CREDIT 1
2-Feb-07 200 DEBIT 1
18-May-07 50 CREDIT 1
9-Mar-07 75 CREDIT 1
CashFlow
accountNo
date amount type
12-Jan-07 100 CREDIT
9-Mar-07 75 CREDIT
CashFlow
date amount type
2-Feb-07 200 DEBIT
CashFlow
trigger : acc.balance -= cf.amounttrigger : acc.balance += cf.amount
select * from Account acc,
Cashflow cf, AccountPeriod ap
where acc.accountNo == cf.accountNo
and
cf.type == CREDIT
cf.date >= ap.start and
cf.date <= ap.end
trigger : acc.balance -= cf.amount
and
cf.type == DEBIT
cf.date <= ap.end

22
date amount type
12-Jan-07 100 CREDIT 1
2-Feb-07 200 DEBIT 1
18-May-07 50 CREDIT 1
9-Mar-07 75 CREDIT 1
CashFlow
accountNo
Account
accountNo balance
1 0
start end
01-Apr-07 30-Jun-07
AccountingPeriod
date amount type
18-May-07 50 CREDIT
CashFlow
date amount type
CashFlow
balance
1 25
accountNo
Creating Views with Triggers
increase balance for AccountPeriod Credits decrease balance for AccountPeriod Debits
and
cf.type == CREDIT
cf.date <= ap.end
trigger : acc.balance -= cf.amount
and
cf.type == DEBIT
cf.date <= ap.end

23
A c c o u n t A c c o u n tin g P e r io d C a s h flo w
v ie w 1 v ie w 2
m a in v ie w
T a b le s
V ie w s
V ie w
A c c o u n t A c c o u n tin g P e r io d C a s h flo w
r u le 1 r u le 2
a g e n d a
O b je c t T y p e s
R u le s
a g e n d a
Rules as Views

24
(rule “increase balance for AccountPeriod Credits”
?ap <- (AccountPeriod)
?acc <- (Account (?accountNo accountNo ) )
(CashFlow { type == CREDIT,
accountNo == ?accountNo,
date >= ?ap.start && <= ?ap.end }
(?amount amount ) )
->
(bind ?acc (balance (+ 1 ?amount) ) )
)
where acc.accountNo == cf.accountNo and
cf.type == CREDIT
cf.date <= ap.end
SQL / S-Expression Rule

25
rule “increase balance for AccountPeriod Credits”
when
ap : AccountPeriod()
acc : Account( $accountNo : accountNo )
CashFlow( type == CREDIT,
accountNo == $accountNo,
date >= ap.start && date <= ap.end,
$amount : amount )
then
acc.balance += $amount;
end
DRL

26
rule “Print blance for AccountPeriod”
salience -50
when
ap : AccountPeriod()
acc : Account()
then
System.out.println( acc.accountNo + “ : “ acc.balance );
end
Salience
Agenda
1 increase balance
arbitrary2 decrease balance
3 increase balance
4 print balance
Conflict Resolution with Salience

28
Same Author Rule
rule sameAuthor
$i1 : Item( )
$i2 : Item( author == $i1.author )
end
select * from
Item i1, Item i2,
where
i1.author == i2.author
i1 != i2
create query(String author)
select * from
Item i1, Item i2,
where
i1.author == author
i1.author == i2.author
i1 != i2

29
Same Author Query
query sameAuthor (Item $i1, Item $i2)
$i1 : Item( )
$i2 : Item( this != $i1, author == $i1.author )
end
Book Author
Poems Tom
Songs Tom
Cars Bob
Largs Cars Jane
 sameAuthor( item1, item2 )
● both are output vars
● returns full cross product
 sameAuthor(new Item( new Book( Poems), item )
● Item1 is input var
● Item2 is output var
● Returns Songs by Tom

30
Recommendations
query sameSubject (Item $i1, Item $i2)
$i1 : Item( )
$i2 : Item( this != $i1 )
exists( $s1 : ItemSubject(item == $i1)
ItemSubject(item == $i2, subject == $s1.subject)
)
end
query recommended (Item $i1, Item $i2)
$i1 : Item( )
$p1 : Purchase( item == $i1 )
$c : Customer( this == $p.customer )
$p2 : Purchase( customer == $c, item != $i1 )
$i2 : Item( this == $p2.item )
( ?sameAuthor( $i1, $i2 ) or ?sameSubject( $i1, $i2 ) )
end
query sameAuthor (Item $i1, Item $i2)
$i1 : Item( )
$i2 : Item( this != $i1, author == $i1.author )
end
'?' incalls the query.
Bound vars are inputs.
Unbound vars are outputs.

32
Functional Programming
Basic Haskel Groups
 List Processing
 Arithmetic, Higher Order Functions
 Tuples
Arithmatic
 Average
● avg [12, 4, 5, 9]
● 7.5

33
Functional Programming
Higher Order Functions
Map, Fold and Filter
 Fold
● Fold (+) 0 [1, 2, 3, 4]
● 10
 Map
● map (*2) [1, 2, 3]
● [2, 4, 6, 8]
 Filter
● Filter (>2) [3, 4, 5]

34
rule "init" when
then
insert( new Acc( "sum red buses",
STAGE.ACC_START ) );
end
rule "action" when
$b : Bus( colour == "red" )
$acc : Acc( name == "sum red buses",
stage == STAGE.ACC_START )
then
$acc.sum( $b.takings ); // do not notify the engine
end
Accumulate CE

35
Accumulate CE
rule "result" when
salience -5
$acc : Acc( name == "sum red buses",
stage == STAGE.ACC_START )
then
modify( $acc ) { stage = STAGE.ACC_END };
End
rule "evaluate result" when
$acc : Acc( sum > 100,
stage == STAGE.ACC_END )
then
print "sum is “ + $acc.sum;
end

36
Functional Support
select
c, sum(bi.price) t
from
Customer c, BaskteItem bi
where
bi.customer == c
group by
c
?c <- (Customer )
acc( ?bi <- (BasketItem {customer == ?c} )
?t <- (sum ?bi.price) )

37
Functional Support
select
c, bi.type type, sum(bi.price) t
from
where
bi.customer == c
group
by c , bi.type
?c <- (Customer)
acc( ?bi <- (BasketItem {customer == ?c} )
?type <- (distinct ?bi.price) )
acc( ?bi <- (BasketItem {customer == ?c,
type == ?type} )
?t <- (sum ?bi.price) )

38
DRL
select
c, sum(bi.price) t
from
where
bi.customer == c
group by
c
$c : Customer()
acc( $bi : BasketItem( customer == $c );
$t : sum( $bi.price) )

39
Nested
acc( acc( $c : Customer()
$bi : BasketItem( customer == $c );
$t : sum( $bi.price) );
$a : avg( $t ) )
 Average the summed expenditure for all customers

40
Nested
$c : Customer()
acc( $bi : BasketItem() from
acc( $bi : BasketItem( customer == $c );
filter( $bi, $bi.price > 100) );
$a : avg( $bi.price ) )
 Average price for all item with a price over 100
Patterns can also filter “from”
an acc result.

43
Discount Alert
rule "alert customer to potential band change"
$c : Customer()
acc( $bi : BasketItem(customer == $c )
$s : sum( $bi.price ) )
acc( $bi : BasketItem(customer == $c )
?recommended( $bi.item, $rec )
$rs : sum( $rec.price ) )
$b : Band( this != $c.band, min >= ($s + $rs),
max <= ($s + $rs) )
then
println( “If you buy all items and all recommendations,
you move up a band with a new
discount of “ + $b.discount );
end
'$rec' is unbound
and thusan output var

Concurrency and Parallelization

45
C : Customer
[John,
Sally]
B : BasketItem
b.customer == c
[John,
Sally]
Simple Rete Rule

46
Simple Rete Rule
 insert “sally”
● propagate to join
● Return as right memory is empty
 insert “john”
● Return as right memory is empty

47
Simple Rete Rule
C : Customer
[John,
Sally]
[BasktItem( John, Book, Cars ),]
[John,
Sally]
[ [John,BasktItem( John, Book, Cars )] ]
B : BasketItem
b.customer == c
[BasktItem( John, Book, Cars )]
Token
Node node
Token leftParent
Token rightParent
List<Token> children

48
Simple Rete Rule
 Insert BasktItem( John, Book, Cars )
● foreach( l-token : left-memory )
if ( r-token.customer == l-token )
create new Token and propagate
[John, BasktItem( John, Book, Cars )]
Token
Node node
Token leftParent
Token rightParent

49
Simple Rete Rule
C : Customer
[John,
Sally]
[BasktItem( John, Book, Cars ),
BasktItem( John, Book, Poems ),
BasktItem( Sally, Book, Cinema )]
[John,
Sally]
[ [John,BasktItem( John, Book, Cars )],
[John, BasktItem( John, Book, Poems ) ],
[Sally, BasktItem( Sally, Book, Cinema ) ] ]
B : BasketItem
b.customer == c
[BasktItem( John, Book, Cars ),
BasktItem( John, Book, Poems ),
BasktItem( Sally, Book, Cinema )]

53
Partitioning for Parellelization

54
Task Oriented
 Submit each Token propagation to Executor pool
● The join node is now an executable job
 Wins
● Only the node is locked, allowing a “pipeline”
● Possible to lock only the current index bucket
● Solves recursive query problem of large stack frames
 Loses
● Only a small amount of time in each node
● Each node repeatedly locked and scheduled, for a single WM
action, due to tree “to leaf” traversal
● Previous papers show increases for only some use cases
● Network thrashing for subnetworks on accumulate

55
Subnetwork Thrashing
$c : Customer()
acc( $b : BasketItem(customer == $c )
$d : Discount( type == $b.type);
$s : sum( $b.price * $d.multiplier ) )
Band( this != $c.band, min >= $s, max <= $s )

56
Set Oriented
 Propagate Collections instead of Tokens
● Left Input trigger
● Iterate right memory create a collection of all matching right
tokens
● Note multi-dimesional array of 1
● [[l1[r1, r2, r3]]]
● Right Input trigger
● Iterate left memory create a collection of all left joins
● Note multi-dimensional array for each left token
● [ [l1, r1],
[l2, r1]
[l3, r1] ]
Token
Node node
Token leftParent
Token rightParent

57
Set Oriented
 Wins
● More work done in a node
● scalable task scheduling
● Less indirection so better statement execution
● All the work for that node done for that WM action in one task
● Single network pass, so nodes locked just once per WM action
● Avoids repeatedly creating and destroying method stack
frames
● Avoids sub network thrashing
 Loses
● More complex algorithm
● Especially on “collection” merging

58
MVCC
 Modifies
● Clone + new values and current transaction id
● Existing instance still there with older transaction id
● Node uses constraint to join with correct version
 Isolation
● Scoped to rule or rule-group
● Scoped rule(s) see new data, others old
● Non blocking so reads (joins) can still continue
 Fully Concurrent safe Queries
● Queries leave no state in join
● Node joins with correct instance, based on transaction id

59
Questions?Questions?
• Dave Bowman: All right, HAL; I'll go in
through the emergency airlock.
• HAL: Without your space helmet, Dave,
you're going to find that rather difficult.
• Dave Bowman: HAL, I won't argue with
you anymore! Open the doors!
• HAL: Dave, this conversation can serve
no purpose anymore. Goodbye.
Joshua: Greetings, Professor Falken.
Stephen Falken: Hello, Joshua.
Joshua: A strange game. The only
winning move is not to play. How
about a nice game of chess?

Hybrid rule engines (rulesfest 2010)

More Related Content

What's hot (20)

Similar to Hybrid rule engines (rulesfest 2010) (20)

More from Geoffrey De Smet (18)

Recently uploaded (20)

Hybrid rule engines (rulesfest 2010)