Module 10: CDB Subscribers

1©2015 Tail-f Systems all rights reservedFebruary 13, 2015
CDB Subscribers

Triggers when
anything in
scope changes
Subscribe to Configuration Changes
subsock = socket(PF_INET, SOCK_STREAM, 0);
cdb_connect(subsock, CDB_SUBSCRIPTION_SOCKET, &confd, size);
cdb_subscribe(subsock, 3, dhcpd__ns, &spoint, "/dhcp");
cdb_subscribe_done(subsock);
/ dhcp/
aaa/
voip/
defaultLe…
maxLeas…
SubNets/ subNet/
mask
range
lowAddr
highAddr
logFacility
net
Managed Objects typically have
several subscription points

Enter Event Loop
• Event loop needs to monitor subscription socket
• Typically using poll()/select()
while(1) {
set[0].fd = subsock;
set[0].events = POLLIN;
set[0].revents = 0;
poll(set, 1, -1); // Monitor one socket, wait forever
if (set[0].revents & POLLIN) {
cdb_read_subscription_socket(subsock, sub_points, &reslen);
// React on changes here…
// sub_points[] contains triggering subscription points,
// reslen how many
}
cdb_sync_subscription_socket(subsock, CDB_DONE_PRIORITY);
}

React to Configuration Changes
• Triggered subscription means something in scope changed, but will not tell us
what (nor to what value)
• To find out more, read all or start a diff walk
/ dhcp/
aaa/
voip/
defaultLe…
maxLeas…
SubNets/ subNet/
mask
range
lowAddr
highAddr
logFacility
net
?
?

Left extreme:
On any change, read all
configuration
• Linux daemon style
• Simple to program
• Simple to test
• Maximum code reuse
• Robust
Right extreme:
On any change, look at
every changed element
• NET-SNMP agent style
• In some cases very much
more efficient

Left extreme:
void startup() {
init_obj_A();
init_obj_B();
init_obj_C();
}
void handle_change() {
if(trigger_A) init_obj_A();
if(trigger_B) init_obj_B();
if(trigger_C) init_obj_C();
}
Right extreme:
void startup() {
init_obj_A();
init_obj_B();
init_obj_C();
}
cdb_diff_iterate(…iterator…);
}
void iterator(…keypath…) {
if(keypath is field /dhcp/logFacility)
...
if(keypath is field /…) …

No need to be extreme
Use balanced approach
Walk the diff when it makes sense, but not down to
individual leaf elements
Use multiple subscription points
• Simple to program with high degree of reuse
• Simple to test
• Robust
• Efficient

void startup() {
init_obj_A();
n = cdb_num_instances(“/B”);
for(i=0; i<n; i++)
init_obj_B(i);
init_obj_C();
}
if(trigger_A) init_obj_A();
if(trigger_B)
cdb_diff_iterate(…iterator_B…)
if(trigger_C) init_obj_C();
}
void iterator_B(…keypath…) {
table_row = extract_row(keypath);
init_obj_B(table_row);
return ITER_CONTINUE;
}

Walking the Diff in a Balanced Way
Find the right level of granularity for your subscriptions and diff walks
/ dhcp/
aaa/
voip/
defaultLe…
maxLeas…
SubNets/ subNet/
mask
range
lowAddr
highAddr
logFacility
net

Event loop with balanced approach
while(1) {
set[0].revents = 0;
for(i=0; i<reslen; i++) {
if(…) cdb_diff_iterate(subsock, sub_points[i], iter, ITER_WANT_PREV, 0);
else if(…) init_A();
}
}
}

An iterator function might look like this
static enum cdb_iter_ret iter(confd_hkeypath_t *kp, enum cdb_iter_op op,
confd_value_t *oldv, confd_value_t *newv, void *state)
{
switch (op) {
case MOP_CREATED: {
confd_value_t *ctag = &kp->v[1][0];
switch (CONFD_GET_XMLTAG(ctag)) {
case root_RFHead: { // an rfhead was created
// keypath is /root/NodeB/RFHead{$key}
// 3 2 1 0
confd_value_t *hkey = &kp->v[0][0];
read_head(cdbsock, hkey);

• Each item seen by the iterator function is one of
• MOP_CREATED – container or leaf created
• MOP_DELETED – container or leaf deleted
• MOP_MODIFIED – container where a child element has been
created/deleted/set
• MOP_VALUE_SET – leaf that has been set
• MOP_MOVED_AFTER – list element that was moved
• Each time iterator function returns
• ITER_RECURSE – examine changes to child elements
• ITER_CONTINUE – skip changes to child elements
• ITER_UP – skip changes to child and sibling elements
• ITER_SUSPEND – suspend the iteration temporarily
• ITER_STOP – stop iteration now

int main(…) {
cdb_subscribe(subsock, 3, root__ns,
&headpoint,"/root/NodeB/RFHead”);
…
cdb_diff_iterate(…, iter, …);
}
}
static enum cdb_iter_ret iter(…) {
…
switch (op) {
case MOP_CREATED: {
fprintf(stderr, "Create: %sn",…);
…
Diff Walk Example
container root {
container NodeB {
list RFHead {
key dn;
leaf dn { type int64; }
leaf SECTORID_ID {
type string; default "N/A";
}
list Child {
key cdn;
leaf cdn { type int64; }
leaf childAttr {
type string; default "N/A";
}
}
}
}
}

Create Object with Attribute
% set root NodeB RFHead 2 SECTOR_ID 2
% commit
Create: /root/NodeB/RFHead{2}
ITER_CONTINUE

Modify Attribute of Object
% set root NodeB RFHead 2 SECTOR_ID 5
% commit
Modified /root/NodeB/RFHead{2}
ITER_RECURSE
Value Set: /root/NodeB/RFHead{2}/SECTORID_ID --> (5)
ITER_CONTINUE

Create Object with Child Object
% set root NodeB RFHead 3 Child 1 childAttr 1
% commit
ITER_CONTINUE

Create additional Child Object
% commit
ITER_RECURSE
Create: /root/NodeB/RFHead{3}/Child{2}
ITER_CONTINUE
ITER_CONTINUE

Set Attribute of Child Object
% commit
ITER_RECURSE
Modified /root/NodeB/RFHead{3}/Child{2}
ITER_RECURSE
Value Set: /root/NodeB/RFHead{3}/Child{2}/childAttr --> (32)
ITER_CONTINUE

Delete Child Object
% delete root NodeB RFHead 3 Child 1
% commit
ITER_RECURSE
Delete: /root/NodeB/RFHead{3}/Child{1}
ITER_CONTINUE

Delete Everything
% delete root
% commit
Delete: /root/NodeB/RFHead{2}
ITER_CONTINUE
Delete: /root/NodeB/RFHead{3}
ITER_CONTINUE

Hashed Key Path (hkeypath)
• To make it easy to program (not necessarily learn)
• To make it efficient
• Encoded as two dimensional array
/root/NodeB/RFHead{1} would be encoded
• kp[0][0] contains ‘1’
• kp[1][0] contains ‘RFHead’
[0][] [1][] [2][] [3][] [4][]
[][0] INT32
‘1’
XMLTAG ‘RFHead’ XMLTAG
‘NodeB’
XMLTAG
‘root’
NOEXISTS
[][1] NOEXISTS NOEXISTS NOEXISTS NOEXISTS NOEXISTS

• The second index is for multiple keys
/routes/route{12.60.0.0 16}/enabled would be encoded
• kp[1][0] contains ’12.60.0.0’
• kp[1][1] contains ‘16’
• There may be up to 8 keys in any table
• I.e. second index ranges from 0 up to possibly 7
[0][] [1][] [2][] [3][] [4][]
[][0
]
XMLTAG
‘enabled’
IPV4
‘12.60.0.0’
XMLTAG
‘route’
XMLTAG
‘routes’
NOEXISTS
[][1
]
NOEXISTS INT8
’16’
NOEXISTS NOEXISTS NOEXISTS

• Sequence of XMLTAG hash values
• An XMLTAG is encoded as a 31-bit integer
• Not as strings: variable length, inefficient with string compare
• Hash function ensures same string always gets same value
• Collisions are rare (we have never seen one to date)
• Use idValue attribute in data model in case of collision
• This allows efficient and easy switch-case statements
switch (CONFD_GET_XMLTAG(ctag)) {
case root_RFHead: { // an rfhead was created
• Header file contains all the hash-constants
• Header file generated from data model using confdc

Ensuring Changes happen in Order
• Consider transaction A
• Add interface eth5
Add route 55.66.0.0/18 over interface eth5
• Transactions are all-at-once, there is no internal ordering
• Transaction A is therefore equivalent to
• Add route 55.66.0.0/24 over interface eth5
Add interface eth5

• Consider transaction A
• Add interface eth5
Add route 55.66.0.0/18 over interface eth5
• Still, in our code, the interface subsystem must be informed of this change before the
routing subsystem
• The routing subsystem will call the interface subsystem referencing eth5
• This dependency information is encoded in
cdb_subscribe(subsock, 20, dhcpd__ns, &spoint, ”/interfaces");
cdb_subscribe(subsock, 30, dhcpd__ns, &spoint, ”/routes");

• Consider then transaction B
• Delete interface eth5
Delete route 55.66.0.0/24 over interface eth5
• In this case the ordering must be reversed
• Again, the routing subsystem will call the interface subsystem and reference eth5
• This is handled by adding more subscription points
// This subscription handles additions and modification of interfaces
cdb_subscribe(subsock, 20, dhcpd__ns, &spoint1, ”/interfaces");
cdb_subscribe(subsock, 30, dhcpd__ns, &spoint2, ”/routes");
// This subscription handles deletes of interfaces
cdb_subscribe(subsock, 40, dhcpd__ns, &spoint3, ”/interfaces");

Event loop with balanced approach
while(1) {
set[0].revents = 0;
for(i=0; i<reslen; i++) {
if(…) cdb_diff_iterate(subsock, sub_points[i], iter, ITER_WANT_PREV, 0);
else if(…) init_A();
}
}
}

Application Initialization / Start Order
• Read, Subscribe
• Protect using read session or operators locked out
• Potentially more efficient than trigger method
• Subscribe, Each application triggers its own subscriptions
• Must be able to handle all configuration changes anyway
• Subscribe, Startup process triggers all subscriptions
• Good at system start when all processes start at the same time
• Subscribe, System process triggers restarting app subscriptions
• Can handle all scenarios

Yang
Validation
Application
Validation
Prepare
Phase
Notify
Applications
Commit
Phase
Point of No Return
After the Point of No Return the transaction cannot be aborted.
• Make sure you validate properly before Point of No Return
• If an application fails after this point, treat it the same regardless of it happening
2 ms/minutes/weeks after commit
Operator
types/clicks
commit
Operator gets OK
and next prompt
Point of
No Return

Yang
Validation
Application
Validation
Prepare
Phase
Notify
Applications
Commit
Phase
Validators and Two Phase Subscribers
Resource reservation can be handled by a
Two Phase Subscriber
• Called during Prepare Phase
• Will get an aborted
callback if transaction
aborted
• Uses cdb_subscribe() mechanism
with an extra flag
• Don’t activate, just check in prepare
MO
Changes that
might happen
Changes that did
happen

ConfD CDB Subscription Handling Examples
• examples.confd/intro/1-2-3-start-query-model
• examples.confd/cdb_subscription/iter
• examples.confd/cdb_subscription/trigger
• examples.confd/cdb_subscription/twophase

Module 10: CDB Subscribers

More Related Content

What's hot (20)

Viewers also liked (9)

Similar to Module 10: CDB Subscribers (20)

Recently uploaded (20)

Module 10: CDB Subscribers

Editor's Notes