Autonomic Computing - Dataflow Programming and Reactive State Machines

AUTONOMIC COMPUTING MEETUP
MARCH 6, 2017
LOS ANGELES

DATAFLOW PROGRAMMING AND REACTIVE STATE MACHINES
Peter Lee @ Corenova Technologies
LA AUTONOMIC COMPUTING MEETUP #3 - MARCH 6, 2017

INTELLIGENCE IS
THE ABILITY TO
ADAPT TO CHANGE.
Stephen Hawking

WHAT IS AN AUTONOMIC SYSTEM?
▸ An approach to enable systems to manage themselves without
direct human intervention
▸ Inspired by human autonomic nervous system which controls
important bodily functions without conscious intervention
▸ A methodology for reducing the complexity of maintaining
distributed systems

THE AUTONOMIC COMPUTING MODEL
▸ A fundamental building block of an autonomic system is
the sensing capability (Sensors Si), which enables the
system to observe its external operational context.
▸ Inherent to an autonomic system is the knowledge of
the Purpose (intention) and the Know-how to operate
itself (e.g., bootstrapping, conﬁguration knowledge,
interpretation of sensory data, etc.) without external
intervention.
▸ The actual operation of the autonomic system is
dictated by the Logic, which is responsible for making
the right decisions to serve its Purpose, and inﬂuence by
the observation of the operational context (based on
the sensor input).

WHAT IS DATAFLOW PROGRAMMING?
▸ A programming paradigm that models a program as a directed graph of the
data flowing between operations, thus implementing dataflow principles and
architecture.
▸ Emphasizes the movement of data and models programs as a series of
connections. Explicitly defined inputs and outputs connect operations, which
function like black boxes. An operation runs as soon as all of its inputs
become valid. Thus, dataflow languages are inherently parallel and can work
well in large, decentralized systems.
▸ Dataflow Programming - Concept, Languages and Applications

BASIC DATAFLOW MODELS
OUTPUT BINPUT A f
OUTPUT D
INPUT B
f
INPUT C
INPUT D f
OUTPUT E
OUTPUT F
REACTOR FUNCTIONS
ONE-TO-ONE
MANY-TO-ONE
ONE-TO-MANY
INPUT F
f
INPUT G
OUTPUT H
OUTPUT I
MANY-TO-MANY

IMPERATIVE VS. DATAFLOW PROGRAMMING
MODULE/SUPERAGENT
HTTP/REQUEST/GET/URL
HTTP/RESPONSE
REQUIRE f
var request = require(“superagent”);
request.get(“http://www.google.com")
request.end((err, response) => {
// logic to deal with response
})
f
IMPERATIVE JAVASCRIPT CODE
require “superagent"
http/request/get/url “http://www.google.com"
DATAFLOW TRIGGERS
synchronous, must
return expected module
asynchronous, must
supply callback function asynchronous, sequence of
triggers doesn’t matter

IMPERATIVE EXAMPLE: REACTING TO ENVIRONMENT
try {
var request = require(“superagent”);
makeRequest(request, “http://www.google.com")
} catch (err) {
var npm = require(“npm”);
npm.load({}, (err, loaded) => {
npm.install(“superagent”, (err, installed) => {
// deal with error, otherwise continue
let request = require(“superagent”)
makeRequest(request, “http://www.google.com")
}
})
}
IMPERATIVE JAVASCRIPT CODE
function makeRequest(request, url) {
request.get(url)
request.end((err, response) => {
// logic to deal with response
})
}
nested async callbacks,
still need to deal with
errors at each level
break out common
function to deal with
multiple entry points for
execution
Logic to deal with missing library dependency, “superagent”

DATAFLOW EXAMPLE: REACTING TO ENVIRONMENT
require “npm”
require “superagent”
http/request/get/url “http://www.google.com"
DATAFLOW TRIGGERS
MODULE/NPM
NPM/INSTALL
REQUIRE
MODULE/SUPERAGENT
HTTP/REQUEST/GET/URL
HTTP/RESPONSEf
f f
NPM/INSTALLEDf
NPM/LOADED
additional dataflow to
support NPM workflows

WHAT IS A DATA PIPELINE?
▸ In computing, a pipeline is a set of data processing elements
connected in series, where the output of one element is the input
of the next one.
▸ Software pipelines, where commands can be written where the
output of one operation is automatically fed to the next, following
operation. The Unix system call pipe is a classic example of this
concept, although other operating systems do support pipes as
well.

TRADITIONAL DATA PIPELINE
FORMAT BFORMAT A f FORMAT Cf
▸ All data chunks of format A runs through a processing node (which internally
buffers incoming/outgoing chunks) and generates stream of format B chunks
▸ All data chunks of format B then runs through another processing node which
generates a stream of format C chunks
▸ You can create a sequence of transforms to achieve desired ﬁnal outcome

DATAFLOW PIPELINE
OUTPUT BINPUT A
OUTPUT Cf
f OUTPUT EINPUT C
OUTPUT FINPUT D
f
INPUT F OUTPUT D
INPUT B
f
CIRCULAR FEEDBACK FLOW!
INPUT OUTPUT

DISTRIBUTED DATAFLOW PIPELINE: MICRO-SERVICES
OUTPUT BINPUT A
OUTPUT Cf
f OUTPUT EINPUT C
OUTPUT FINPUT D
f
INPUT B
ANY DATA f SOCKET ANY DATAfSOCKET
▸ You can build data pipelines across any instance of the Flow running anywhere!

DISTRIBUTED DATAFLOW PIPELINE

CREATIVITY IS JUST
CONNECTING THINGS.
Steve Jobs

KOS: A DATAFLOW STREAMING FRAMEWORK FOR CREATING AWESOME DATA PIPELINES
▸ JavaScript (Node.js and the web browser)
▸ Open-Source License
▸ https://github.com/corenova/kos

THANKS FOR COMING!
SEE YOU NEXT TIME
LA Autonomic Computing Meetup

Autonomic Computing - Dataflow Programming and Reactive State Machines

More Related Content

Viewers also liked (10)

Similar to Autonomic Computing - Dataflow Programming and Reactive State Machines (20)

Recently uploaded (20)

Autonomic Computing - Dataflow Programming and Reactive State Machines