AWS IoT 핸즈온 워크샵 - 실습 6. 긴급 데이터를 Kinesis Streams으로 보내기 (김무현 솔루션즈 아키텍트)

For this lab were going to look at a combination of systems to capture different kinds of data streams. We'll
start with the usual AWS IoT system using rules to capture our data but during certain events, as determined
by our rules we'll activate a Direct-To-Kinesis stream to capture realtime information.
Why would you ever do this? AWS IoT has limits in place for the size of each message but also the number of
transactions per second. When streaming realtime telemetry information, if you ﬁnd you need more than
50TPS you would want to consider going to directly to Kinesis.
Let's setup a Kinesis Shard. Let's setup a single shard to capture our stream when it is activated.
Open the Kinesis dashboard and pick Kinesis Streams.
Click Create Stream.
Let's call this "iotshard", you'll enter just 1 shard.
Lab 9 - Switch to Kinesis during critical events
Step 1

We're going to setup some Lambda functions to handle logging for us so we can keep track of our "thing".
Open the Lambda dashboard.
Click Create a lambda function.
You can use the hello-world blue print.
Let's call this function, "iotLambda2", accept the basics.
You can also enter the code below to turn on some basic logging.
Step 2

Note: We are not creating these function in a VPC for this lab.
Next we need to create another Lambda function to capture our Kinesis stream data.
While in the Lambda dashboard, create a new function.
Use the Kinesis-Process-Record blue print.
Select the Kinesis Stream you created in step 1.
Leave the other options as their defaults.
We are going to call this iotLambda3.
Make sure your role is a Kinesis Role.
On the review screen, make sure you pick "Enable Now" so the event source is active.

Here is the code if you want to copy & paste:
console.log('Loading function');
exports.handler = function(event, context) {
console.log('IoT Kinesis Event:', JSON.stringify(event, null, 2));
event.Records.forEach(function(record) {
// Kinesis data is base64 encoded so decode here
var payload = new Buffer(record.kinesis.data, 'base64').toString('ascii');
console.log('Decoded payload:', payload);
});
context.succeed("Successfully processed " + event.Records.length + " records.");
};
Lastly we need a Lambda function that can publish a change to our shadow state to turn on Kinesis for our
"thing".
Create another new Lambda function, use the hello-world blue print.
We will call this iotShadowUpdate.
The code for this is shown below.
JavaScript

You will need to pick an execution role that has access to the IoT service.
Shadow Update Lambda code is as follows:

Open the AWS IoT Dashboard.
Click create resource and pick a new rule.
Let's call this iotCritical.
We'll select * from our iotCritical topic.
Pick the default action as Lambda.
Pick iotLambda2 that we created earlier.
Let's now create the Kinesis rule.
Click create resource and pick a new rule again.
Let's call this iotKinesis.
We'll select * again from our iotCritical topic.
We want to add a condition this time, temperature > 100.
Pick the default action as Lambda.
Pick the iotShadowUpdate we created earlier.
Step 4

Our script will now connect and send regular AWS IoT messages but one of these will be a high temperature
value that will trigger our state change, once we detect the change we'll stream to Kinesis.
Let's create a new script for this lab. We'll call this "iotCritical.js". This script is going to be using the full AWS
SDK. If you don't have that installed you can run
npm install aws-sdk
We are also using a sleep library to run this script a little slower:
npm install sleep
First, create your bootstrap connection code as follows:
note: You'll need to add your Kinesis stream endpoint.
Step 5
Bash
Bash

if (x<100) {
x++;
normalMsg.seq = x;
// on our 10th message we'll pass the threshold just once and see if our
if (x==10) normalMsg.temperature = 110;
}, 500);
console.log("Sent regular message " + x + " at temp: " + normalMsg.tempe
//sleep.sleep(1); // seconds
}
else {
console.log('Done!');
}
}
});
The above is pretty simple. We're ﬁrst checking our streaming state and after that we're using the callback
functionailty to trigger a slower loop to keep sending messages to AWS IoT.
Once the critical temp is sent (see the line setting our high temp on the 10th message) we see the shadow
change to turn on our Kinesis stream.
Let's now add the stream() function which just blasts data to Kinesis:

device.on('message', function(topic, message) {
console.log('message', topic, message.toString());
if (topic == "$aws/things/thingtest/shadow/update/delta") {
var shadow = JSON.parse(message);
streamState = shadow.state.streamState;
// report back our state change
msg.state.reported.streamState = streamState;
if (streamState == 'on') {
//startStream();
stream();
}, 2500);
}
}
// Start our main loop
if (topic == '$aws/things/thingtest/shadow/update/accepted') {
// Let's loop slowly with regular messages
normalMsg.seq = x;
console.log("Sent regular message " + x + " at temp: " + normalMsg.temperature )
sleep.sleep(1); // seconds
}
if (topic == 'iotCritical') {
if (x<100) {
x++;
normalMsg.seq = x;
// on our 25th message we'll pass the threshold just once and see if our
if (x==10) normalMsg.temperature = 110;
}, 500);
console.log("Sent regular message " + x + " at temp: " + normalMsg.tempe
//sleep.sleep(1); // seconds
}
else {
//process.exit();
console.log('Done!');

}
}
});
var stream = function() {
console.log('Starting Kinesis Stream ...');
for(t=1;t<50;t++) {
// Let's publish our telemtry data
kinesisMsg.id = t;
kinesisMsg.x = t;
kinesisMsg.y = Math.random() * (5000 - 1) + 1;
kinesisMsg.z = kinesisMsg.y/t;
kinesis.putRecord({
StreamName : "iotshard",
Data : JSON.stringify(kinesisMsg),
PartitionKey : "id"
}, function(err, data) {
if (err) {
console.log(err, err.stack); // an error occurred
} else {
console.log("Sent stream message to Kinesis: " + JSON.string
}
});
}
}
device.on('connect', function() {
console.log('Connected!');
device.subscribe('$aws/things/thingtest/shadow/update/delta');
device.subscribe('$aws/things/thingtest/shadow/update/accepted');
device.subscribe('iotCritical');
console.log("Set our thing shadow.");
console.log('Waiting ...');
}, 2500);
});
You can run this now and you see the switch between AWS IoT and Kinesis.
node iotCritical.js
Bash

AWS IoT 핸즈온 워크샵 - 실습 6. 긴급 데이터를 Kinesis Streams으로 보내기 (김무현 솔루션즈 아키텍트)

AWS IoT 핸즈온 워크샵 - 실습 6. 긴급 데이터를 Kinesis Streams으로 보내기 (김무현 솔루션즈 아키텍트)

More Related Content

What's hot (6)

Viewers also liked (20)

Similar to AWS IoT 핸즈온 워크샵 - 실습 6. 긴급 데이터를 Kinesis Streams으로 보내기 (김무현 솔루션즈 아키텍트) (18)

More from Amazon Web Services Korea (20)

Recently uploaded (20)

AWS IoT 핸즈온 워크샵 - 실습 6. 긴급 데이터를 Kinesis Streams으로 보내기 (김무현 솔루션즈 아키텍트)