Gustavo Zastrow - Introduction to AWS IoT Core and MQTT

Editor's Notes

• #3: * The Internet of Things refers to the network of physical objects connected to the Internet, and the communication between these objects and other Internet-enabled devices and systems. * Not all IoT devices have an IP address, as some needs a gateway to connect to the Internet. In a example a Fitbit activity tracker that uses Bluetooth to connect to a smartphone App to send data to the cloud. * Simpler definition: Devices connected to the Internet, that used to be standalone. Example: On average a car has 50 microcontrollers, which are not connected to the Internet. But that will soon change with autonomous vehicles and 5G
• #4: How they can interact with devices when they are offline? Using a “Device Shadow”
• #5: * Devices: Also known as “Things” in AWS vocabulary * Message broker: It’s a MQTT server responsible for receiving and filtering messages, determining who is subscribed to each message, and sending the message to these subscribed clients. * Device Shadow: Is a persistent, virtual representation of your devices or Things. Basically, it’s a JSON State Document. Its main advantage of is that you can interact with it, regardless of whether the thing is connected to the Internet or not. IoT applications, like control dashboards can be developed for iOS & Android platforms (i.e. remote control to open/close garage door). Another idea is to use the smartphone sensors and publish their metrics to IoT Core. * Security and identity: Provided by the use of certificates and policies. * Rules engine: To “act” upon the data published by the Things
• #6: * AWS IoT Core makes it “relatively” easy to use AWS services like Amazon Kinesis, S3, Machine Learning, Lambda, DynamoDB, CloudWatch, CloudTrail and Elasticsearch to build IoT applications that gather, process, analyze and act on data generated by connected devices. * Examples of scalability: garden monitoring, bike sharing, etc can easily scale from a dozen to a few thousands without having to worry about infrastructure. * IoT Core is a good candidate for Lambda usage.
• #7: * It’s a good idea to develop a standardized naming convention to name devices, groups and topics in a consistent way * I will cover more about Endpoint and Topics in the next slides * Groups example: Smoke detectors group. Living room group. * Device shadow: is a replica or model of the IoT device
• #8: * Before learning more about IoT Core we need to review basic concepts about MQTT, the most popular communication protocol used by IoT devices. * Although I'm only going to cover the basics of MQTT, other protocols like WebSockets and HTTP 1.1 are also supported by AWS IoT Core.
• #9: * MQTT original application was to link sensors on oil pipelines with satellites * IBM MQ is a family of message-oriented middleware products that IBM launched in 1993. It was originally called MQSeries, and was renamed WebSphere MQ in 2002
• #10: Lightweight protocol allows it to be implemented on both heavily constrained device hardware and high latency and limited bandwidth networks
• #11: * The act of sending the message is referred to as “publishing”. The act of registering to receive messages for a topic filter is referred to as “subscribing”* Although the client can subscribe to “any topic in the broker”, AWS policies are used to allow/deny subscription to topics
• #12: * Default TCP ports a MQTT broker listens: 1883 and 8883 (for MQTT over SSL) * There are no communications between clients, always through the broker* This Publish and Subscribe model is also called “Pub/Sub”* Publish/Subscribe decouples the client that sends a message (the publisher) from the clients that receive the messages (the subscribers)
• #13: * I mentioned broker many times. An MQTT broker is just a server that receives the published messages, filter then by topic and send it to subscribers of the same topic* Another responsibility of the broker is the authentication and authorization of clients* AWS IoT Core only uses port 8883 * Keep in mind that MQTT brokers do not store messages for long, you will need to connect it to a database for long term data storage
• #14: The smart homes “hubs” are actually MQTT brokers
• #15: * QoS 0 - at most once: Best-effort delivery. There is no guarantee of delivery (like TCP). The recipient does not acknowledge receipt of the message and the message is not stored and re-transmitted by the sender. * QoS 1 - at least once: It guarantees that a message is delivered at least one time to the receiver, but it is possible for a message to be sent or delivered multiple times. * QoS 2 - exactly once: It’s the highest level of service in MQTT. This level guarantees that each message is received only once by the intended recipients. It is the safest and slowest quality of service level. Currently not supported by AWS IoT Core
• #17: * Prices are going down, you can build a WiFi connected sensor for less than $20 * Size: We don’t want a desktop PC :-) * How to power the device? Solar, batteries or just an AC outlet? * Connectivity ranges from BLE to satellite. 5G connectivity coming to IoT devices. LORA is popular for agriculture. * For security consider encryption in the device, on transit and at rest. Certificates management. OTA updates. What happens if the device is stolen/tampered? Could it be cloned? * How it will scale as number of sensors increase? * The 3 laws: The law of physics, law of economics, law of the land (legal and geographical restrictions, GDPR, )
• #18: * The Raspberry Pi Zero W uses a 1GHz BCM2835 single-core processor (700MHz in idle mode), and 512MB RAM. Comes with a CSI connector to connect a camera. HAT-compatible 40-pin header.* The Zero W includes 802.11n WiFi and Bluetooth 4.0. * The DHT22 sensor provides temperature and humidity measures. For indoor usage only (it’s not water proof)
• #19: A case is also recommended (not shown in the picture)
• #21: * Message payload should also be defined, i.e ON and OFF for the topic home/basement/restroom/fan/set * How about adding a topic for device health status? to monitor CPU load, memory, uptime, etc * Or a topic for command or control (to do a remote update or reboot) * There is no right or wrong way, but there are some best practices when choosing topic naming conventions * Topics are case sensitive so try using lowercase topic names * For the project I choose the topic home/livingroom/raspi01
• #22: * If you like to assemble IKEA furniture you will love this part :-) * Test and calibration is needed as different sensors will show slightly different values
• #23: * The most popular OS for Raspberry Pi is Raspbian which is a Linux distribution based in Debian.* Use “raspi-config” to connect to a WiFi network, change hostname, enable SSH access, disable GUI and change screen resolution * Before buying a sensor search online for available documentation, tutorials and reviews * Example for the sensor I purchased: http://osoyoo.com/2018/12/raspberry-pi-lesson-measure-humidity-and-temperature-with-dht22/ * Other settings: Lower clock speed to save power (underclocking). Solder pin heads. Add a CPU cooler. Protective case.
• #24: To install Python packages for the DHT sensor: pip install Adafruit_DHT or pip3 install Adafruit_DHT (for Python 3.x)
• #25: * To install the IoT SDK for Python: pip install AWSIoTPythonSDK or pip3 install AWSIoTPythonSDK (for Python 3.x)* There are SDKs for C++, Embedded C, Java, JavaScript, Python... and more to come * Minimum Python versions required: Python 2.7+ or Python 3.3+
• #26: * Although I used the AWS console to interactively setup IoT Core, a more programmatic approach using the AWS CLI is better method for scalable projects. * Starter kits are available from AWS partners like AAEON Technology, AT&T, Digi International, ST Microelectronics, Texas Instruments.
• #27: Wizard like interface to guide you through the steps to setup IoT Core for the new Thing
• #28: SDK platform selection: For our Raspberry Pi we choose Linux and Python
• #29: * Decide on a naming convention for the Things* Recommendation: Keep thing names in lower case
• #30: * Download the connection kit * Check the “Preview policy” link which is very restrictive initially and we will modify later in another slide
• #31: Unzip the connection kit
• #32: * Certificates are used for authentication and to send data to AWS we need to be able to make a secure connection to AWS IoT * I found permission issues running start.sh so I downloaded the CA certificate manually
• #33: Endpoint is the name of the assigned MQTT Broker in AWS
• #34: Policies are used for authorization: What can the IoT devices do? For example connect, publish messages, subscribe to topics
• #35: * This is the section of the poly document showing the policies for Publish and Receive * Added the line with the topic home/livingroom/raspi01* The other policies were already present are created by default to support the examples in the connection package (They can be deleted)* For quick and dirt test you put a policy to accept all topics using iot:*
• #36: Similar policies for Subscribe and Connect
• #37: Screenshot of the basicPubSub.pyexample showing the parameters needed to connect basicPubSub.py is provided with the connection kit when start.sh is run
• #38: * Very simple Python code based in basicPubSub.py but without logging or sensor data validation * Added to CRON so it runs at specific times (like every 5 minutes)
• #39: If connection is successful it will be detected by the broker and show as small dot in the Monitor section
• #40: * Type the topic name and press the subscribe to topic button to check if the messages are being received* As this is a MQTT client you can also use it to publish
• #41: * You should see the values in JSON format being updated every time the device publish then to this topic * Topic name and time stamping are at the top of the message
• #42: Once we confirm the reception of the published messages it’s time to act upon them
• #43: * Choose the name of the rule and create a SQL SELECT statement which allows you to extract data from an incoming MQTT message* The rule is triggered when an MQTT message is received on the topic and matches the query* In this case the overheat rule needs the temperature value to be over 79 degrees to act
• #44: * Rule actions allow you to extract data from MQTT messages and send it to another AWS service* We will choose “Send a message as an SNS push notification” to send a SMS alert and later we will choose “Send message data to CloudWatch” to plot temperature and humidity values.
• #45: * Create an SNS target and role for the action * Next step will be to setup the SNS service and add a phone number for SMS, which is outside the scope of this presentation
• #46: Choose the name for the CloudWatch plot rule and create a SQL statement to select all measures from topic home/livingroom/raspi01
• #47: * Select the name and values to plot temperature * To extract the value type ${temperature}
• #48: Select name and values to plot humidity
• #49: * And finally CloudWatch is able to plot values of temperature and humidity* These metrics which are being published by the IoT device raspi01 to the MQTT topic home/livingroom/raspi01
• #50: * Use ML to predict future trends. Would it be cool if extreme environment conditions could be predicted? SageMaker? * Other device metrics such as available RAM, uptime, battery level, CPU usage, microSD storage space, CPU temperature, etc. can be published in a different topic for device health monitoring.
• #54: Don’t forget the IoT Core Test section in AWS Console is a very basic MQTT client with publish and subscribe capabilities and it’s very useful for troubleshooting
• #55: * Deployment of the device simulator is done using a AWS CloudFormation template * Great solution for simulations of a large pool of connected devices (fleets) and for stress-tests * Too complex for a small project like the one this presentation is based on
• #57: * Each connected device must have a credential to access the message broker or the Device Shadow service* All traffic to and from AWS IoT must be encrypted over Transport Layer Security (TLS)* Device credentials must be kept safe in order to send data securely to the message broker* AWS Cloud security mechanisms protect data as it moves between AWS IoT and other devices or AWS services.