![Using CoRE in Real Applications
58
• Resources need meaningful naming (rt=)
• A resource needs an interface (if=)
• Use WADL for this
• A payload needs a format (EXI, JSON, etc.)
• Deployment or industry specific today
• oBIX, SensorML, EEML, sMAP, etc.
• SenML is a promising format [draft-jennings-senml]
• What can we make universal? What should be market specific? How
do we enable innovation?](https://image.slidesharecdn.com/iot-application-layer-protocols-v1-200125143512-241015165251-feb60aaf/85/iot-application-layer-protocols-v1-200125143512-pptx-58-320.jpg)
iot-application-layer-protocols-v1-200125143512.pptx
- 1. Application Layer Protocols for the IoT Damien Magoni University of Bordeaux 2017/10/18 Version 1
- 2. Attribution 2 • The material contained inside is intended for teaching. • This document is licensed under the CC BY-NC-SA license. • Relevant sources are listed on the following References slide. • All figures and text borrowed from these sources retain the rights of their respective owners.
- 3. References 3 • https://www.oasis-open.org/standards#mqttv3.1.1 • http://mqtt.org/ • https://github.com/mqtt/mqtt.github.io/wiki • http://www.cs.wustl.edu/~jain/cse570-15/m_14mqt.htm • http://mqtt.org/new/wp-content/uploads/2009/06/MQTT- SN_spec_v1.2.pdf • https://www.slideshare.net/zdshelby/coap-tutorial • https://tools.ietf.org/html/rfc7252 • https://www.slideshare.net/zdshelby/oma-lightweightm2- mtutorial
- 4. Acknowledgments 4 • Special thanks to Zach Shelby and Raj Jain for their high-quality teaching material that I have extensively reproduced here.
- 5. Table of Contents 5 1. IoT and Web of Things networking 2. Message Queuing Telemetry Transport 3. Constrained Application Protocol 4. Implementations on Raspberry Pi 5. Programming with MQTT/CoAP APIs on Raspbian
- 7. Networking on the Edge 7 • Low power, low speed, lossy wireless networks • Huge number of low capability end devices • Automatically generated (big) data • Terse, purposeful, uncritical
- 8. Current Human-oriented Protocol Stack 8 • Communications are mostly one-to-one/unicast and sender-oriented • M2M comms do not need protocols for human comms • High overhead features: smooth streaming, character echoing, presentation format, etc. • Low-end devices can not store/use a full network protocol stack • Functionnality needs hardware which costs money
- 9. IoT Paradigms 9 • Wireless networks to connect them all • Cheap and adaptive infrastructures compared to wired networks • Many-to-few/few-to-many/many-to-many communications • Group-oriented/multicast communications • Can cope with huge number of devices • Reliability through redundancy • Losses and errors are frequent and unavoidable • Large number of unreliable, uncritical messages • Simple and autonomous devices
- 10. Zones of Interest 10 • In nature, zones of interest are formed by affinities • Common medium used by all • Individuals act upon specific signals among countless other signals • Interested observing individuals can select a zone of interest • Analyze/send data only to/from this zone • Receiver-oriented • Wireless channels can broadcast all messages to neighbors • Neighbors process messages if they wish
- 11. Publish/Subscribe Paradigm 11 • Devices publish information tagged with category (i.e. zone) • They broadcast it to everyone • Devices subscribe to some categories • They act only on what they are interest in • No tight coupling between publishers and subscribers • Unmanaged loose coupling • Signals/messages are simple and small • This communication paradigm scales
- 12. Publish/Subscribe Topologies 12 • With intermediary • Publishers post messages to an intermediary message broker • Subscribers register subscriptions with that broker • The broker performs message filtering and “store and forward” routing from publishers to subscribers • The broker may prioritize messages in a queue before routing • Without intermediary • Each publisher and subscriber shares meta-data about each other via multicast • Publishers and subscribers cache this information locally and route messages based on the discovery of each other interests
- 13. Functional Levels (IoT Hierarchy Revisited) 13 • End devices: mostly publishers • Meshed to one another and connected to propagators • Generate data (sensors) or act on command (actuators) • Propagator nodes: brokers • Connected to the Internet • Process and aggregate data from end devices to integrators • Integrator nodes: mostly subscribers • Connected to the Internet • Analyse data, control the system, and interface with humans
- 14. Web of Things 14
- 16. Web Naming 16
- 18. HTTP Request 18
- 19. Web Paradigms 19
- 20. REST Request 20
- 22. What is MQTT 22 • ISO standard publish-subscribe-based lightweight messaging protocol • Telemetry = tele-metering = remote measurements • However, no message queuing in MQTT • Lightweight = low network bandwidth and small code footprint • For use on top of the TCP/IP protocol stack • The publish-subscribe messaging pattern requires a message broker • The broker is responsible for distributing messages to interested clients based on the topic of a message • Data agnostic, adapt to any content formats
- 23. Publish / Subscribe • Clients can subscribe to a topic or a set of related topics • Clients can also publish to one or more topics • Publish/Subscribe via message exchanges • Topics organized as trees using / character • /# matches all sublevels • /+ matches only one sublevel 23
- 24. Quality of Service Levels 24 • Three levels • 0 = At most once (best effort, no Ack) • 1 = At least once (Acked, retransmitted if Ack not received) • 2 = Exactly once • Control Messages Sequence: Publish → Pubrec → Pubrel → Pubcomp • Server keeps/retains messages even after sending it to all subscribers • New subscribers get the retained messages
- 25. Control Messages 25 • Indicate the desired action to be performed on the identified resource/server • Publish messages can be to/from Client/Server • Often, the resource corresponds to a file or the output of an executable residing on the server • CONNECT/CONNACK • PUBLISH/PUBACK (QoS 1) • PUBREC/PUBREL/PUBCOMP (QoS 2) • SUBSCRIBE/SUBACK • UNSUBSCRIBE/UNSUBACK • PINGREQ/PINGRESP • DISCONNECT
- 26. Sessions and Connections 26 • Clean or continuous sessions with durable connections • At connection set up, if Clean Session flag → all subscriptions are removed on disconnect • Otherwise subscriptions remain in effect after disconnection → subsequent messages with high QoS are stored for delivery after reconnection • Wills • at connection set up, a client can set a Will flag to inform that it has will message that should be published if unexpected disconnection → alarm if the client looses connection • Periodic keep-alive messages → if a client is still alive
- 27. Example • Sensors publish • Brokers can be bridged 27
- 29. MQTT for Sensor Networks (MQTT-SN) 29 • Variation of the main protocol aimed at embedded devices on non- TCP/IP networks such as ZigBee • Zigbee is a low-power, low data rate, close proximity wireless ad hoc PAN • Optimized for implementation on low-cost, battery-operated devices with limited processing and storage resources • Enforce short messages (<<128B) • Support for sleeping clients, CONNECT message split in 3, topic names replaced by 2B topic ids, predefined topic ids with no registration, server/gateway network @ discovery procedure
- 30. MQTT-SN Architecture The forwarder encapsulates/ decapsulates the MQTT-SN frames it receives from the client/the GW and forwards them unchanged 30
- 32. Client State Transition Diagram 32
- 33. Alternative Protocols 33 • Constrained Application Protocol (CoAP) • Advanced Message Queuing Protocol (AMQP) • Streaming Text Oriented Messaging Protocol (STOMP) • Extensible Messaging and Presence Protocol (XMPP) • Web Application Messaging Protocol (WAMP) • Object linking and embedding for Process Control - Unified Architecture (OPC UA) • M2M communication protocol for industrial automation
- 37. What is CoAP 37 • CoAP is • A very efficient RESTful protocol • Ideal for constrained devices and networks • Specialized for M2M applications • Easy to proxy to/from HTTP • CoAP is not • A general replacement for HTTP • HTTP compression • Restricted to isolated “automation” networks
- 38. Features 38 • Embedded web transfer protocol (coap://) • Asynchronous transaction model • UDP binding with reliability and multicast support • GET, POST, PUT, DELETE methods • URI support • Small, simple 4 byte header • DTLS based PSK, RPK and Certificate security • Subset of MIME types and HTTP response codes • Built-in discovery • Optional observation and block transfer
- 39. Transaction Model • Transport • CoAP currently defines UDP binding with DTLS security • CoAP over SMS or TCP possible • Base Messaging • Simple message exchange between endpoints • Confirmable or Non-Confirmable Message • Acknowledgement or Reset Message • REST Semantics • REST Request/Response piggybacked on CoAP Messages • Method, Response Code and Options (URI, content-type, etc.) 39
- 41. Option Format 41
- 43. Request 43
- 44. Dealing with Packet Loss 44
- 47. Caching 47 • CoAP includes a simple caching model • Cacheability determined by response code • An option number mask determines if it is a cache key • Freshness model • Max-Age option indicates cache lifetime • Validation model • Validity checked using the Etag Option • A proxy often supports caching • Usually on behalf of a constrained node, • a sleeping node, • or to reduce network load
- 49. Observation 49
- 51. Web Linking 51 • Web Linking formalizes links with defined relations, typed links • HTML and Atom have allow links • RFC5988 defines a framework for Web Linking • Combines and expands the Atom and HTML relation types • Defines a unified typed link concept • A link can be serialized in any number of formats • RFC5988 revives the HTTP Link Header and defines its format • Atom and HTML are equivalent serializations
- 52. Typed Link 52 • A type link consists of • Context URI – What the link is from • Relation Type – Indicates the semantics of the link • Target URI – What the link is to • Attributes – Key value pairs describing the link or its target • Relations include e.g. copyright, author, chapter, service, etc. • Attributes include e.g. language, media type, title, etc. • Example in HTTP Link Header format Link: <http://example.com/TheBook/chapter2>; rel="previous"; title="previous chapter"
- 53. Resource Discovery 53 • Service Discovery • Find the IP address, port and protocol of the service • Usually performed by DNS-SD when DNS is available • Resource Discovery • Finding URIs • Performed using Web Linking or some REST interface • CoRE Link Format is designed to enable resource discovery
- 54. CoRE Link Format 54 • RFC6690 is aimed at Resource Discovery for M2M • Defines a link serialization suitable for M2M • Defines a well-known resource where links are stored • Enables query string parameters for filtered GETs • Can be used with unicast or multicast (CoAP) • Resource Discovery with RFC6690 • Discovering the links hosted by CoAP (or HTTP) servers • GET /.well-known/core?optional_query_string • Returns a link-header style format • URL, relation, type, interface, content-type etc.
- 56. Resource Directory • CoRE Link Format only defines • The link format • Peer-to-peer discovery • A directory approach is also useful • Supports sleeping nodes • No multicast traffic, longer battery life • Remote lookup, hierarchical and federated distribution • The CoRE Link Format can be used to build Resource Directories • Nodes POST (register) their link-format to an RD • Nodes PUT (refresh) to the RD periodically • Nodes may DELETE (remove) their RD entry • Nodes may GET (lookup) the RD or resource of other nodes 56
- 58. Using CoRE in Real Applications 58 • Resources need meaningful naming (rt=) • A resource needs an interface (if=) • Use WADL for this • A payload needs a format (EXI, JSON, etc.) • Deployment or industry specific today • oBIX, SensorML, EEML, sMAP, etc. • SenML is a promising format [draft-jennings-senml] • What can we make universal? What should be market specific? How do we enable innovation?
- 59. CoRE Link Format Semantics 59 • RFC6690 = Simple semantics for machines • IANA registry for rt= and if= parameters • Resource Type (rt=) • What is this resource and what is it for? • e.g. Device Model could be rt=“ipso.dev.mdl” • Interface Description (if=) • How do I access this resource? • e.g. Sensor resource accessible with GET if=“core.s” • Content Type (ct=) • What is the data format of the resource payloads? • e.g. text/plain (0)
- 60. CoRE Interfaces A paradigm for REST profiles made up of function sets 60
- 61. Open Mobile Alliance (OMA) 61 • Standards body which develops open standards for the mobile phone industry • Only standardizes applicative protocols • Specifications are meant to work with any cellular network technologies • Example specifications • MMS multimedia messaging • Instant Messaging and Presence Service • Lightweight M2M
- 62. Lightweight M2M 62 • Provide device management functionality over sensor or cellular networks • 6LoWPAN, WiFi, ZigBee, any IP based constrained devices/networks • Transfer service data from the network to devices • Can be extended to meet the requirements of any application • Supported in OneM2M and integrated with ETSI M2M
- 63. LWM2M 1.0 Features 63 • Simple object based resource model • Global registry and public lookup of all objects • Standard device management objects already defined by OMA • Resource operations of creation/retrieval/update/deletion/configuration of attribute • Resource observation/notification • TLV/JSON/Plain Text/Opaque data format support • UDP and SMS transport layer support • DTLS based security • Pre-shared and Public Key modes, Provisioning and Bootstrapping • Queue mode for NAT/Firewall environment • Multiple server support • Basic M2M functionalities • Access Control, Device, Connectivity, Firmware Update, Location, Connectivity Statistics
- 65. IoT Standardization 65 • IETF • 6LoWPAN Working Group (IPv6 anywhere) • ROLL (Routing Over Low-power Lossy Networks) WG • CoRE WG (REST for IoT, CoAP, Resource Directory etc.) • TLS WG (DTLS) • OMA • Lightweight M2M Enabler Standard (CoAP/DTLS based) • Device Management 2.0 Enabler Standard (HTTP/TLS based) • ETSI / OneM2M • Ongoing work on M2M system standardization (CoAP, HTTP binding) • W3C • Efficient XML Interchange (EXI) standardization • ZigBee IP • An open-standard 6LoWPAN stack for e.g. Smart Energy 2.0
- 66. Implementations of Protocols on the Raspberry Pi Part 4
- 67. Eclipse for IoT 67 • https://iot.eclipse.org/ • Plenty of open-source protocol implementations • https://iot.eclipse.org/standards/
- 68. MQTT 68 • Eclipse Mosquitto server (in C) • https://projects.eclipse.org/projects/technology.mosquitto • https://mosquitto.org/ • https://github.com/eclipse/mosquitto • Eclipse Paho client • Open-source client implementations of MQTT/MQTT-SN protocols • http://www.eclipse.org/paho/ • MQTT Paho client library (in Python) • https://pypi.python.org/pypi/paho-mqtt • pip install paho-mqtt
- 69. CoAP 69 • Implementations • http://coap.technology/impls.html • libcoap (client and server in C) • https://libcoap.net/ • OpenWSN CoAP Python library • https://github.com/openwsn-berkeley/coap • txThings (in Python) • https://github.com/mwasilak/txThings • CoAPthon • https://github.com/Tanganelli/CoAPthon • Copper (Cu) CoAP user-agent for Firefox • https://addons.mozilla.org/en-US/firefox/addon/copper-270430/
- 70. Programming with MQTT and CoAP APIs on Raspbian Part 5
- 71. MQTT Client Subscription Example 71 import paho.mqtt.client as mqtt # The callback for when the client receives a CONNACK response from the server. def on_connect(client, userdata, rc): print("Connected with result code "+str(rc)) # Subscribing in on_connect() means that if we lose the connection and # reconnect then subscriptions will be renewed. client.subscribe("$SYS/#") # The callback for when a PUBLISH message is received from the server. def on_message(client, userdata, msg): print(msg.topic+" "+str(msg.payload)) client = mqtt.Client() client.on_connect = on_connect client.on_message = on_message client.connect("iot.eclipse.org", 1883, 60) # Blocking call that processes network traffic, ..dispatches callbacks and handles reconnecting. # Other loop*() functions are available that give .. a threaded interface and a manual interface. client.loop_forever()
- 72. Other MQTT Client Examples 72 • https://github.com/eclipse/paho.mqtt.python/tree/master/examples
- 73. CoAP GET Request Client Example 73 • https://github.com/mwasilak/txThings/blob/master/examples/clientGET.py