IoT Innovation Lab Berlin @relayr - Kay Lerch on Getting basics right for your IoT business (2016-05-04)
- 1. www.immobilienscout24.de Get the basics right for your IoT business IoT Innovation Lab @ Relayr – 2016-05-04 – Speaker: @KayLerch
- 2. Agenda 1. Know the tech stack 2. AWS can manage things for you 3. AWS is not the only truth 4. That’s Apache NiFi 5. AWS IoT basic concepts (Recap) 6. AWS IoT & Apache NiFi 7. Focus on business value 8. Focus on innovation Seite 2 Get the basics right for your IoT business | Kay Lerch
- 3. Know the tech stack before you do business Seite 3 Get the basics right for your IoT business | Kay Lerch
- 4. A reference architecture for IoT Seite 4 Source: WSO2 (IoT Whitepaper) Value Value Value Value Effort Effort Effort Challenges: • Set the basics right (EFFORT) • Integrate (VALUE) in your current business / IT landscape • Be innovative! Solution: Minimize (EFFORT) • Rely on managed services and tools Maximize (VALUE) • Have the basics right • rely on established tools • Think over your integration strategy Drive innovation • Think beyond borders • disrupt business with combining established things in a totally new fashion Get the basics right for your IoT business | Kay Lerch
- 5. AWS can manage those basics with a managed IoT solution Seite 5 Get the basics right for your IoT business | Kay Lerch
- 6. What AWS (IoT) provides … Seite 6 Device Registry TLS Certificates with associated IAM-like policies Communication over HTTP and MQTT Message broker with topics State persistence with shadows Event processing and forwarding with IoT Rules and Rule actions Device SDKs in Node.JS, C and for Arduino Yún Get the basics right for your IoT business | Kay Lerch
- 7. What AWS (IoT) provides … Seite 7 Event processing and analytics with Kinesis Eventstreams, ML, EMR and Lambda Data / event stores like S3 or Dynamo DB API Gateway for public endpoint management Have your IoT related key metrics in CloudWatch EC2 as a ground for your (web) applications communicating with devices Get the basics right for your IoT business | Kay Lerch
- 8. AWS IoT core tents Seite 8 Read this: “Core tenets of IoT” by AWS (as of April 2016) Get the basics right for your IoT business | Kay Lerch
- 9. AWS is not the only truth You surely have your lovely tools outside AWS Seite 9 Get the basics right for your IoT business | Kay Lerch
- 10. A brief overview of data processing and analysis That‘s quite a lot of tooling and technology … Seite 10 Hidden complexities Data Producers Data Consumers Event Broker (Big) data stores Data analysis challenges (Realtime) Data Processing & Analytics Integration challenges Ingestionchallenges Data security challenges Get the basics right for your IoT business | Kay Lerch
- 11. A brief overview of data processing and analysis That‘s quite a lot of tooling and technology … Seite 11 Hidden complexities Data Producers Data Consumers Event Broker (Big) data stores Data analysis challenges (Realtime) Data Processing & Analytics Integration challenges Ingestionchallenges Data security challenges Get the basics right for your IoT business | Kay Lerch If you want … a (realtime) big picture of your dataflows an option to overlook lineage of each data element have the flexibility to change things on the fly prioritize data overcome challenges of integrating the variety of technologies with one overarching solution enforce security and compliance along dataflows rely on extensibility driven by OS community satisfy all those needs and keep your tools get rid of only those tools focused on moving data without making concessions to overall performance … then you might love:
- 12. Just to let you know … Seite 12 Source: WSO2 (IoT Whitepaper) Value Value Value Value Effort Effort Effort We are here And a bit of here Get the basics right for your IoT business | Kay Lerch
- 13. That’s Apache NiFi Seite 13 Get the basics right for your IoT business | Kay Lerch
- 14. Seite 14 That‘s Apache NiFi in one page Apache nifi supports powerful and scalable directed graphs of data routing, transformation, and system mediation logic. Web-based user interface Seamless experience between design, control, feedback, and monitoring Highly configurable Loss tolerant vs guaranteed delivery, Low latency vs high throughput, Dynamic prioritization, Flow can be modified at runtime, Back pressure Data Provenance Track dataflow from beginning to end Designed for extension Build your own processors and more, Enables rapid development and effective testing Secure SSL, SSH, HTTPS, encrypted content, etc..., Pluggable role-based authentication/authorization Source: https://nifi.apache.org/ Get the basics right for your IoT business | Kay Lerch
- 15. Seite 15 That‘s Apache NiFi in real and feel Go to NiFi’s interface and understand: Processors Templates Concept of back pressure Concept of data prioritization Provenence Graph Get the basics right for your IoT business | Kay Lerch
- 16. NiFi Cluster NiFi Cluster Manager (NCM) JVM Node (Primary) NiFi Clustered Architecture JVM Webserver Provenance Repository Content Repository Flowfile Repository REST-APIAdminUI Webserver REST-APIAdmin UI Flow Controller Cluster Manager Processor 1 Processor 2 Isolated Processor Controller Service 1 Controller Service 2 Controller Service n Heartbeat Leader election Report change Embedded Apache Zookeeper Node (Slave) JVM Webserver Provenance Repository Content Repository Flowfile Repository REST-APIAdminUI Flow Controller Processor 1 Processor 2 Isolated Processor Controller Service 1 Controller Service 2 Controller Service n Heartbeat Report change Sync State Sync State
- 17. AWS IoT Basics (Recap) Seite 17 Get the basics right for your IoT business | Kay Lerch
- 18. Seite 18 AWS IoT The Shadow AWS IoT Thing Thing Shadow Rule Reports State Mirrors State in Shadow Gets reported state or sets desired state Propagates desired state Receives desired state Fulfills desiredstate Subscribes to particular messages AWS Services Some AWS Resource Routes message TLS 1.2 TLS 1.2 Policy Get the basics right for your IoT business | Kay Lerch
- 19. AWS IoT MQTT topics AWS IoT Thing Shadow get get/ accepted get/ rejected Request state Get shadow state Get error update update/ accepted update/ rejected update/ delta Update state Confirmation Get error Changed state 1 2 1 2 3 Thing topics name pattern: $aws/things/thing_name/... Get the basics right for your IoT business | Kay Lerch
- 20. Apache NiFi & AWS IoT New processors Seite 20 Get the basics right for your IoT business | Kay Lerch
- 21. Seite 21 Apache NiFi & AWS IoT GetIOTMqtt – a MQTT client AWS IoT Thing Shadow update Update state Establish Connection Subscribe Receive state 1 2 3 Flow file Get the basics right for your IoT business | Kay Lerch
- 22. Seite 22 Apache NiFi & AWS IoT GetIOTMqtt – Configuration Get the basics right for your IoT business | Kay Lerch
- 23. Seite 23 Apache NiFi & AWS IoT GetIOTMqtt – Live demo Get the basics right for your IoT business | Kay Lerch
- 24. Seite 24 Apache NiFi & AWS IoT GetIOTMqtt – Live demo Get the basics right for your IoT business | Kay Lerch
- 25. Seite 25 Apache NiFi & AWS IoT PutIOTMqtt – instruct a „thing“ (but bypass the shadow) AWS IoT Thing Shadow update / delta Update state Establish Connection Publish state 1 2 Flow file Flow file Get the basics right for your IoT business | Kay Lerch
- 26. Seite 26 Apache NiFi & AWS IoT PutIOTMqtt – Configuration Get the basics right for your IoT business | Kay Lerch
- 27. Seite 27 Apache NiFi & AWS IoT PutIOTMqtt – Live demo Get the basics right for your IoT business | Kay Lerch
- 28. Seite 28 Apache NiFi & AWS IoT GetIOTShadow – constantly check last reported state AWS IoT Thing Shadow update Report state Request Shadow Flow file Flow file Get the basics right for your IoT business | Kay Lerch
- 29. Seite 29 Apache NiFi & AWS IoT GetIOTShadow – Configuration Get the basics right for your IoT business | Kay Lerch
- 30. Seite 30 Apache NiFi & AWS IoT PutIOTShadow – instruct a „thing“ over its shadow AWS IoT Thing Shadow update / delta Desire state Update Shadow Flow file Flow file Get the basics right for your IoT business | Kay Lerch
- 31. Seite 31 Apache NiFi & AWS IoT PutIOTShadow – Configuration Get the basics right for your IoT business | Kay Lerch
- 32. We got the basics Let’s focus on business value Seite 32 Get the basics right for your IoT business | Kay Lerch
- 33. Homework almost done Seite 33 Source: WSO2 (IoT Whitepaper) Value Value Value Value Effort Effort Effort Challenges: • Set the basics right (EFFORT) • Integrate (VALUE) in your current business • Be innovative! Solution: Minimize (EFFORT) Rely on managed services and tools Maximize (VALUE) Have the basics right rely on established tools • Think over your integration strategy Drive innovation • Think beyond borders • disrupt business with combining established things in a totally new fashion Get the basics right for your IoT business | Kay Lerch
- 34. Agile integration strategy with AWS Simple Workflows at ImmobilienScout Legacy / 3rd Party AWS Simple Workflows (SWF) AWSSNS AWS Lambda AWSS3 AWSEC2 AWSECS OnPremise Broker Start Workflow Put File Bucket Topic Publish Message Start Workflow Activity Task Func tion Func tion Trusted Applications / Mobile Apps AWS Flow Framework Start Workflow Event Trigger Send Data Invoke Service Decider Apps AWS Cloudwatch AWS CloudTrail Logs Metrics Alarms Log Measure Activity Task Activity Task Decision Task Decision Task Workflow History AWS Flow Framework AWS*** Any Activity Task Decision Task Start Workflow Trigger Trigger Notify API Call Discover Decision Task Service Apps AWS Flow Framework Activity Task AWSIAM Policies Manage Permissions Func tion Lambda Task API Call IoT Device AWSIoT Topic Publish Message Func tion Start Workflow Rule Dynamo Start Workflow Put File Table Func tion Trigger Partner AWSAPI Gateway Start Workflow Invoke Func tion Route Service Lambda Task Decision Task
- 35. Hybrid integration platform aiming for Self-Service integration
- 36. We got the basics Let’s focus on innovation Seite 36 Get the basics right for your IoT business | Kay Lerch
- 37. Homework done! Seite 37 Source: WSO2 (IoT Whitepaper) Value Value Value Value Effort Effort Effort Challenges: • Set the basics right (EFFORT) • Integrate (VALUE) in your current business • Be innovative! Solution: Minimize (EFFORT) Rely on managed services and tools Maximize (VALUE) Have the basics right rely on established tools Think over your integration strategy Drive innovation • Think beyond borders • disrupt business with combining established things in a totally new fashion Get the basics right for your IoT business | Kay Lerch
- 38. Gain real business value Seite 38 Source: WSO2 (IoT Whitepaper) Value Value Value Value Effort Effort Effort 1) Combine values of different worlds to find a breathtaking use case 2) Rely on your basics 3) Build a product around your solution 4) Change the world Value Value+ Effort Effort Effort+ + + = + Adopt new things… Get the basics right for your IoT business | Kay Lerch
- 39. Gain value Demo (Smart home) Seite 39 + + Get the basics right for your IoT business | Kay Lerch Pick some fancy frontend (VALUE) Pick some fancy “backend” (VALUE) Rely on your basics (EFFORT) = Have Smart Home
- 40. Gain value Demo (VoiceOps) Seite 40 + + Get the basics right for your IoT business | Kay Lerch Pick some fancy frontend (VALUE) Pick some fancy “backend” (VALUE) Rely on your basics (EFFORT) = Have VoiceOps
- 41. Once you got the basics right, it’s up to you to … Seite 41 Get the basics right for your IoT business | Kay Lerch
- 42. www.immobilienscout24.de Thanks for you attention. Any questions? Contact: Immobilien Scout GmbH Andreasstraße 10 10243 Berlin Kay Lerch Fon +49 30 24 301-1149 kay.lerch@immobilienscout24.de
- 43. Backup Seite 43 Get the basics right for your IoT business | Kay Lerch
- 44. T-Cobbler I need some “thing” …
- 45. Input & Output units on your breadboard I need some “thing” …
- 46. Control GPIO-Pins with sending or receiving electric impulses Control an output Control an input
- 47. Communicating with AWS IoT over MQTT topics
- 48. Alexa SkillAWS Lambda AWS API Gateway Lambda Post Message to Slack Lambda Update Shadow Rpi 2 AWS IoT Thing Shadow Python ScriptGPIO Rule Invoke MQTT Publish MQTT Receive Update Shadow on state change Impulse Sequence SVC Outgoing Webhook Post Message over Slack Web API 433 Mhz Signal C++ D/A Converter Post Message if no state changes
- 49. AWS Lambda AWS EC2 Lambda Start / Stop EC2- Instance Rpi 2 AWS IoT Thing Shadow Python Script GPIO Rule Invoke MQTT Publish Impulse Sequence Post Message over Slack Web API 433 Mhz Signal C++ A/D Converter Start / Stop over AWS SDK AWS CW Lambda React on EC2 state change Rule Update Shadow MQTT Receive Impulse On state change
- 50. Rpi 2 AWS IoT Thing Shadow Python Script GPIO Rule MQTT Publish Impulse Sequence Post Message over Slack Web API AWS CW Alarm On Humidity > 30% Metric Metric Alexa Skill Read out temperature and humidity