IoT Architecture - are traditional architectures good enough?
11 likes7,136 views
The document discusses the readiness of traditional architectures to manage the Internet of Things (IoT) and highlights the evolving landscape of IoT applications. It emphasizes the need for real-time data processing and the transformation of existing systems to handle large volumes of events and integrate with modern technologies effectively. Key challenges and considerations for IoT architecture are outlined, reinforcing the importance of understanding specific use cases and requirements for successful implementation.
IoT Architecture - are traditional architectures good enough?
- 1. BASEL BERN BRUGG DÜSSELDORF FRANKFURT A.M. FREIBURG I.BR. GENEVA HAMBURG COPENHAGEN LAUSANNE MUNICH STUTTGART VIENNA ZURICH Internet of Things (IoT) Are traditional architectures good enough? Guido Schmutz
- 2. Guido Schmutz Working for Trivadis for more than 18 years Oracle ACE Director for Fusion Middleware and SOA Co-Author of different books Consultant, Trainer, Software Architect for Java, SOA & Big Data / Fast Data Member of Trivadis Architecture Board Technology Manager @ Trivadis More than 25 years of software development experience Contact: guido.schmutz@trivadis.com Blog: http://guidoschmutz.wordpress.com Slideshare: http://www.slideshare.net/gschmutz Twitter: gschmutz
- 3. Our company. Trivadis is a market leader in IT consulting, system integration, solution engineering and the provision of IT services focusing on and and Open Source technologies in Switzerland, Germany, Austria and Denmark. We offer our services in the following strategic business fields: Trivadis Services takes over the interacting operation of your IT systems. O P E R A T I O N
- 4. COPENHAGEN MUNICH LAUSANNE BERN ZURICH BRUGG GENEVA HAMBURG DÜSSELDORF FRANKFURT STUTTGART FREIBURG BASEL VIENNA With over 600 specialists and IT experts in your region. 14 Trivadis branches and more than 600 employees 200 Service Level Agreements Over 4,000 training participants Research and development budget: CHF 5.0 million Financially self-supporting and sustainably profitable Experience from more than 1,900 projects per year at over 800 customers
- 5. Agenda 1. Introduction 2. Are Traditional Solution architectures good enough? 3. IoT Architecture & Components 4. Demo Use Case – Truck Movements 5. Oracle IoT Cloud Service 6. Summary
- 6. Introduction
- 7. Internet of Things Wave Internet of Things (IoT): Enabling communication betweendevices,people& processesto exchangeusefulinformation & knowledgethat create value for humans Term was first proposedby Kevin Ashtonin 1999 The conceptof IoT first became popularat the Auto-ID centerand MIT IoT can also be named M2M
- 8. Internet Of Things – Sensors are/will be everywhere There are more devices tapping into the internet than people on earth How do we prepare our systems/architecture for the future? Source: Cisco Source: The Economist
- 9. Reasons why IoT opportunity is occurring now ? Affordable hardware • Costs of actuators & sensors have been cut in half over last 10 years Smaller, but more powerful hardware • Form factors of hardware have shrunk to millimeter or even nanometer levels Ubiquitous & cheap mobility • Cost for mobile devices, bandwidth and data processing has declined as much as 97% over last 10 years Availability of supporting tools • Big data tools & cloud based infrastructure have become widely available and fairly sophisticated Mass market awareness • IoT has surpassed a critical tipping point • Vision of a connected world has reached such a followership that companies have initiated IoT developments • Commitment is irreversible
- 11. Are Traditional Solution architectures good enough?
- 12. A little story of a “real-life” customer situation Traditional system interact with its clients and does its work Implemented using legacy technologies (i.e. PL/SQL) New requirement: • Offer notification service to notify customer when goods are shipped • Subscription and inform over different channels • Existing technology doesn’t fit delivery Logistic System Oracle Mobile Apps Sensor ship sort 12 Rich (Web) Client Apps DB schedule Logic (PL/SQL) delivery
- 13. A little story of a “real-life” customer situation Events are “owned” by traditional application (as well as the channels they are transported over) Implement notification as a new Java-based application/system But we need the events ! => so let’s integrate delivery Logistic System Oracle Mobile Apps Sensor ship sort 13 Rich (Web) Client Apps DB schedule Notification Logic (PL/SQL) Logic (Java) delivery SMS Email …
- 14. A little story of a “real-life” customer situation integrate in order to get the information! Oracle Service Bus was already there Rule Engine implemented in Java and invoked from OSB message flow Notification system informed via queue Higher Latency introduced (good enough in this case) delivery Logistic System Oracle Oracle Service Bus Mobile Apps Sensor AQship sort 14 Rich (Web) Client Apps DB schedule Filter Notification Logic (PL/SQL) JMS Rule Engine (Java) Logic (Java) delivery shipdelivery delivery true SMS Email …
- 15. A little story of a “real-life” customer situation Treat events as first-class citizens Events belong to the “enterprise” and not an individual system => Catalog of Events similar to Catalog of Services/APIs !! Event (stream) processing can be introduced and by that latency reduced! delivery Logistic System Oracle Oracle Service Bus Mobile Apps Sensor AQship sort 15 Rich (Web) Client Apps DB schedule Filter Notification Logic (PL/SQL) JMS Rule Engine (Java) Logic (Java) delivery shipdelivery delivery true SMS Email …
- 16. Treat Events as Events and make them globally available delivery Logistic System Oracle Oracle Service Bus Mobile Apps Sensor ship sort 16 Rich (Web) Client Apps DB schedule Filter Notification Logic (PL/SQL) JMS Rule Engine (Java) Logic (Java) delivery ship delivery true SMS Email … Event Bus/Hub Stream/Event Processing
- 17. Traditional Data Processing - Challenges • Introduces too much “decision latency” • Responses are delivered “after the fact” • Maximum value of the identified situation is lost • Decision are made on old and stale data • “Data a Rest”
- 18. The New Era: Stream Data Processing / Fast Data • Events are analyzed and processed in real- time as the arrive • Decisions are timely, contextual and based on fresh data • Decision latency is eliminated • “Data in motion”
- 19. IoT Architecture
- 20. Key Challenges for building an IoT application 1. Connect: How to collect data from intelligent devices? • Abstract complexity associated with device connectivity • Standardize integration of devices with enterprise 2. Analyze: How to analyze IoT data? • Reduce noise and detect business event at real-time • Enable historical big-data analysis 3. Integrate: How to integrate IoT data & events with enterprise infrastructure? • Make enterprise processes IoT friendly • Allow enterprise & mobile applications to control devices
- 21. Today) Existing Service-/API-based Architecture 21 Mobile Apps DB Rich (Web) Client Apps DB API Gateway Enterprise Service Bus (ESB) / Data Integration Enterprise Applications WS External Cloud Service Providers BPM and SOA Platform Event Business Logic/Rules Business Intelligence Services WS Event Processes Visualization Analytics DB = one way = request/response REST SOAP REST SOAP SOAP SOAP any SQL SOAP REST API Gateway Service BusOracle Data Integrator SOA Suite BPM Suite Business Activity Monitoring
- 22. IoT 1a) Reuse Service-/API-based Architecture IoT Device 22 Mobile Apps DB Rich (Web) Client Apps DB API Gateway Enterprise Service Bus (ESB) / Data Integration Enterprise Applications WS External Cloud Service Providers BPM and SOA Platform Event Business Logic/Rules Business Intelligence Services WS Event Processes Visualization Analytics DB = one way = request/response SOAP REST REST HTTP JMS SOAP any SQL SOAP REST WebSocket REST SOAP REST SOAP API Gateway Service BusOracle Data Integrator SOA Suite BPM Suite Business Activity Monitoring JMS HTTP
- 23. IoT 1b) Reuse existing Service-/API-based Architecture IoT Device 23 Mobile Apps DB Rich (Web) Client Apps DB API Gateway Enterprise Service Bus (ESB) / Data Integration Enterprise Applications WS External Cloud Service Providers BPM and SOA Platform Event Business Logic/Rules Business Intelligence Services WS Event Processes Visualization Analytics DB WebSocket REST SOAP REST SOAP REST SOAP REST HTTP REST API Gateway Service BusOracle Data Integrator SOA Suite BPM Suite JMS Business Activity Monitoring REST HTTP JMS = one way = request/response
- 24. IoT 1c) Reuse existing Service-/API-based Architecture 24 Mobile Apps DB Rich (Web) Client Apps DB Enterprise Service Bus (ESB) / Data Integration API Gateway IoT Device Gateway IoT Smart Device Enterprise Applications WS External Cloud Service Providers BPM and SOA Platform Event Business Logic/Rules Business Intelligence Services WS Event Processes Visualization Analytics DB REST SOAP REST SOAP SOA Suite BPM Suite API Gateway Service BusOracle Data Integrator Business Activity Monitoring REST HTTP JMS MQTT Kafka REST HTTP JMS MQTT Kafka = one way = request/response
- 25. Event Hub - Apache Kafka Distributed publish-subscribe messaging system Designed for processing of real time activity stream data (logs, metrics collections, social media streams, …) Topic Semantic Does not use JMS Initially developed at LinkedIn, now part of Apache Kafka Cluster Consumer Consumer Consumer Producer Producer Producer
- 26. Service Bus as a consumer of Kafka Service Bus 12c Cloud Apps Business Service Cloud Proxy Service Kafka Cloud API Mobile Apps Pipeline Routing Kafka Sensor / IoT Web Apps Business Service REST Business Service WSDL Backend Apps REST Backend Apps WSDL Proxy Service Kafka Pipeline Routing Database DB CDC Stream Processing
- 27. IoT 2) Stream Processing / Analytics in Backend 27 Mobile Apps DB Rich (Web) Client Apps DB (ESB) / Data Integration API Gateway IoT Device Gateway IoT Smart Device Event Hub Stream Processing Infrastructure ESP / CEP DB DB Event Hub Enterprise Applications WS External Cloud Service Providers BPM and SOA Platform Event Business Logic/Rules Business Intelligence Services WS Event Processes Visualization Analytics DB = one way = request/response REST SOAP REST SOAP Business Activity Monitoring SOA Suite BPM Suite API Gateway Service BusOracle Data Integrator Stream Analytics
- 28. Oracle Stream Analytics as consumer of Kafka/MQTT Oracle Stream Analytics Stream Analytics Kafka Kafka Mobile Apps Kafka Sensor / IoT Web Apps Machine Data DB CDC Kafka MQTT
- 29. Distribute runtime portion of IoT Analytics to the edge Some IoT Analytics applications need to be distributed, so that processing can take place in devices, gateways, smart routers, servers at the site where sensor data is generated Three main reasons: • The factory, vehicle, home or other edge location needs to stay in operation, even when the corporate data center or cloud is down or not reachable • Wide-are communication is generally to slow • Transmitting all sensor data to corporate or cloud data center may be impractical or impossible if the volume of data is high or if reliable, high- bandwidth networks are unavailable
- 30. IoT 3) Stream Processing / Analytics at Edge Enterprise Applications WS External Cloud Service Providers BPM and SOA Platform Event Business Logic/Rules Business Intelligence Services WS Event Processes Visualization Analytics DB 30 Mobile Apps DB Rich (Web) Client Apps DB (ESB) / Data Integration API Gateway IoT Device Gateway IoT Smart Device Event Hub Stream Processing Infrastructure ESP / CEP DB DB Event Hub ESP / CEP = one way = request/response DB REST SOAP REST SOAP Stream Analytics Edge Analytics Business Activity Monitoring SOA Suite BPM Suite Service BusOracle Data Integrator API Gateway
- 31. IoT 4) Raw Data Storage and Batch Analytics Enterprise Applications WS External Cloud Service Providers BPM and SOA Platform Event Business Logic/Rules Business Intelligence Services WS Event Processes Visualization Analytics DB 31 Mobile Apps DB Rich (Web) Client Apps DB (ESB) / Data Integration API Gateway = one way = request/response IoT Device IoT Smart Device Event Hub Stream Processing Infrastructure ESP/CEP DB DB Event Hub Big Data Infrastructure HDFS Batch Processing DB REST SOAP REST SOAP Gateway ESP / CEP DB Edge Analytics Stream Analytics Business Activity Monitoring SOA Suite BPM Suite Service BusOracle Data Integrator Big Data Appliance API Gateway
- 32. IoT 4a) Raw Event Replay Feature Enterprise Applications WS External Cloud Service Providers BPM and SOA Platform Event Business Logic/Rules Business Intelligence Services WS Event Processes Visualization Analytics DB 32 Mobile Apps DB Rich (Web) Client Apps DB (ESB) / Data Integration API Gateway = one way = request/response IoT Device IoT Smart Device Event Hub Stream Processing Infrastructure ESP / CEP DB DB Event Hub Big Data Infrastructure HDFS Replay REST SOAP REST SOAP Gateway ESP / CEP DB Edge Analytics Business Activity Monitoring SOA Suite BPM Suite Service BusOracle Data Integrator Stream Analytics Big Data Appliance API Gateway
- 33. IoT 5) Data Mining and Model evaluation Enterprise Applications WS External Cloud Service Providers BPM and SOA Platform Event Business Logic/Rules Business Intelligence Services WS Event Processes Visualization Analytics DB 33 Mobile Apps DB Rich (Web) Client Apps DB (ESB) / Data Integration API Gateway = one way = request/response IoT Device IoT Smart Device Event Hub Stream Processing Infrastructure ESP/CEP DB DB Event Hub Big Data Infrastructure HDFS Batch Processing DB REST SOAP REST SOAP Gateway ESP / CEP DB Edge Analytics Business Activity Monitoring SOA Suite BPM Suite Service BusOracle Data Integrator Stream Analytics Big Data Appliance API Gateway
- 34. IoT Services IoT Logical Reference Architecture IoT Device Sensor Actuator IoT Gateway Storage UIApp Streaming Analytics Enterprise Applications BPM and SOA PlatformStreaming Analytics Storage Endpoint Management Event Hub Service Bus Event Hub Event Hub Service Bus Big Data / BI Storage Services Processes UIApp Storage Bulk Analytics UI Bulk Analytics UI Storage Streaming Analytics Service Bus API REST SOAP HTTP KAFKA MQTT CoAP XMPP DDS AMQP KAFKA WIFI BLE ZigBee WIFI Wired
- 35. IoT Services IoT Logical Reference Architecture – Oracle on premises IoT Device Sensor Actuator IoT Gateway Storage UIApp Streaming Analytics Enterprise Applications BPM and SOA PlatformStreaming Analytics Storage Endpoint Management Event Hub Service Bus Event Hub Event Hub Service Bus Big Data / BI Storage Services Processes UIApp Storage Bulk Analytics UI Bulk Analytics UI Storage Streaming Analytics Service Bus API REST SOAP HTTP KAFKA MQTT CoAP XMPP DDS AMQP KAFKA WIFI BLE ZigBee WIFI Wired Edge Analytics Business Activity Monitoring SOA Suite BPM Suite Service Bus Oracle Data Integrator Stream Analytics Big Data Appliance Stream Analytics Service Bus API Gateway
- 36. IoT Services IoT Device Sensor Actuator IoT Gateway Storage UIApp Streaming Analytics Enterprise Applications BPM and SOA PlatformStreaming Analytics Storage Endpoint Management Event Hub Service Bus Event Hub Event Hub Service Bus Big Data / BI Storage Services Processes UIApp Storage Bulk Analytics UI Bulk Analytics UI Storage Streaming Analytics Service Bus API REST SOAP HTTP KAFKA MQTT CoAP XMPP DDS AMQP KAFKA WIFI BLE ZigBee WIFI Wired IoT Logical Architecture – Open Source
- 37. Demo Use Case – Truck Movements
- 38. Demo Use Case – Truck Movements Truck Data Ingestion Geo-Fencing 2016-06-02 14:39:56.605|98|27|Mark Lochbihler|803014426|Wichita to Little Rock Route 2|Normal|38.65|- 90.21|5187297736652502631 {"timestamp": "2016-06-02 14:39:56.991", "truckId": 99, "driverId": 31, "driverName": "Rommel Garcia", "routeId": 1565885487, "routeName": "Springfield to KC Via Hanibal", "eventType": "Normal", "latitude": 37.16, "longitude": "-94.46", "correlationId": 5187297736652502631} Reckless Driving Detector NEAR ENTER Truck Driver DashboardMovement Movement JSON Reckless Driver
- 39. Oracle IoT Cloud Service
- 40. Oracle IoT Cloud Service within Oracle Cloud Platform Documents Cloud Process Cloud Sites Cloud Social Cloud CONTENT AND COLLABORATION Business Intelligence Cloud Big Data Discovery Cloud Big Data Preparation Cloud Internet of Things Cloud BUSINESS ANALYTICS NoSQL Database Cloud Big Data SQL Cloud Exadata Cloud Big Data Cloud Database Backup Cloud Database Cloud Big Data Preparation Cloud DATA MANAGEMENT Node Cloud Developer Cloud Mobile Cloud Java Cloud Application Builder Cloud APPLICATION DEVELOPMENT Integration Cloud SOA Cloud GoldenGate Cloud Internet of Things Cloud INTEGRATION Application Performance Monitoring Cloud IT Analytics Cloud Log Analytics Cloud ENTERPRISE MANAGEMENT Mobile Cloud MOBILEIntegration Cloud Internet of Things Cloud Sites Cloud Documents Cloud
- 41. Oracle IoT Cloud Service Device Virtualization High Speed Messaging Stream Processing Endpoint Management Event Store IoT Cloud Service Enterprise Connectivity Integration Cloud Service BI & Big Data Cloud Service Oracle Cloud Services Mobile Cloud Service 3rd party apps Industry Vertical Apps Enterprise Apps Cloud or On Premise Manufacturing Transportation Service Mgmt Asset Mgmt Firewall Oracle IoT CS Gateway s/w 3rd party gateway s/w with Oracle IoT Client Library IoT Cloud Service Client Libraries & Gateway Indirectly connected Devices Directly connected device
- 42. Anatomy of an IoT Application in Oracle IoT CS IoT Application Devices Messages Analytics Integrations Configurations
- 43. Real-time Analytics Applications Workflow Devices Data/Alerts Devices Data/Alerts Devices Data/Alerts Stream Stream . . . . . . Exploration Patterns Stream Processing Engine Publish Stream Stream Explorer Editor Stream Processing Engine Stream Processing Applications Analyzed DataIoT Cloud Service
- 44. Summary
- 45. Summary Treat events as events! Infrastructures for handling lots of events are available! IoT tends to make Big Data / Fast Data infrastructures necessary Know your use case/requirements to choose the right architecture! • Can my existing backend landscape handle the new IoT load? • Do I have to handle huge amount of events in “real-time”? • Do I need to filter/aggregate data before invoking existing backend systems? • Do I want to do Advanced Analytics (predictive analytics) where historical information is necessary? • What is the network bandwidth between device/gateway and cloud/backend? • Centralized or Decentralized IoT solution?
- 46. J