Serverless Messaging with Microsoft Azure by Steef-Jan Wiggers
- 1. #AIMSTalks
- 2. Serverless Messaging with Microsoft Azure Steef-Jan Wiggers Azure & IoT Domain Lead
- 3. steefjan.wiggers@codit.eu +31 653 12 29 57 @SteefJan nl.linkedin.com/in/steefjan
- 4. What can you expected in this session • Serverless messaging • Messaging In Azure • Messaging scenarios • Demo’s
- 5. Serverless 5
- 6. Evolution to Serverless • You can choose: • VM → • Containers → • Orchestrators → • PaaS → • Serverless On-premise IaaS PaaS Serverless
- 7. “Serverless computing is a cloud-computing execution model in which the cloud provider dynamically manages the allocation of machine resources. Pricing is based on the actual amount of resources consumed by an application, rather than on pre- purchased units of capacity. It is a form of utility computing.” Source (Wikipedia Serverless Computing): https://en.wikipedia.org/wiki/Serverless_computing Defining Serverless
- 8. Thoughts on Serverless - Infinite scale requires atomic workloads and infinite resources. - Reality is resources are always finite and many workloads not atomic. - Messaging is for decoupling and transferring state or data secure and reliably.
- 9. Serverless in Azure - Serverless Compute → Azure Functions - Serverless Database → Azure Cosmos DB - Serverless Events → Azure Event Grid - Serverless Realtime → Azure Signal R Service - Serverless Workflow → Logic Apps & Durable Functions - Serverless IoT → IoT Hubs - Serverless Analytics → Application Insights
- 10. Serverless Messaging Message: >Down Message: >Stay Message: >Come Is this serverless messaging?
- 11. What I mean is: You have options! We have various messaging capabilities in Azure! Event Hubs Telemetry stream ingestion Service Bus & Azure Queues Cloud messaging Event Grid Event distribution Logic Apps Workflow and LOB Integration IoT Hub IoT messaging and manage- ment Notification Hubs Mobile push notifications Relay Discovery, Firewall/NAT Traversal
- 13. What do people do with messaging? - Order processing - Logging / telemetry - Connected devices - Notification / event driven systems
- 14. Messaging Services in Azure •Azure Service Bus •Event Hubs •Event Grid •Storage Queue
- 15. Service Bus Event Hubs Storage Queues Big data streamingEnterprise messaging Simple task queues Event Grid Cross cloud reactive eventing Building Blocks
- 16. Enterprise Messaging Business state transition • Transfer of money and material • Central arbiter of state / truth of transition or ownership • Rich control of communication features • You know a lot ahead of time • You know the nouns: • Who • What • Where • Competitive offering from Amazon: SQS, Amazon MQ, Google Pub/Sub
- 17. Reacting dynamically to the world around you • Cross App / Service / Cloud / Organization • Light weight • Low cost • Few features, but important ones • You probably don’t know the nouns and may not care Eventing
- 18. Service Bus Enterprise Messaging Azure Event Grid Reactive Programming • Queue Semantics • Rich filters • Instantaneous consistency • Strict ordering • JMS • Geo-replication & Availability • Rich broker features • Event raised by sources • Independent individual messages • Push mechanism • Filtering and routing • Pay as you go • Fan out Push-Pull - Push-Push Cloud Pub/Sub Models
- 19. Task Queue Coordinate simple tasks across compute (workers) • Low cost • Pay as you Go • Few features • Easy to use • Have as many queues as you like
- 20. Cloud Queues Service Bus Queues - Storage Queues Service Bus Queues Storage Queues • Queue Semantics • Rich filters • Instantaneous consistency • Strict ordering • JMS • Geo-replication & Availability • Rich broker features • Part of Storage infrastructure • Simple REST-based interface • Pull mechanism • Reliable • Persistence messaging • Only queues
- 21. Big Data Streams Flow data and telemetry in real-time • Durable re-playable buffer • Designed as a stream • Events to an event hub via AMQP or HTTP • Real-time and batch processing • Feeding Big Data and Analytics • Pre-purchase capacity in terms of throughput units • Offers an Kafka Endpoint • Competitive offering from Amazon Kinesis, Apache Kafka, and Google Pub/Sub
- 22. Brought to you by Wagner Silveira - https://goo.gl/LcjMQB Kafka Endpoint - Demo Consumer Groups Producer Consumer Event Hubs
- 24. .NET Code
- 25. Messaging considerations Protocol Format Size Security Frequency Reliability Monitoring Networking
- 26. VMSS Cloud Services VMs Logic Apps SQL DB Express Route ASA SQL DW Power BI Data Lake Cosmos DB Functions AKS ACS
- 28. Tollbooth License Plate Recognition
- 29. License plate OCR • Use the Recognition Services Computer Vision API and its built- in OCR capabilities • The image processing function can make a REST call to send the photo and read the JSON data in return Smart solution
- 30. Storage blob Azure Event Grid Function Cognitive Services Vision API Smart Services - Demo
- 31. Change feed
- 32. Serverless Home Automation - Demo data data data data start/stop data start/stop start/stop Cosmos DB Event Hubs IoT Hub Functions data data notify
- 33. The processing required by an application can easily be broken down into a set of independent steps. Pipes and filters
- 34. Microservices containers Request DataProvisioning API Gateway Provision Request Message Storage SQL Instance in ElasticPool Key Vault Provisioning Queue Provisioning WebJob Trigger Provision Store Connection strings Provision Container Azure AppService Provision Request Message
- 35. Microservice processing - Demo Request message Container ServiceBus Listener Service Get Blob Edi-file Blob Storage Json-file Blob Storage Storage Explorer message ServiceBus Listener Service Next step!
- 36. K8S – Microservice Log
- 37. Use Azure Messaging Services together
- 38. Data and event pipeline - Demo
- 39. Summary Choose the right messaging service(s) Workloads Cross-platform Costs DevOps and Management Security and compliance
- 40. Happy Messaging!
- 41. Thank you! Keep in touch. Call or mail us. Ask us. Happy to help.