Creating an Event Backbone for the Hybrid Cloud
1 like309 views
The document discusses using an event broker to enable event-driven interactions across distributed systems. It describes how systems are becoming more distributed and complex, with events needing to move across cloud, on-premise, and hybrid environments. An event broker like Solace can support this by providing unified messaging, streaming, and request/reply capabilities to integrate systems and leverage cloud services in a loosely coupled, event-driven way. The event broker acts as the central nervous system to reliably transmit events anywhere.
![How is data routed in an event broker?
• A quick lesson on wildcard topics
– Topic: type/vehicle/bus_number/latitude/longitude/route_number
• Example: geo/bus/1234/053.1282/-078.1793/34
– Subscriptions:
• geo/bus/#
• geo/bus/*/*/*/34
– Solace supports prefix wildcards in subscriptions:
• geo/bus/*/053.12*/-078.17*/34
• All Route 34 buses with latitude [53.1200,53.1299] and longitude [-78.1700,-78.1799]
© Solace
43](https://image.slidesharecdn.com/solacecodethefuture2-180924183029/85/Creating-an-Event-Backbone-for-the-Hybrid-Cloud-43-320.jpg)
Creating an Event Backbone for the Hybrid Cloud
- 1. Creating an Event Backbone for the Hybrid Cloud Aaron Lee Developer Advocate, Office of the CTO
- 3. Systems today are getting more distributed, making data movement more complicated. Context © Solace 3
- 5. In the beginning… • Companies ran their own datacentres – Huge investment in hardware, maintenance, support, and expertise – VMs, virtualization, and containerization allowed more efficient use of hardware – Wouldn’t it be nice if someone else could manage all the hardware, storage, networking? – The birth of IaaS, and PaaS 5 © Solace Proprietary & Confidential
- 6. In the beginning… • Software was monolithic – Database-centric, batch-oriented systems – Proprietary protocols and APIs – Concept of ESB to help integration and orchestration of services (SOA) – Rise of open-source, open standards, open protocols, and open APIs – Decomposing of larger systems into collections of microservices – Move to real-time, increase In volume, rethinking to Event-Driven Architectures 6 © Solace Proprietary & Confidential
- 7. In the beginning… • The Internet was simple – Client-Server interactions for webpages – RESTful Request-Reply HTTP protocol – Everything understands REST: firewalls, routers, gateways, load balancers, etc. – But also synchronous; but also one-way – Need for efficient, asynchronous, bidirectional protocols • WebSockets, MQTT, etc. 7 © Solace Proprietary & Confidential
- 8. In the beginning… © Solace Proprietary & Confidential 8 • “Things” were dumb – Telephones, cars, lights, refrigerators, houses, browsers, devices of all sorts – Massive improvements in miniaturized computing power and network access promoted connectivity – Growth continues, with 28B devices by 2021
- 9. Intersection of Trends Software Platforms Networking Volume & Real-time MAGIC!
- 11. Meet Solace Solace Overview and Evolution
- 12. World’s foremost experts in enterprise-grade messaging Seamless connectivity 250+ Employees 14 Offices 21 Patents
- 13. Telecom Financial Services / FinTech Gaming/GamblingGovernment Transportation Experienced, proven, trusted
- 14. Seamless Security Authentication Authorization Encryption Unified Administration Monitoring Management Upgrades Solace HybridEdge Solace Microgateway Messaging APIs Open APIs & Protocols Integration With Existing Applications and Infrastructure Solace PubSub+ Platform © Solace Proprietary & Confidential 14 Software Standard Edition Enterprise Edition Appliance 3530 3560 Cloud Starter Plans Enterprise Plans
- 15. Industry trends & challenges
- 16. Quest for agility is driving you to the cloud, to microservices, to EDA, to new tools… © Solace 16 Executives Digital Ambition • Transform? • Optimize? Cloud Opportunity • Cloud de jour? • Paas? Accelerate Innovation • Continuous intelligence • Combine data/event silos: IoT, ML/AI, Legacy Systems, etc. Architects Digital Optimization • Legacy systems • iPaaS vs. API Gateway? • Microservices? Cloud Necessary • Churn: target cloud keeps changing • Which PaaS? • Hybrid reality Innovate… • Events not data • Leverage ML/AI • Big data + IoT Sensors + Mobile Developers Sounds Great! • How do I integrate to get access to data? • Springboot for microservices? Cloud Native • Lambda? • Containers? • CI/CD and Infra as code Innovate! • Streaming • Event sourcing • Spark • I only know about MY silo’s API and data definitions
- 17. How do you enable event-driven interactions across all this stuff? Making the distribution of data more complicated © Solace 17 Legacy Applications IoT Legacy Application Infrastructure Cloud Private Public iPaaS / APIs SaaS
- 18. Start by migrating applications to the cloud, and building new microservices © Solace 18 You’ve got to get them talking Lambda Machine Learning EMR Kinesis And tap into that cloud’s services And each cloud provides its own way to do both
- 19. • Enable bi-directional data exchange? • Prevent on-premise and cloud lock in? • Support cloud bursting and arbitrage? • Integrate with cloud services? Now how do you integrate legacy/on-prem apps? Messaging Mainframe ? © Solace 19
- 20. • Take advantage of cloud provider innovations • Exchange data between clouds and on-premise • Managing cloud egress costs • Move cloud native apps between clouds for arbitrage What about communications between clouds? Messaging Mainframe ? ? © Solace 20
- 21. Enabling Event-Driven Communications in Context of Layers of Enterprise Requirements © Solace 21 Enabling Event-Driven Communications Performance Security / Governance High Availability / Disaster Recovery Elastic Scalability Flexibility / Future-proofing • API, environment and protocol agnostic • Authentication, encryption, auditability of data and events • Robustness, replication and real-time recovery • Automated provisioning, cloud bursting • High throughput, low latency
- 22. How do we enable this data movement and reduce communication complexity? Patterns © Solace 22
- 23. REST isn’t Always Best Great for: • Synchronous interactions • Externally facing APIs But: • Point-to-point • Tight service coupling • Not natively event-driven © Solace 23
- 24. Don’t use Y2K techniques to deal with 2018 realities How do you support the shift to “event-driven”? © Solace 24
- 25. Think Event Driven –Microservices have inputs and outputs • Inputs and outputs can be thought of as “events” –So what is an Event? • An event is an action or occurrence recognized by software, often originating asynchronously from the external environment, that may be handled by the software –Stop thinking about services INVOKING services! • Instead services emit events! –What/where do we emit these events to? • Messaging (preferability Solace!) © Solace 25
- 26. Message Exchange Patterns StreamingRequest/Reply Publish/Subscribe © Solace 26
- 27. Messaging APIs • The absolute basics: – connect(host, user, pw); disconnect() – subscribe(topic); unsubscribe(topic) – bind(queue); unbind(queue) – publish(message, destination) – onReceive(message) • And for good measure: – Message doRequest(requestMessage, destination) © Solace 27
- 28. Event Driven Interaction Considerations • Benefits – Limited service to service coupling (excludes data level) – No impact service upgrades – Easy to create new services and fail fast – Resilient to service failures – Generally more resource efficient – Enables fine grained service scaling • Challenges: – Not as Natural: Many architects and developers think in a “synchronous” way. Design pattern education is required. – Correlation: Services must be able to correlate different events since they occur asynchronously © Solace 28
- 29. Why Messaging? • Supports standard, open, event-driven APIs • Native pub/sub and ordering • Supports request/reply Eventing Requires Messaging & Streaming Why Streaming? • Replay • Native pub/sub and ordering © Solace 29
- 30. How most IT systems work today… How Humans Process the World © Solace 30 Central Nervous System Microservices Legacy Applications Users Log Aggregation Enterprise Monitoring Machine Learning NoSQL Database Relational Database Web Apps Vs. a system that makes you nervous
- 31. A system that reliably transmits events to interested apps and microservices across environments. Event Broker © Solace 31
- 32. Event Broker Event-driven systems need an event broker Unified support for: • Synchronous interactions Recognizes need for APIs and fact that they represent an event • Messaging guaranteed delivery of events regardless of source • Streaming partitioning and replay © Solace 32 Log Aggregation Enterprise Monitoring Machine Learning Microservices NoSQL Database Relational Database Legacy Applications Users Web Apps
- 33. How do you enable event-driven interactions across all this stuff? An event broker needs to event-enable everything © Solace 33 Legacy Applications IoT Legacy Application Infrastructure Cloud Private Public iPaaS / APIs SaaS
- 34. And Solace is the ultimate event broker © Solace 34 Legacy Applications IoT Legacy Application Infrastructure Cloud Private Public iPaaS / APIs SaaS Pub/Sub + Queueing + Request/Reply + Streaming
- 35. Solace makes it easy to create new applications and microservices… faster and easier! Solace & Pivotal Cloud Foundry © Solace 35
- 36. Chosen IaaS Elastic Runtime Solace Messaging Routers Other PCF Tiles Other PCF Tiles Diego Loggregator Service Brokers Blobstore Solace Service Broker PCF Services & Add Ons Apps Manager Auto-Scaling API Gateway Data Sync IPSec Add-on Redis MySQL Push Notifications GemFire Single Sign-On Metrics Server Ops Metrics Notifications BOSH Pivotal Elastic Runtime Solace Messaging Solace in PCF
- 37. Solace for Spring • https://github.com/solaceproducts – Spring Boot Auto-config for Solace Java APIs – Spring Cloud Connectors – Spring Streams coming soon! Presentation Title ©Solace — Confidential 37
- 38. Solace in Google • https://github.com/solaceproducts – Quickstart for GKE – Quickstart for Kubernetes • https://github.com/solacelabs – Quickstart for GCP • Also! Solace Cloud in GCP – Aug 2018 Presentation Title ©Solace — Confidential 38
- 40. 40 Copyright Solace Confidential NETS • Enabling cashless economy/transactions in Singapore and beyond – Millions of users and over 100k access points • Focused on shifting from offline processing to real-time • Real-time authentication is critical – Hinges on messaging – Enable AliPay, Tencent, IoT – They own datapath from point of sale to back-end © Solace 40
- 41. 41 Copyright Solace Confidential FAA • Solace fans out National Airspace System SWIM data feeds to airlines, airports, FlightAware, etc. • Traffic Flow/ Flight Plans STDDS – Collect/publish data from 150+ airports, SFPDS , TBFM, TFDM • Why Solace? – Predictable, high performance – High fanout for large number of connected systems over WANs – Fast fail over time to support business continuity – Easy to manage with more real-time monitoring and response – Built-in hardware compression reduce WAN bandwidth © Solace 41
- 42. Bidirectional, real-time streaming communication with advanced filtering Demo Time © Solace 42
- 43. How is data routed in an event broker? • A quick lesson on wildcard topics – Topic: type/vehicle/bus_number/latitude/longitude/route_number • Example: geo/bus/1234/053.1282/-078.1793/34 – Subscriptions: • geo/bus/# • geo/bus/*/*/*/34 – Solace supports prefix wildcards in subscriptions: • geo/bus/*/053.12*/-078.17*/34 • All Route 34 buses with latitude [53.1200,53.1299] and longitude [-78.1700,-78.1799] © Solace 43
- 44. Connect your apps. Try us free! dev.solace.com © Solace 44
- 45. GeoFiltering in Action © Solace 45