Citi Tech Talk: Messaging Modernization
0 likes94 views
This document discusses moving to an event-driven architecture using Confluent. It begins by outlining some of the limitations of traditional messaging middleware approaches. Confluent provides benefits like stream processing, persistence, scalability and reliability while avoiding issues like lack of structure, slow consumers, and technical debt. The document then discusses how Confluent can help modernize architectures, enable new real-time use cases, and reduce costs through migration. It provides examples of how companies like Advance Auto Parts and Nord/LB have benefitted from implementing Confluent platforms.
![Legacy infrastructure and architecture did not meet
real-time performance requirements [Before]
CRM
WMS
MDM
HRM Data Lake Finance
Legacy ITSM
DC MOBILE/WEB
STORE
(PoS)
DC
STORE
(PoS)
Branch (PoS)
ECOMM Catalog
OMS
Inventory
ON PREMISES
Merchandising
(Legacy)
CCDB
CTDB
HJ1
HJ2
JDA
?](https://image.slidesharecdn.com/confluentformessagingmodernization-techtalk-231102093124-ce1f55db/85/Citi-Tech-Talk-Messaging-Modernization-32-320.jpg)
![Future-proof data architecture for real-time invoicing and
dynamic pricing [After]
CRM
WMS
MDM
HRM Finance
Legacy ITSM
DC MOBILE/WEB
STORE (PoS) DC
STORE (PoS) BRANCH (PoS)
?
Revenue (5)
Location (2)
Pricing (1)
Catalog
Customer (6)
Inventory (4)
Employee (7) Orders (8)
Data Lake
ECOMM Catalog
OMS
Inventory
ON PREMISES
Merchandising
(Legacy)
Products (3)](https://image.slidesharecdn.com/confluentformessagingmodernization-techtalk-231102093124-ce1f55db/85/Citi-Tech-Talk-Messaging-Modernization-33-320.jpg)
Citi Tech Talk: Messaging Modernization
- 1. Modern Messaging : Journey to Event Driven Architecture Pete Godfrey - Manager, Solutions Engineering TECH TALK
- 2. Confidential and Proprietary. Intro 2 Pete Godfrey Solutions Engineering Manager Confluent
- 3. Confidential and Proprietary. Mortgage Every Business is Becoming Software Taxi Grocery Banking Then Now c 3
- 4. Example: Loan Application Process Software-using 1 3 5 4 6 2 PATIENT INSURANCE POLICY WAITING PERIOD COVERAGE AND PAYMENT TEST REQUEST APPROVE DENY Software-defined 1 PATIENT TEST REQUEST (APP) 3 APPROVE DENY $ INSURANCE POLICY COVERAGE AND PAYMENT ! WAITING PERIOD 2 Seconds 4 1-2 Days
- 5. Copyright 2021, Confluent, Inc. All rights reserved. This document may not be reproduced in any manner without the express written permission of Confluent, Inc. Unlocking Value from Data, in Real-time, is Imperative to Transformation 5 Data Data leveraged Interactions Security events App data Workflows Search data Purchases IoT and so much more Data Time Customers Employees
- 6. Copyright 2021, Confluent, Inc. All rights reserved. This document may not be reproduced in any manner without the express written permission of Confluent, Inc. Enterprises are Undertaking Multiple Initiatives to Enable Transformation 6 Microservices Cloud Machine Learning Automation
- 7. The Problem: your use of data has changed but the supporting infrastructure has not!
- 8. Paradigm for Data-at-Rest: Databases Databases Slow, daily batch processing Simple, static queries
- 9. Paradigm for Data Movement: Messaging Middleware Producer of messages Producer of messages Message Oriented Middleware Consumer of messages Consumer of messages
- 10. Message System Benefits Message System Drawbacks Real-time (low latency) Broad Adoption (familiar technology) Lacks Common Structure No Stream Processing No Persistence After Consumption Low Fault Tolerance at Scale Slow Consumers Drag Performance Complexity and Technical Debt
- 11. Message System Benefits Message System Drawbacks Real-time (low latency) Broad Adoption (familiar technology) Lacks Common Structure No Stream Processing No Persistence After Consumption Low fault Tolerance at Scale Slow Consumers Drag Performance Complexity and Technical Debt Confluent Real-time Broad Adoption Stream Processing Durable & Persistent Elastically Scalable Reliable Schema Registry
- 13. Confidential and Proprietary. Confluent: A New Paradigm for Data-in-Motion 13 Rich front-end customer experiences Real-time Data Real-time Stream Processing Real-time backend operations QUERY A Sale A shipment A Trade A Customer Experience
- 14. Copyright 2021, Confluent, Inc. All rights reserved. This document may not be reproduced in any manner without the express written permission of Confluent, Inc. Implication: Central Nervous System for Enterprise Real-time Inventory Real-time Fraud Detection Real-time Customer 360 Machine Learning Models Real-time Data Transformation ... Stream Processing Applications Data-in-Motion Pipeline ... ... ... ... Data Stores Logs 3rd Party Apps Custom Apps/Microservices SaaS Apps
- 15. Copyright 2020, Confluent, Inc. All rights reserved. This document may not be reproduced in any manner without the express written permission of Confluent, Inc. Confluent Modernizes Enterprise Messaging Infrastructure Enable real-time workloads and applications Achieve any scale Build a modern architecture Future-proof data architecture and achieve high performance and availability at scale Migrate with ease Easily augment or migrate apps to Confluent Reduce TCO and tech debt Future proof cloud-ready architecture
- 16. Confluent offers a way to modernize your data architecture and accelerates your developer velocity Build a modern architecture Embrace modern DevOps and easily deploy, manage, and scale your infrastructure, whether on your private cloud or a public cloud Easily replay messages, provide event sourcing and command sourcing for error handling and eliminate complexity for resending messages Use best-of-breed solutions across your architecture and easily integrate them with Confluent connectors, reducing your reliance on a single infrastructure vendor Cloud-native design Message replay Simple integrations Confluent offers a way to modernize your data architecture and accelerates your developer velocity
- 17. Legacy messaging brokers are monolithic Kafka supports containerization and multi-DC deployments Broker Broker Broker Producers Consumers Broker Producers Consumers More point of failure, lack of horizontal scalability & higher risk of downtime: poor fit for mission-critical use cases Fault tolerant and horizontally scalable: ensures high availability for mission-critical apps and more cloud-native experience Confluent offers a way to modernize your data architecture and accelerates your developer velocity Build a modern architecture
- 18. Confluent offers a way to modernize your data architecture and accelerates your developer velocity Build a modern architecture ksqlDB provides everything you need to build a complete real-time application entirely with SQL syntax DB APP APP DB PULL PUSH CONNECTORS STREAM PROCESSING MATERIALIZED VIEWS ksqlDB 1 2 APP
- 19. Confluent offers a way to modernize your data architecture and accelerates your developer velocity Build a modern architecture Effortlessly filter, join, and enrich your data streams with Flink, the de facto standard for stream processing Enable high-performance and efficient stream processing at any scale, without the complexities of infrastructure management Experience Kafka and Flink as a unified platform, with fully integrated monitoring, security, and governance Confluent Cloud for Apache Flink® Simple, Serverless Stream Processing Easily build high-quality, reusable data streams with the industry’s only cloud- native, serverless Flink service Available for preview in select regions – see the docs for regional availability
- 20. Confluent offers a way to modernize your data architecture and accelerates your developer velocity Build a modern architecture App 1 ! Schema Registry Kafka topic ! Serializer App 1 Deserializer Data discovery • Validate data compatibility and get warnings • Let developers focus on deploying apps Scale with confidence • Store a versioned history of all schemas • Enable evolution of schemas while preserving backwards compatibility for existing consumers
- 21. Confluent was designed for high performance and high availability at massive scale to support your mission-critical use cases Confluent can support all of your enterprise’s streaming use cases by achieving a dramatically higher throughput 15x improvement in throughput performance One platform to deploy, secure, and manage to support all of your streaming workloads. Read more about our internal performance benchmarking Achieve any scale
- 22. Confluent was designed for high performance at massive scale to support your mission-critical use cases 5ms Confluent achieves <5ms latency at massive throughput Synchronize data across your organization in real-time Take action on insights from your data immediately Remove data silos by moving from batch to data streaming Confluent was designed for high performance at massive scale to support your mission-critical use cases Achieve any scale
- 23. We offer a robust set of connectors to pull data from your MQs into Confluent... ...and connectors to push data from Confluent into your modern, cloud-native sinks Confluent provides the tools and services required to effectively migrate from your messaging queue Migrate with ease
- 24. Professional Services Expert Training Enterprise Support Confluent provides committer-driven expertise to support you throughout your migration and modernization effort Confluent is responsible for over 80% of the commits to Kafka, providing you with the right expertise to ensure data streaming success Confluent provides the tools and services required to effectively migrate from your messaging queue Migrate with ease
- 25. Reduce your messaging TCO 30% TCO Reduction Illustrative TCO reduction from modernizing legacy messaging to Confluent & Kafka Reduces to $0 past year 2 Confluent helps you reduce your messaging TCO as you migrate to Kafka and beyond Reduce TCO and tech debt
- 26. How We Do It
- 27. Copyright 2021, Confluent, Inc. All rights reserved. This document may not be reproduced in any manner without the express written permission of Confluent, Inc. Current approach to Messaging Third Party Consumer Apps Integration layer Producer Apps Consumer Apps Messaging middleware
- 28. Copyright 2021, Confluent, Inc. All rights reserved. This document may not be reproduced in any manner without the express written permission of Confluent, Inc. Step 1: Use pre-built connectors to start integrating data from your applications Schema Registry ksqlDB Producer Apps Consumer Apps Messaging middleware Third Party Consumer Apps
- 29. Copyright 2021, Confluent, Inc. All rights reserved. This document may not be reproduced in any manner without the express written permission of Confluent, Inc. Step 2: Refactor applications to use JMS APIs and replace messaging middleware over time JMS API Producer Apps Consumer Apps JMS API Schema Registry ksqlDB Third Party Consumer Apps
- 30. Copyright 2021, Confluent, Inc. All rights reserved. This document may not be reproduced in any manner without the express written permission of Confluent, Inc. Step 3: Move off JMS and build streaming applications with new event-driven patterns Kafka API Producer Apps Consumer Apps Push Schema Registry ksqlDB Third Party Consumer Apps Pull New Event-driven patterns ● Event notification ● Event carried state storage ● Event sourcing ● CQRS (Command Query Responsibility Segregation) Unify and simplify ✓ Lower TCO ✓ ✓ Future-proof data architecture
- 31. Case Study: Advance Auto Parts
- 32. Legacy infrastructure and architecture did not meet real-time performance requirements [Before] CRM WMS MDM HRM Data Lake Finance Legacy ITSM DC MOBILE/WEB STORE (PoS) DC STORE (PoS) Branch (PoS) ECOMM Catalog OMS Inventory ON PREMISES Merchandising (Legacy) CCDB CTDB HJ1 HJ2 JDA ?
- 33. Future-proof data architecture for real-time invoicing and dynamic pricing [After] CRM WMS MDM HRM Finance Legacy ITSM DC MOBILE/WEB STORE (PoS) DC STORE (PoS) BRANCH (PoS) ? Revenue (5) Location (2) Pricing (1) Catalog Customer (6) Inventory (4) Employee (7) Orders (8) Data Lake ECOMM Catalog OMS Inventory ON PREMISES Merchandising (Legacy) Products (3)
- 35. Challenge: Enable advanced, real-time analytics across the bank to support improvements in fraud detection, customer retention, and trade/investment analysis, and help differentiate NORD/LB from its competitors in the market Solution: Use Confluent to build a new, event streaming-based core banking platform Results: ● Improved competitive differentiation ● More event streaming adoption ● Reduction of streaming infrastructure costs “Confluent is enabling us to address our need for a scalable, highly available messaging infrastructure that allows us to decouple our producers from consumers, setting the stage for us to be more flexible, agile, and responsive to change.” — Sven Wilbert, Data Manager at NORD/LB
- 36. Challenge: Build a conversational chatbot service that incorporates complex technologies such as fulfillment, natural-language understanding, and real-time analytics Solution: Use Confluent to build a fast, super-scalable event-driven architecture that could handle immense traffic spikes and also provide other guarantees around delivery semantics Results: ● Near-zero downtime even during huge traffic spikes ● Rapid acceleration of new-skill onboarding ● Doubling of NPS rating “We chose event-driven architecture as the core of our platform, for which we needed a messaging service that gave us all the guarantees…not to mention that it had to be extremely scalable, highly available, and simple to use. Kafka hit all of these markers, and by using Confluent Cloud, our team was able to reduce the bottom line and operational burden.” — Ravi Vankamamidi, Senior Director, Technology, at Expedia Group
- 37. Challenge: Power a national digital bank initiative and enterprise-wide digital transformation by democratizing data and decoupling systems across the IT landscape Solution: Use Confluent Platform to create a center of excellence that helps teams harness data in motion and make data available to systems throughout the bank Results: ● 50% reduction in time-to-market ● Reduced mainframe and message queue costs ● De-risked adoption of new technology paradigms “When we saw the demand for harnessing data in motion growing, working with Confluent and investing in our center of excellence enabled us to implement a solution in a way that best benefits the enterprise by reducing complexity, costs, and time to market.” — Mike Onders, EVP Chief Data Officer and Divisional CIO at KeyBank