Redis + Kafka = Performance at Scale | Julien Ruaux, Redis Labs
0 likes1,993 views
The document discusses the integration and performance capabilities of Redis and Kafka, particularly highlighting their use in microservices architectures and real-time applications. It emphasizes Redis Enterprise's features such as multi-tenancy, hybrid cloud deployment, and caching patterns for enhancing data processing and continuity. Additionally, it presents use cases like real-time inventory tracking and online fraud detection, demonstrating the efficiency of combining Redis with Kafka in various scenarios.
![Redis Data Structures
Lists
[ A → B → C → D → E ]
Hashes
{ A: "foo", B: "bar" }
Bitmaps
0011010101100111001010
Strings
"I'm a Plain Text String!"
Bit Field
{23334}{112345569}{766538}
Streams
{id: 1, body: {A: "foo"}}
Hyperloglog
00110101 11001110
Sorted Sets
{ A: 0.1, B: 0.3, C: 100 }
Sets
{ A , B , C , D , E }
Geospatial
{A: (51, 0.12), B: (32, 34)}
Key](https://image.slidesharecdn.com/julienruaux-210928232320/85/Redis-Kafka-Performance-at-Scale-Julien-Ruaux-Redis-Labs-5-320.jpg)
![Brewdis
Order
Management
System
Redis
Sink
Connector
Inventory
Manager
Update stream
"@style:IPA @abv:[6 8]"
Redis
Source
Connector
Ful
fi
llment](https://image.slidesharecdn.com/julienruaux-210928232320/85/Redis-Kafka-Performance-at-Scale-Julien-Ruaux-Redis-Labs-20-320.jpg)
Redis + Kafka = Performance at Scale | Julien Ruaux, Redis Labs
- 1. Julien Ruaux julien@redis.com Redis + Kafka Performance at Scale Kafka Summit 2021
- 2. Redis + Kafka Performance at scale Redis & Redis Enterprise Kafka Connector Industry Solution Spotlight Demo
- 3. Redis
- 4. The leading in-memory database platform ➡ Performance ➡ Simplicity ➡ Multi-Model Redis
- 5. Redis Data Structures Lists [ A → B → C → D → E ] Hashes { A: "foo", B: "bar" } Bitmaps 0011010101100111001010 Strings "I'm a Plain Text String!" Bit Field {23334}{112345569}{766538} Streams {id: 1, body: {A: "foo"}} Hyperloglog 00110101 11001110 Sorted Sets { A: 0.1, B: 0.3, C: 100 } Sets { A , B , C , D , E } Geospatial {A: (51, 0.12), B: (32, 34)} Key
- 7. BloomFilter Search Graph TimeSeries AI JSON Gears Redis Enterprise Data Models Linear Scalability HA Geo-Distribution Durability Backup & Restore Tiered-Memory Access Security Redis Enterprise Core Capabilities Multi-Tenant Redis Data Structures https://redislabs.com/redis-enterprise/advantages/
- 8. Hybrid Cloud Deployment | Database Migration | Cloud Bursting MANAGED SERVICE HYBRID DEPLOYMENT ON-PREMISES Conflict Free Active-Active Geo-Replication Redis Enterprise Managed Service | 1st Party SaaS Seamless Migration from Legacy Data Platforms 99.999% SLA* Seamless Upgrades A-A Geo-Replication Security Customer Success Private Link Continuous real-time replication and consistency w/ heterogenous platforms Kafka Connect
- 9. Caching Patterns using Redis Enterprise Cache-Aside Write-Behind Write-Through Read-Through Read-Replica Cache Prefetching Service Layer Service Layer Service Layer Service Layer read write read read read write write write read write write read read write Kafka Connect
- 10. Microservice Architecture with Redis + Kafka Real-Time Online Fraud Detection Kafka Summit 2020: Redis and Kafka - Simplifying Advanced Design Patterns within Microservices Architectures
- 11. Microservice Architecture - Industry and Architectural Analysis https://redislabs.com/blog/microservices-and-the-data-layer-new-idc-infobrief https://thenewstack.io/how-redis-simplifies-microservices-design-patterns
- 12. Kafka Connector for Redis Enterprise
- 13. Kafka Connector for Redis Enterprise 2-way bridge between Redis and Kafka Source Connector Any data structure Sink Connector Stream
- 14. Primary Use Cases Cloud Bursting | Business Continuity Hybrid Cloud Deployment Real-Time Search Cache Prefetching Blue-Green Deployment | Data Migration Multi-Phase Replatforming | Microservices Strangler Application Microservices CQRS | Event-Sourcing Real-Time ETL
- 15. Microservices CQRS | Cache Prefetching | Real-Time Search API Gateway Approve | Decline Payment Payment History read write Clearing & Settlement read consume publish https://thenewstack.io/how-redis-simplifies-microservices-design-patterns RediSearch Kafka Connect
- 17. Challenges Real-Time Inventory Track quantities in real-time ATP = Supply - Demand What is available where Products * Stores
- 18. Solution: Kafka + Redis Enterprise Real-Time Inventory
- 19. Demo
- 20. Brewdis Order Management System Redis Sink Connector Inventory Manager Update stream "@style:IPA @abv:[6 8]" Redis Source Connector Ful fi llment
- 21. Resources
- 22. Resources Redis Enterprise Kafka Connector Brewdis Demo on Github Seamless database migrations with Redis Connect Redis Enterprise Cloud Overview