Maximizing the Capabilities of Kafka – Real-Time Streaming of Event-Driven Data | Sanjai Marimadaiah, Push Technology
0 likes381 views
The document discusses the capabilities of Push Technology's Diffusion® platform, which extends Kafka for real-time streaming of event-driven data at internet scale. It highlights key requirements such as scalability, efficient bandwidth usage, and real-time response. The platform features a hierarchical topic structure, open protocol support, and fine-grained security for managing data distribution and access.
Maximizing the Capabilities of Kafka – Real-Time Streaming of Event-Driven Data | Sanjai Marimadaiah, Push Technology
- 1. © 2021 Push Technology Ltd Real-Time Streaming of Event-Driven Data Maximizing Capabilities of Kafka
- 2. © 2021 Push Technology Ltd Sanjai Marimadaiah Chief Product Officer Push Technology Inc. PushTechnology.com Extending Kafka for Distribution at Internet Scale TOPIC
- 3. © 2021 Push Technology Ltd Use Case - Streaming Market Data Risk Modeling Market Data FLAT TOPIC STRUCTURE Perfect for large volumes of Data Kafka Streams API Library to build streaming apps Responsibility on Consumer LOG-BASED EVENT-BROKER Efficient for replaying/distributing data to services & micro-services Applications over the Internet Audit
- 4. © 2021 Push Technology Ltd Requirements to Distribute Data at Internet Scale v MASSIVE SCALABILITY Millions of Topics & Connections v EFFICIENT USE OF BANDWIDTH Reduce cost of data consumption Flow-Control, Delay, Conflation v REAL-TIME RESPONSE Latency in milli-seconds v SIMPLIFIED CLIENT SOFTWARE Time to Market and Maintenance v OPEN PROTOCOLS & INTERFACES MQTT, REST API, HTTP Pub/Sub, Req-Response REAL-TIME EVENT BROKER An event-broker with critical real-time streaming functionality DEEP HIERARCHICAL TOPIC TREE Organize and enrich data for efficient distribution of specific data at scale SECURITY : FINE GRAINED & ROLE-BASED Access to specific parts of data in a JSON Per-topic permissions in a hierarchical model CAPABILITIES OF SOLUTION
- 5. © 2021 Push Technology Ltd Extend Kafka with Diffusion® to Distribute Data at Scale Derivative Pricing Risk Modeling Audit Dashboards Market Data Diffusion® Intelligent Event-Data Platform Client Applications over the internet
- 6. © 2021 Push Technology Ltd Diffusion® Extends Kafka to Distribute Data at Scale BUILT-IN ADAPTER TO KAFKA Easy to ingest data from Kafka Topics REAL-TIME EVENT BROKER Queue-based event broker with Event History, Pub/Sub, Request-Response, Flow-Control DELTA STREAMING of EVENT DATA Sends only bytes changed from previous message OPEN PROTOCOL SUPPORT MQTT, REST API, HTTP to server variety of clients REAL-TIME DATA ENRICHMENT Deep Hierarchical Topic Trees DSLs to enrich data and set security policies
- 7. © 2021 Push Technology Ltd Learn more about Kafka + Diffusion® Go.PushTechnology.com/KafkaSummit v Kafka Tutorials v Sample Code on GitHub v Implement on Diffusion Cloud for FREE
- 8. © 2021 Push Technology Ltd © 2020 Push Technology Ltd Private and Confidential Thank You