Building a Distributed Data Streaming Architecture for Modern Hardware with Seastar

1 like1,775 views

Redpanda is a new real-time streaming storage engine designed for modern hardware, emphasizing operational simplicity, zero data loss, and significantly improved speed compared to traditional systems. It leverages advancements in hardware, including SSDs and multi-core processors, while supporting Kafka API compatibility for easy integration. The architecture utilizes a thread-per-core model and async scheduling to optimize performance and parallelism.

Related topics:

Apache Kafka • NoSQL Database Insights

Building a Distributed Data Streaming Architecture for Modern Hardware with Seastar

Building a Distributed Data Streaming Architecture for Modern Hardware with Seastar

Building a Distributed Data Streaming Architecture for Modern Hardware with Seastar

Building a Distributed Data Streaming Architecture for Modern Hardware with Seastar

Building a Distributed Data Streaming Architecture for Modern Hardware with Seastar

Building a Distributed Data Streaming Architecture for Modern Hardware with Seastar

Building a Distributed Data Streaming Architecture for Modern Hardware with Seastar

Building a Distributed Data Streaming Architecture for Modern Hardware with Seastar

Building a Distributed Data Streaming Architecture for Modern Hardware with Seastar

Building a Distributed Data Streaming Architecture for Modern Hardware with Seastar

Building a Distributed Data Streaming Architecture for Modern Hardware with Seastar

Building a Distributed Data Streaming Architecture for Modern Hardware with Seastar

Building a Distributed Data Streaming Architecture for Modern Hardware with Seastar

Ad

Recommended

PPTX

Scylla @ Disney+ Hotstar

PDF

Scylla Summit 2016: Graph Processing with Titan and Scylla

PPTX

High-Load Storage of Users’ Actions with ScyllaDB and HDDs

PDF

Seastar Summit 2019 vectorized.io

PPTX

ScyllaDB's Avi Kivity on UDF, UDA, and the Future

PDF

Scylla Summit 2022: ScyllaDB Rust Driver: One Driver to Rule Them All

PPTX

How Opera Syncs Tens of Millions of Browsers and Sleeps Well at Night

PPTX

FireEye & Scylla: Intel Threat Analysis Using a Graph Database

PDF

Latency and Consistency Tradeoffs in Modern Distributed Databases

PDF

Running a DynamoDB-compatible Database on Managed Kubernetes Services

PPTX

Seastar Summit 2019 Keynote

PPTX

Scylla Summit 2019 Keynote - Avi Kivity

PDF

Introducing Scylla Open Source 4.0

PDF

Scylla Summit 2022: Rakuten’s Catalog Platform Migration from Cassandra to Sc...

PDF

Scylla Virtual Workshop 2020

PDF

Lookout on Scaling Security to 100 Million Devices

PDF

Scylla Summit 2016: Analytics Show Time - Spark and Presto Powered by Scylla

PPTX

MongoDB vs Scylla: Production Experience from Both Dev & Ops Standpoint at Nu...

PDF

Running Scylla on Kubernetes with Scylla Operator

PPTX

Scylla Summit 2019 Keynote - Dor Laor - Beyond Cassandra

PPTX

Scylla Summit 2018: Cassandra and ScyllaDB at Yahoo! Japan

PPTX

Scaling Up Machine Learning Experimentation at Tubi 5x and Beyond

PDF

Introducing Scylla Manager: Cluster Management and Task Automation

PDF

ScyllaDB @ Apache BigData, may 2016

PPTX

Scylla Summit 2022: Scylla 5.0 New Features, Part 1

PPTX

How to be Successful with Scylla

PDF

RedisConf17 - Lyft - Geospatial at Scale - Daniel Hochman

PPTX

Scylla Summit 2022: New AWS Instances Perfect for ScyllaDB

PDF

Redpanda and ClickHouse

PDF

Migrating and Running DBs on Amazon RDS for Oracle

More Related Content

PPTX

Scylla @ Disney+ Hotstar

PDF

Scylla Summit 2016: Graph Processing with Titan and Scylla

PPTX

High-Load Storage of Users’ Actions with ScyllaDB and HDDs

PDF

Seastar Summit 2019 vectorized.io

PPTX

ScyllaDB's Avi Kivity on UDF, UDA, and the Future

PDF

Scylla Summit 2022: ScyllaDB Rust Driver: One Driver to Rule Them All

PPTX

How Opera Syncs Tens of Millions of Browsers and Sleeps Well at Night

PPTX

FireEye & Scylla: Intel Threat Analysis Using a Graph Database

Scylla @ Disney+ Hotstar

Scylla Summit 2016: Graph Processing with Titan and Scylla

High-Load Storage of Users’ Actions with ScyllaDB and HDDs

Seastar Summit 2019 vectorized.io

ScyllaDB's Avi Kivity on UDF, UDA, and the Future

Scylla Summit 2022: ScyllaDB Rust Driver: One Driver to Rule Them All

How Opera Syncs Tens of Millions of Browsers and Sleeps Well at Night

FireEye & Scylla: Intel Threat Analysis Using a Graph Database

What's hot (20)

PDF

Latency and Consistency Tradeoffs in Modern Distributed Databases

PDF

Running a DynamoDB-compatible Database on Managed Kubernetes Services

PPTX

Seastar Summit 2019 Keynote

PPTX

Scylla Summit 2019 Keynote - Avi Kivity

PDF

Introducing Scylla Open Source 4.0

PDF

Scylla Summit 2022: Rakuten’s Catalog Platform Migration from Cassandra to Sc...

PDF

Scylla Virtual Workshop 2020

PDF

Lookout on Scaling Security to 100 Million Devices

PDF

Scylla Summit 2016: Analytics Show Time - Spark and Presto Powered by Scylla

PPTX

MongoDB vs Scylla: Production Experience from Both Dev & Ops Standpoint at Nu...

PDF

Running Scylla on Kubernetes with Scylla Operator

PPTX

Scylla Summit 2019 Keynote - Dor Laor - Beyond Cassandra

PPTX

Scylla Summit 2018: Cassandra and ScyllaDB at Yahoo! Japan

PPTX

Scaling Up Machine Learning Experimentation at Tubi 5x and Beyond

PDF

Introducing Scylla Manager: Cluster Management and Task Automation

PDF

ScyllaDB @ Apache BigData, may 2016

PPTX

Scylla Summit 2022: Scylla 5.0 New Features, Part 1

PPTX

How to be Successful with Scylla

PDF

RedisConf17 - Lyft - Geospatial at Scale - Daniel Hochman

PPTX

Scylla Summit 2022: New AWS Instances Perfect for ScyllaDB

Latency and Consistency Tradeoffs in Modern Distributed Databases

Running a DynamoDB-compatible Database on Managed Kubernetes Services

Seastar Summit 2019 Keynote

Scylla Summit 2019 Keynote - Avi Kivity

Introducing Scylla Open Source 4.0

Scylla Summit 2022: Rakuten’s Catalog Platform Migration from Cassandra to Sc...

Scylla Virtual Workshop 2020

Lookout on Scaling Security to 100 Million Devices

Scylla Summit 2016: Analytics Show Time - Spark and Presto Powered by Scylla

MongoDB vs Scylla: Production Experience from Both Dev & Ops Standpoint at Nu...

Running Scylla on Kubernetes with Scylla Operator

Scylla Summit 2019 Keynote - Dor Laor - Beyond Cassandra

Scylla Summit 2018: Cassandra and ScyllaDB at Yahoo! Japan

Scaling Up Machine Learning Experimentation at Tubi 5x and Beyond

Introducing Scylla Manager: Cluster Management and Task Automation

ScyllaDB @ Apache BigData, may 2016

Scylla Summit 2022: Scylla 5.0 New Features, Part 1

How to be Successful with Scylla

RedisConf17 - Lyft - Geospatial at Scale - Daniel Hochman

Scylla Summit 2022: New AWS Instances Perfect for ScyllaDB

Ad

Similar to Building a Distributed Data Streaming Architecture for Modern Hardware with Seastar (20)

PDF

Redpanda and ClickHouse

PDF

Migrating and Running DBs on Amazon RDS for Oracle

PPTX

Keine Angst vorm Dinosaurier: Mainframe-Integration und -Offloading mit Confl...

PDF

JDD2014: Real Big Data - Scott MacGregor

PDF

Big problems Big Data, simple solutions

Claudio Pontili

PDF

Big problems Big data, simple AWS solution

Jean-Claude Sotto

PDF

Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...

PDF

A Big Data Lake Based on Spark for BBVA Bank-(Oscar Mendez, STRATIO)

PDF

2017 AWS DB Day | AWS 데이터베이스 개요 - 나의 업무에 적합한 데이터베이스는?

Amazon Web Services Korea

PDF

Architecting Data in the AWS Ecosystem

PDF

Streaming architecture patterns

PDF

Bangalore Meetup - Enable realtime machine learning with streaming data

PPTX

Rapid Prototyping in PySpark Streaming: The Thermodynamics of Docker Containe...

Richard Seymour

PDF

Transforming the Database: Critical Innovations for Performance at Scale

PDF

Webinar - Big Data: Let's SMACK - Jorg Schad

PDF

Data Engineering Quick Guide

ODP

Hadoop Ecosystem Overview

Gerrit van Vuuren

PPTX

One Tool to Rule Them All- Seamless SQL on MongoDB, MySQL and Redis with Apac...

Tim Vaillancourt

PDF

IMCSummit 2015 - Day 2 Developer Track - A Reference Architecture for the Int...

In-Memory Computing Summit

PDF

Reference architecture for Internet of Things

Redpanda and ClickHouse

Migrating and Running DBs on Amazon RDS for Oracle

Keine Angst vorm Dinosaurier: Mainframe-Integration und -Offloading mit Confl...

JDD2014: Real Big Data - Scott MacGregor

Big problems Big Data, simple solutions

Claudio Pontili

Big problems Big data, simple AWS solution

Jean-Claude Sotto

Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...

A Big Data Lake Based on Spark for BBVA Bank-(Oscar Mendez, STRATIO)

2017 AWS DB Day | AWS 데이터베이스 개요 - 나의 업무에 적합한 데이터베이스는?

Amazon Web Services Korea

Architecting Data in the AWS Ecosystem

Streaming architecture patterns

Bangalore Meetup - Enable realtime machine learning with streaming data

Rapid Prototyping in PySpark Streaming: The Thermodynamics of Docker Containe...

Richard Seymour

Transforming the Database: Critical Innovations for Performance at Scale

Webinar - Big Data: Let's SMACK - Jorg Schad

Data Engineering Quick Guide

Hadoop Ecosystem Overview

Gerrit van Vuuren

One Tool to Rule Them All- Seamless SQL on MongoDB, MySQL and Redis with Apac...

Tim Vaillancourt

IMCSummit 2015 - Day 2 Developer Track - A Reference Architecture for the Int...

In-Memory Computing Summit

Reference architecture for Internet of Things

Ad

More from ScyllaDB (20)

PDF

Understanding The True Cost of DynamoDB Webinar

PDF

Database Benchmarking for Performance Masterclass: Session 2 - Data Modeling ...

PDF

Database Benchmarking for Performance Masterclass: Session 1 - Benchmarking F...

PDF

New Ways to Reduce Database Costs with ScyllaDB

PDF

Designing Low-Latency Systems with Rust and ScyllaDB: An Architectural Deep Dive

PDF

Powering a Billion Dreams: Scaling Meesho’s E-commerce Revolution with Scylla...

PDF

Leading a High-Stakes Database Migration

PDF

Achieving Extreme Scale with ScyllaDB: Tips & Tradeoffs

PDF

Securely Serving Millions of Boot Artifacts a Day by João Pedro Lima & Matt ...

PDF

How Agoda Scaled 50x Throughput with ScyllaDB by Worakarn Isaratham

PDF

How Yieldmo Cut Database Costs and Cloud Dependencies Fast by Todd Coleman

PDF

ScyllaDB: 10 Years and Beyond by Dor Laor

PDF

Reduce Your Cloud Spend with ScyllaDB by Tzach Livyatan

PDF

Migrating 50TB Data From a Home-Grown Database to ScyllaDB, Fast by Terence Liu

PDF

Vector Search with ScyllaDB by Szymon Wasik

PDF

Workload Prioritization: How to Balance Multiple Workloads in a Cluster by Fe...

PDF

Two Leading Approaches to Data Virtualization, and Which Scales Better? by Da...

PDF

Scaling a Beast: Lessons from 400x Growth in a High-Stakes Financial System b...

PDF

Object Storage in ScyllaDB by Ran Regev, ScyllaDB

PDF

Lessons Learned from Building a Serverless Notifications System by Srushith R...

Understanding The True Cost of DynamoDB Webinar

Database Benchmarking for Performance Masterclass: Session 2 - Data Modeling ...

Database Benchmarking for Performance Masterclass: Session 1 - Benchmarking F...

New Ways to Reduce Database Costs with ScyllaDB

Designing Low-Latency Systems with Rust and ScyllaDB: An Architectural Deep Dive

Powering a Billion Dreams: Scaling Meesho’s E-commerce Revolution with Scylla...

Leading a High-Stakes Database Migration

Achieving Extreme Scale with ScyllaDB: Tips & Tradeoffs

Securely Serving Millions of Boot Artifacts a Day by João Pedro Lima & Matt ...

How Agoda Scaled 50x Throughput with ScyllaDB by Worakarn Isaratham

How Yieldmo Cut Database Costs and Cloud Dependencies Fast by Todd Coleman

ScyllaDB: 10 Years and Beyond by Dor Laor

Reduce Your Cloud Spend with ScyllaDB by Tzach Livyatan

Migrating 50TB Data From a Home-Grown Database to ScyllaDB, Fast by Terence Liu

Vector Search with ScyllaDB by Szymon Wasik

Workload Prioritization: How to Balance Multiple Workloads in a Cluster by Fe...

Two Leading Approaches to Data Virtualization, and Which Scales Better? by Da...

Scaling a Beast: Lessons from 400x Growth in a High-Stakes Financial System b...

Object Storage in ScyllaDB by Ran Regev, ScyllaDB

Lessons Learned from Building a Serverless Notifications System by Srushith R...

Recently uploaded (20)

PDF

The Rise and Fall of 3GPP – Time for a Sabbatical?

PPTX

Cloud computing and distributed systems.

PDF

cuic standard and advanced reporting.pdf

SimoneTitaniumTomase

PPTX

VMware vSphere Foundation How to Sell Presentation-Ver1.4-2-14-2024.pptx

blackmambaettijean

PDF

Spectral efficient network and resource selection model in 5G networks

PPTX

MYSQL Presentation for SQL database connectivity

PPTX

Big Data Technologies - Introduction.pptx

DOCX

The AUB Centre for AI in Media Proposal.docx

PDF

Agricultural_Statistics_at_a_Glance_2022_0.pdf

SandeepSingh286037

PPTX

sap open course for s4hana steps from ECC to s4

sreeni2106invites

PDF

Diabetes mellitus diagnosis method based random forest with bat algorithm

PPTX

KOM of Painting work and Equipment Insulation REV00 update 25-dec.pptx

muhammadmuneebnaeem7

PDF

Blue Purple Modern Animated Computer Science Presentation.pdf.pdf

PDF

Architecting across the Boundaries of two Complex Domains - Healthcare & Tech...

PDF

Empathic Computing: Creating Shared Understanding

Mark Billinghurst

PDF

Electronic commerce courselecture one. Pdf

PDF

Encapsulation_ Review paper, used for researhc scholars

PDF

How UI/UX Design Impacts User Retention in Mobile Apps.pdf

PPTX

20250228 LYD VKU AI Blended-Learning.pptx

PDF

KodekX | Application Modernization Development

The Rise and Fall of 3GPP – Time for a Sabbatical?

Cloud computing and distributed systems.

cuic standard and advanced reporting.pdf

SimoneTitaniumTomase

VMware vSphere Foundation How to Sell Presentation-Ver1.4-2-14-2024.pptx

blackmambaettijean

Spectral efficient network and resource selection model in 5G networks

MYSQL Presentation for SQL database connectivity

Big Data Technologies - Introduction.pptx

The AUB Centre for AI in Media Proposal.docx

Agricultural_Statistics_at_a_Glance_2022_0.pdf

SandeepSingh286037

sap open course for s4hana steps from ECC to s4

sreeni2106invites

Diabetes mellitus diagnosis method based random forest with bat algorithm

KOM of Painting work and Equipment Insulation REV00 update 25-dec.pptx

muhammadmuneebnaeem7

Blue Purple Modern Animated Computer Science Presentation.pdf.pdf

Architecting across the Boundaries of two Complex Domains - Healthcare & Tech...

Empathic Computing: Creating Shared Understanding

Mark Billinghurst

Electronic commerce courselecture one. Pdf

Encapsulation_ Review paper, used for researhc scholars

How UI/UX Design Impacts User Retention in Mobile Apps.pdf

20250228 LYD VKU AI Blended-Learning.pptx

KodekX | Application Modernization Development