Real-Time or Analytics Workloads... Why Not Both?
Download as PPTX, PDF0 likes321 views
Felipe Cardeneti Mendes discusses the capabilities of ScyllaDB as an analytics engine capable of handling both real-time and batch workloads efficiently, processing over 1.2 billion points per second. He highlights the importance of workload prioritization and characterization to optimize resource usage and performance, particularly during high demand. By defining service levels and managing workload characteristics, ScyllaDB can cater to both interactive and batch tasks successfully.
Real-Time or Analytics Workloads... Why Not Both?
- 1. Real-Time or Analytics Workloads... Why Not Both? Felipe Cardeneti Mendes, Solution Architect at ScyllaDB
- 2. Felipe Mendes ■ Published Author ■ ScyllaDB Committer ■ IT Specialist & Solution Architect ■ Open Source Enthusiast Your photo goes here, smile :)
- 3. ScyllaDB as an Analytics Engine
- 4. An IoT Application Total amount of data points 526 billion temperature readings 1,000,000 sensors, representing homes in an area 365 days (1 year storage requirement) 1 reading per minute
- 5. Analytics over the entire data? How long would it take at normal speeds? We need more if analytics are a part of the pipeline We need ScyllaDB 200,000 points/second 730 hours (30 days) 1 million points/second 146 hours (almost a week)
- 6. Easy Peasy Scanning 3 months of data Finished Scanning. Succeeded 132,480,000,000 rows. Failed 0 rows. RPC Failures: 0. Took 110,892.91 ms Processed 1,194,666,083 rows/s Absolute min: 19.71, date 2019-08-27, sensorID 473869 Absolute max: 135.21, date 2019-08-27, sensorID 473869
- 7. Easy Peasy Scanning the entire dataset Finished Scanning. Succeeded 525,599,474,400 rows. Failed 0 rows. RPC Failures: 0. Took 542,191.31 ms Processed 969,398,554 rows/s Absolute min: 68.00, date 2019-05-28, sensorID 82114 Absolute max: 79.99, date 2019-03-19, sensorID 152594
- 8. We can efficiently process over 1.2 billion points per second (we’ll process whatever you need, too!)
- 10. 10
- 11. P99 climbs to unacceptable values The Latency Problem
- 12. Throughput gradually drops, load distribution becomes unfair The Throughput Problem
- 13. Why Contention Happens? Primarily lack of system resources (disk I/O, CPU time) ■ Not necessarily a problem ■ Introduces queueing
- 14. Easy then, let's simply ... Addressing the Problem? ■ Divide and conquer! ■ Division in Space (Multi DC) ■ Division in time (off peak OLAP)
- 16. Isolation and Performance Optimization Background Tasks User Tasks
- 17. Shares Different workloads require different priorities ■ Meet SLAs ■ Flexible Configuration ■ Adaptability to Changing Conditions
- 18. Getting Started ■ Set up authentication, assign Roles to each workload ■ Prioritization is wired on the authenticated users role ■ Define your Service Levels ■ For a primary workload: ■ For a secondary one: ■ Profit! CREATE SERVICE LEVEL main WITH shares = 200 CREATE SERVICE LEVEL secondary WITH shares = 600
- 19. Prioritization and Isolation are NOT enough!
- 21. Workloads Characteristics: Time 21 The timeout dilemma: 1. Timeout should follow: 𝑇𝑠𝑒𝑟𝑣𝑒𝑟 ≤ 𝑇𝑐𝑙𝑖𝑒𝑛𝑡 2. For Real-time: ■ Can’t be too high ■ Incurs retries or dropped requests ■ Excessive retries result in wasted resources 3. For Analytics: ■ Can’t be too low ■ Otherwise Batch will likely fail ■ High throughput will typically increase latencies due to contention
- 22. Workloads Characteristics: Shedding 22 Overload response: 1. Interactive workload: ■ Throttling won't help ■ Delaying response to user A will not cause some user B to stop sending requests ■ Unbound concurrency 2. Batch workload: ■ Just throttle ■ Allow us to have a knob that controls the pace of the analytics workload ■ Bounded concurrency
- 23. Introducing Workload Characterization 23 Ideally – we want the database to behave differently: ■ For Real-time: ■ Have low timeout ■ Load shedding (fail excessive requests), as the database can NOT slow down interactive workloads. ■ Dedicate most of the resources to this workload. ■ For Batch: ■ Relatively higher timeout ■ Apply back-pressure via throttling ■ Use mostly unused resources
- 24. ■ For Real-Time: ■ For Analytics: Why not just hint the database with specifics? Introducing Workload Characterization 24 Have low timeout (30ms) timeout=30ms Load shedding AND workload_type=interactive Dedicate most of the resources AND shares=800 Have relatively high timeout (5s) timeout=5s Throttling AND workload_type=batch Use mostly unused resources AND shares=200
- 25. Takeaways 25 ■ ScyllaDB powers both Analytics & Real-time intensive workloads ■ Workload Prioritization helps with: ■ Infrastructure Consolidation ■ Resource Optimization ■ Performance Isolation ■ Workload Characterization provide: ■ Workload specific settings ■ Distinct overload and timeout responses
- 26. Stay in Touch Felipe Mendes felipemendes@scylladb.com @cardeneti82118 fee-mendes Find me on LinkedIn