Radically Outperforming DynamoDB @ Digital Turbine with SADA and Google Cloud
Download as PPTX, PDF0 likes886 views
Digital Turbine, under VP Joseph Shorter, migrated from AWS DynamoDB to ScyllaDB due to performance and cost challenges. The transition was smooth, with a 20% reduction in monthly costs and the capability to handle significantly more traffic. Future plans involve further optimizations and integrating machine learning to enhance performance.
Radically Outperforming DynamoDB @ Digital Turbine with SADA and Google Cloud
- 1. Radically Outperforming DynamoDB @ Digital Turbine Joseph Shorter, VP Platform Architecture at Digital Turbine Miles Ward, CTO at SADA
- 2. WHO ARE THESE GUYS?
- 3. Joseph Shorter VP Platform Architecture @ Digital Turbine Since joining Digital Turbine Joe has established the SDLC process, led enterprise architecture, Data Architecture, Security, and more. He has successfully led the automation and modernization of a major business unit and most recently completed the migration from AWS to GCP with the help of Google, SADA Systems, and several internal technical experts. As a key decision maker for determining technology choices Joe is the right person to discuss ScyllaDB and why it was chosen. ScyllaDB Customer Tech Executive Proud Nerd Author Artist….. :)
- 4. WINNER, GOOGLE CLOUD SALES PARTNER OF THE YEAR– NORTH AMERICA For the 6th year, SADA has been recognized for delivering transformative customer experiences with the power of Google Cloud. Largest GCP Reseller and Implementer. ScyllaDB Partner. Cloud geeks :)
- 5. Brand Advertisers ● Award-winning creatives ● Industry-leading attention App Advertisers ● Direct-to-Device UA ● Single-Tap Programmatic ● Store-like Hubs Telecom Partners ● Value-added user engagement ● New revenues from media dollars Digital Turbine’s Combination of On-Device and In-App Tech Transforms The Mobile Ecosystem Opportunity App Publishers ● UA & monetization growth engine ● Premium brand ads with superior UX Tech In Apps Tech On-Device SUPPLY DEMAND
- 6. 06 Proprietary Google Cloud Next ‘23 ~120B Requests per day (1.5M QPS) >1B Daily impressions >1T Outgoing transactions per day ~10 TB Of data produced daily Hundreds of Millions Of unique devices 30K-50K Cores running at any given time 21M Devices per month 135M Preloads per month Digital Turbine Scale
- 7. ■ First, it is vendor specific and was not an option for GCP since we were migrating away from AWS we needed a viable alternative ■ Scalability and cost was increasing based on out monthly reports ■ With a sizeable cluster we were still seeing performance issues with write operations. This concern was begging to prompt conversations about re- architecting some of our solutions as our workloads and client bases increases month over month. What’s the problem with DynamoDB?
- 8. HERE’S WHAT WE DID
- 9. Why we made the ScyllaDB choice? ■ We want easy. ■ ScyllaDB had a DynamoDB API that would allow us to simplify the migration. Minimal code refactoring. ■ We want to save money. ■ We spent time sizing the cluster based on historical metrics from AWS and determined that the ScyllaDB cluster we needed should provide cost savings. However, we would need to validate after migration and validating performance. ■ We want it faster too. ■ We hoped that the performance gains would be significant enough to control cost as we onboard new clients and scale our services Our Reasons for Migration
- 10. It was pretty straight forward actually. We did a POC to prove out this would work as we expected and created an easy to use library that each of our microservices could use and make things just work. It all took less than a sprint (2 weeks) of 1 developer time to get it all working and tested. So, turned out it was very easy How hard was it?
- 11. So, in the comparison we did we stuck with apples to apples and only included our production clusters since those workloads would be fairly stable. We also wanted to ensure the baseline in GCP was as clean as possible so we waited until we were running in production a couple of months before taking the metrics. What we found was that on average we saw 20% reduction in monthly cost. Other points that should be made clear: ■ We have still more cost optimization opportunities left to explore ■ The ScyllaDB clusters can handle much more traffic so our steady need to expand resources when using DynamoDB will not be as frequent How much did we save?
- 12. With same work loads it is clear that ScyllaDB outperforms DynamoDB where it counts ■ ScyllaDB Alternator cluster,Cluster ~2,000 ops/sec (90% reads , 10% Writes) ■ Leveraging DynamoDB API ■ DynamoDB ~1,400 ops/sec (90% reads , 10% Writes) ■ We observed light throttling on the DynamoDB cluster Was it faster?
- 13. OK, SO WHAT’S NEXT?
- 14. ■ Optimization, Optimization, Optimization ■ We are looking into using Vertex ML to provide better predictions for some of the APIs used by our partners. Once we train our models, the next step will involve integrating ScyllaDB to provide that enhanced result serving and performance. What’s next?
- 15. Do a comprehensive POC to gain real understanding if the solution and architecture you are proposing ( and possibly betting your company’s future on) will have the outcomes you are counting on. ■ Our POC did not just include SCyllaDB but it also included other vendor tools and new infrastructure because we wanted to see the solution working in a more realistic scenario. Any advice for the rest of us?
- 16. Thanks!