AdStage: Monacella: An Relational Object Database using Cassandra as the Datastore
0 likes1,773 views
The document discusses the evolution of Monacella, an object-relational database developed to manage ad network data using a highly parallelized ETL pipeline. Due to limitations with Postgres, the team opted for Cassandra, leveraging their existing experience to build a simplified, object-oriented data model. Future plans include further optimizations, potential shifts to different data stores, and comparisons with similar systems like Doradus.
AdStage: Monacella: An Relational Object Database using Cassandra as the Datastore
- 1. Monacella: An Object-Relational Database
- 2. Origin Story • need to import ad network data • highly parallelized ETL pipeline • hit Postgres connection limits during beta • considered sharding, but didn't have ops experience for it • instead decided to try Cassandra because • we already had ops experience with it • low maintenance • complexity would be in app layer where we were experts © 2015. All Rights Reserved. 2
- 3. Naive Architecture © 2015. All Rights Reserved. 3 campaign_id: string account_id: string created_at: timestamp data: string:string /network/adwords/ad/1234 ads campaigns accounts secondary indexes ad_id: string
- 4. Sensible Architecture © 2015. All Rights Reserved. 4 filter_name: string shard_id: integer entity_id: string none | active | inactive | ... ads campaigns accounts secondary indexes stored_filters ... facet_id: string ...
- 5. Oops! I Made an Object-Relational Database • at first didn't know what I had, but then slowly realized: • rows are objects with types/classes • objects have type-based relationships to other objects • support for object relationships but not arbitrary joins • who would want this? • need horizontal scalability • don't need all the features of a relational database... • ...but do need object relationships © 2015. All Rights Reserved. 5
- 6. Monacella and the Future • build out as a database with Cassandra as the datastore • simplify data model • stored filter optimizations • change datastore to Dynomite over Redis or Postgres • Doradus • Spider does the same thing as Monacella but maybe better • and it has an OLAP layer - something we also built! • sharded Postgres? © 2015. All Rights Reserved. 6
- 7. Thank you