Datastax Expedia
4 likes12,479 views
This document summarizes Cassandra adoption and use cases at Expedia over time. In 2011, Expedia implemented Cassandra for pricing indexes, session state caching, and transaction logging. In 2012, they upgraded to DataStax and expanded Cassandra's use to additional pricing indexes, search via SOLR, and high-speed transaction logging. For pricing indexes, Expedia moved from a 9-node Cassandra cluster to a 27-node cluster with DataStax to support more markets and elements. For logging, they moved from SQL servers to a 72-node DataStax cluster with Hadoop for reporting, reducing costs by $1.5M. By the end of 2012, Expedia planned to have 240 DataSt
Datastax Expedia
- 1. Cassandra Evolution at Expedia
- 2. Adoption timeline 2011 • Q2 Cassandra 0.6 live for First Pricing index • Q3 Cassandra 0.7 live for Session state cache • Q3 Cassandra 0.7 for high speed transaction logging • Q4 Cassandra Splunk integration complete for put/get • Q4 Cassandra Web Portal and RESTful interface available 2012 • Q2 DataStax upgrade for first pricing index & enable SOLR • Q2 DataStax live for Second Pricing index & enable SOLR • Q2 DataStax live for high speed transaction logging • Q2 Onboarding new markets to indexes • Q2 DataStax PoC for Content cache • Q3 Upgrade Web Portal to use SOLR and Hadoop functionality
- 3. Pricing Index Use Case
- 4. Pricing Index Ring Solution Current Solution • 9 nodes split across 3 racks running Cassandra 1.0 • There is a separate 2 node App tier written in Java that access data in CFS • On top of that is a 4 node SOLR tier for search Next Gen 2Q12 solution • DataStax upgrade and elimination of app and SOLR nodes • Given this change we can onboard two new indexes and all major markets • New ring is 27 nodes in one physical datacenter and will expand to 54 nodes in June split across two datacenters • Expanding from initial small market set to more markets, price elements and options • Using DataStax solution is showing positive results in the lab and shaving seconds off of user search times
- 5. High Speed Logging Use Case
- 6. Logging Ring Solution Current Solution • 40 MSSQL database high end servers using 160TB Tier 1 SAN and 14 days of online information and no redundancy or replication • 5 Reporting archive MSSQL databases that keep 30 day roll ups of the 40 databases for trending • Need to know SQL to pull any data or reports, have multiple db developers working on team Next Gen 2Q12 solution • DataStax deployment of 72 commodity nodes across 2 physical datacenter with replication • 1/3 of nodes will have Hadoop enabled for reporting and will be available via web service for non-SQL business personnel • 90 day retention window of raw data • Cost Savings $1.5M in 2012
- 7. Logging Ring Architecture Data Data Center 1 Center 2
- 9. Summary • Current footprint 132 nodes production, 27 nodes dev, 24 nodes PoC • Expedia is investing heavily in Cassandra and specifically DataStax solution to optimize user experience • Heavily leveraging Hadoop and SOLR integrations to get rapid value from solutions • Using DataStax is shaving seconds off of search times which equates to significantly better customer experience • Continuing to look at new use cases and expect to have 240 nodes of DataStax in production by Q412
- 10. Questions ?