Inside Tripadvisor’s real-time personalization with ScyllaDB and AWS

SPONSORED BY SCYLLADB
Inside Tripadvisor’s
real-time personalization
with ScyllaDB and AWS
Dean Poulin
DAT204-S
Data Engineering Team Lead
Felipe Cardeneti Mendes
Technical Director

Technical Director, ScyllaDB
Felipe Cardeneti Mendes is an IT Specialist with years of experience on
distributed systems and open source technologies. He has co-authored
three Linux books and is a frequent speaker on public events and
conferences to promote open source technologies. At ScyllaDB, he
works as a Technical Director.
Speaker info

What is important for data-intensive
applications?
High
throughput
Low
latency
Predictable
cost

ScyllaDB: The database for game changers
• For data-intensive applications that require high
throughput and predictable low latencies
• Close-to-the-metal design takes full advantage of
modern infrastructure
▪ >5x higher throughput
▪ >20x lower latency
▪ >75% TCO savings
• Compatible with Apache Cassandra and
Amazon DynamoDB
• DBaaS/cloud, enterprise, and open source
solutions
“ScyllaDB stands apart...It’s the rare
product that exceeds my expectations.”
– Martin Heller, InfoWorld Contributing Editor and Reviewer
“For 99.9% of applications, Scylla delivers
all the power a customer will ever need, on
workloads that other NoSQL databases
can’t touch – and at a fraction of the cost
of an in-memory solution.”
– Dor Laor, CEO of ScyllaDB, in article by
Forbes Contributor Adrian Bridgwater

AWS relationship
• ScyllaDB is an AWS ISV Accelerate Partner
• ScyllaDB has two AWS Marketplace listings
▪ ScyllaDB Enterprise
▪ ScyllaDB Cloud (DBaaS)
• AWS Marketplace TCV grew over 200% in FY23; expected to grow 3x growth this year
• Foundation of the relationship is technical excellence on EC2 instances; led to ScyllaDB being
mentioned in the I4i launch blog and demonstrating superior AWS Graviton performance
• ScyllaDB is an AWS Graviton Ready Partner
• ScyllaDB was a sponsor at AWS Summits in London, India, and Tel Aviv and is
a sponsor at AWS re:Invent
• ScyllaDB sponsored the AWS Marketplace Conclave and participated in the
AWS Startup Roadshow in India

Game changers leveraging ScyllaDB
Powering India’s top
social media platform
Video recommendation
management
Real-time fraud
detection
Seamless experiences
across content & devices
Network security
threat detection
Power ~50M X1 DVRs
with billions of reqs/day
Precision healthcare
via Edison AI
Inventory hub for
retail operations
Property listings
and updates
Cryptocurrency
exchange app
Geography-based
recommendations
Predictable performance
for on sale surges
Online gaming ad
targeting
Media streaming
for 45M+ subscribers
IT service
management
Real-time ML-driven
recommendations
Connecting people around
the globe
GPS-based exercise
tracking
Global operations – Avon,
The Body Shop & more
Real-time analytics
Uniﬁed ML feature store
across the business
Distribution of game
assets in Unreal Engine
Uber scale,
mission-critical chat &
messaging app
2,000,000 SKU
-commerce management

Finding the perfect ﬁt for high-throughput/
low-latency workloads
ScyllaDB is an excellent choice for
business-critical workloads that need
high throughput and predictable, low
latency
To understand if it’s right for you,
ﬁrst consider where your workload
falls with respect to high throughput
and/or low latency

Tripadvisor’s workload

Dean Poulin
Data Engineering Team Lead,
AI Services & Products, Tripadvisor
With over two decades of experience, Dean has led engineering efforts
across a variety of domains, from frontend and backend development to
technical management. Currently, as the Data Engineering Team Lead
at Tripadvisor, he guides a talented team in building and scaling
real-time personalization systems, leveraging microservices and
cutting-edge architectures.
Dean’s background includes key roles at startups like MDconnectME,
where he developed HIPAA-compliant platforms on AWS, and Orbius,
where he helped build the core technology. He’s passionate about
creating scalable solutions and building strong tech teams. Dean’s
extensive experience across startups, product management, and
scalable architectures uniquely positions him to solve complex technical
challenges. Dean holds a Bachelor of Science in Computer Science from
the University of Maine.
Speaker info

Tripadvisor at a glance
2000
Founded
TRIP
Stock ticker
$1.8B
Revenue
2,800+
Employees
400M+
Monthly users
1B+
Reviews

The challenge at a glance
Delivering real-time, hyper-relevant recommendations to users while
navigating scale and latency constraints
1 ms
P99 latency
50M+
Daily users
400M
Monthly users
2B+
Daily requests

Powered by ML personalization

Model serving architecture
• Runs in Amazon EKS
for high scalability
• 100+ ML models
serving live traffic
• Each model is
independently
scalable in Kubernetes

Our custom feature store
• Serves up to 5 million
features/sec
• Static data is served
directly from Redis
• 500,000 user
features/sec retrieved
from ScyllaDB
• User features require
real-time user-based
calculations

What’s an ML feature?
• An ML feature is an individual measurable property or
characteristic of the data being used to make predictions
• Features are the input variables or attributes that the ML model uses
to learn patterns and relationships from the data in order to make
predictions or classiﬁcations
Static features
• Restaurant awards
• Dining restrictions
User features
• Hotel bookings last year
• Reviews submitted last 30 days

The technologies powering Visitor Platform
• ScyllaDB for the online database
• Spring Boot microservices on
Amazon ECS Fargate
• Spark for data retention
• Spark on Kubernetes for loading offline
data into ScyllaDB
• Amazon Kinesis for streaming events
• Spark for point-in-time queries for
training ML models
• Data warehouse for ETL processing
Amazon
Kinesis

The Visitor Platform data ﬂow

Visitor Platform by the numbers
>2.1B unique visitors
500+ user audiences
Up to 425K OPS on ScyllaDB
9 TB stored in ScyllaDB (online)
125 TB in data warehouse (offline)

Why two databases?
Online real-time data
• Focused on now/today
• Identity management lookup
• Short-term retention with TTL
• For fast rapid retrieval of speciﬁc data
• Focus on real-time live site requirements
Offline data warehouse
• Huge date range
• For ML model training
• Long-term storage
• Less latency-sensitive
• Multipurpose

Visitor Platform microservices
Visitor Core
• Performs identity
management
• Maps users across
devices
• Records data online
and offline
• Drives data retention
Visitor Metric
• Enables robust querying
of facts and metrics for a
given visitor
• Uses custom
domain-speciﬁc
language known as
Visitor Query Language
(VQL)
• Example VQL:
Visitor Publisher
• Primary entry point to save
visitor facts and metrics into
Visitor Platform
• Validates input data with
custom validation rules
• Asynchronously calls Visitor
Saver and Audience Manager
Visitor Saver
• Saves visitor facts and
metrics online and offline
• Sends facts and metrics to
the offline data warehouse
through an Amazon Kinesis
stream

Roundtrip microservice latency
• Visitor Platform microservices operate
with extremely low latency requirements
• 1B+ requests per day

ScyllaDB latency

Partitioning data into ScyllaDB

Why ScyllaDB?
A monstrously fast live serving database with
the lowest possible latencies
• Better performance than Cassandra
• Eliminate the operational burden of Cassandra
• Ease of migration – with zero downtime
• AWS BYOA option

ScyllaDB on AWS: Cloud BYOA

Thank you!
Please complete the session
survey in the mobile app
Dean Poulin
Data Engineering Team Lead
Technical Director

Inside Tripadvisor’s real-time personalization with ScyllaDB and AWS

More Related Content

Similar to Inside Tripadvisor’s real-time personalization with ScyllaDB and AWS (19)

More from ScyllaDB (20)

Recently uploaded (20)

Inside Tripadvisor’s real-time personalization with ScyllaDB and AWS