[ScaleConf 2020] How to Tame Your Microservices: Evolving Airbnb's Architecture

How to Tame
Your Services:
Evolving Airbnb’s
Architecture
Jessica Tai / 5 March 2020 / ScaleConf

Scaling our engineering team
Beijing
San Francisco
Portland
New York City
MontrealSeattle
Los Angeles
San Jose
@jessicamtai

Hi. I’m Jessica.
I pair program with my Corgi dog.
@jessicamtai

Agenda
Architecture history
Monolith
Scaling challenges
Services
SOA v2 patterns
Abstraction
Lessons learned
Takeaways
@jessicamtai

Agenda
Monolith
Scaling challenges
Service
SOA v2 patterns
Abstraction
@jessicamtai
Lessons learned
Takeaways

Agenda
Monolith
Scaling challenges
Services
SOA v2 patterns
Abstraction
@jessicamtai
Lessons learned
Takeaways

Growth, growth,
growth!
Vanja Josifovski, Airbnb Homes CTO
“
@jessicamtai

SOA V2SOA V1Monolith
2008 - 2016 2016 - 2019 2019 - present
Airbnb architecture phases
@jessicamtai

Monolith
2008 - 2016
@jessicamtai

Monorail,ourRubyonRailsmonolith
@jessicamtai

Architecture pre-2016
Monolith
Client traffic
Shared
database
Monorail
Data access query
Business logic
Presentation view
@jessicamtai

Architecture pre-2016
Monolith
Client traffic
Shared
database
Monorail
@jessicamtai

Moreincidents Slowerdeploytrains
@jessicamtai

SOA v1
2016 - 2019
@jessicamtai

Architecture 2016
Service-oriented arch (SOA)
API traffic
Routing & view
Business logic, model, data
via services
Client traffic
@jessicamtai
Monorail

Architecture 2016
@jessicamtai

Architecture 2016
@jessicamtai
Data service:
Entity read and writes

Architecture 2016
@jessicamtai
Computed data
service:
Combines data
sources
for multiple contexts

Architecture 2016
@jessicamtai
Middle-tier service:
Complex business
logic

Architecture 2016
@jessicamtai
Presentation service:
Synthesize data for
web and mobile UI

Checkout presentation
@jessicamtai
Homes data
Validation
mid-tier
Pricing
computed data

Architecture 2018
API gateway
Middleware
Session data
service
Authentication
data service
Oauth data
service
Risk computed
data service
...
Request
context
Web rendering
service
HTML view
@jessicamtai
Routing
Presentation, logic, data

2019
After 3 years of migration,
enforced “Monorail freeze”
@jessicamtai

SOA wins
Performance wins due to
parallelization of dependencies
@jessicamtai

SOA wins
Faster code deployment
@jessicamtai

SOA wins
Better guarded, reliable code
@jessicamtai

Scaling challenges
with SOA v1
@jessicamtai

@jessicamtai
Monorail
Data access query
Business logic
Presentation view
Monorail mapped to SOA v1

Monorail mapped to SOA v1
@jessicamtai
Monorail
Data access query
Business logic
Presentation view

Data
Mid-tier
Mid-tier
DataData
@jessicamtai
Outgrowing SOA v1

Number of services at Airbnb
0
150
300
450
600
2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020
500+ engineers
2,000+ engineers
@jessicamtai

Challenges with SOA v1
Defect rate
Fragmented business
logic
Developer velocity
Too many
dependencies
New verticals
Scattered, custom
APIs & schemas
Scalability
Unguarded data
access
@jessicamtai

SOA v2
2019 - present
@jessicamtai

Slow developer velocity:
Too many dependencies
@jessicamtai

SOA v2 patterns:
Decompose
presentation services
@jessicamtai

SOA v1
Bloated presentation services
Presentation
UI logic
Data fetching
Business logic
@jessicamtai

SOA v2
Decompose presentation services
Presentation
UI logic
Data aggregation service
Business logic
@jessicamtai

Presentation
UI logic
A) Server-driven UI:
Display layout of home highlights
@jessicamtai

Presentation
UI logic
Reviews UserHomes
B) Aggregate data:
homes, reviews, user data stores
@jessicamtai

Presentation
UI logic
Business logic
DataData
C) Apply business logic:
Rank top highlights
@jessicamtai

Lack of scalability:
Unguarded, suboptimal access to data
@jessicamtai

SOA v2 patterns:
Aggregator
@jessicamtai

DataDataData
@jessicamtai
[SOA v1] No access control
Any service can call any other service

Presentation
Data DataData DataDataData
@jessicamtai
[SOA V2] Aggregator

{
reservation {
checkin_date,
guest {
first_name,
picture
},
host {
first_name,
picture
},
}
}
@jessicamtai
[SOA V2] Aggregator
Resolver Resolver Resolver
Map fields to
data source

{
reservation {
checkin_date,
guest {
first_name,
picture
},
host {
first_name,
picture
},
}
}
Map fields to
data source
@jessicamtai
[SOA V2] Aggregator

[SOA V2] Aggregator
{
reservation {
checkin_date,
guest {
first_name,
picture
},
host {
first_name,
picture
},
}
}
ServiceService
Batch fetch
fields
@jessicamtai

Migration to aggregator
Presentation
API unchanged
@jessicamtai

Poor defect rate:
Scattered, duplicated business logic
@jessicamtai

SOA v2 patterns:
Computed data as a
platform
@jessicamtai

@jessicamtai
“Superhost” status
•# of reservations
•High review ratings
•No cancellations

Service
[SOA v1] Fragmented computed data
Superhost example
Reservations
data service
@jessicamtai
Reviews data
service
Cancellations
data service
Client
Service

Superhost example
Owned by
separate teams
Reservations
data service
Reviews data
service
Cancellations
data service
@jessicamtai
Client
[SOA v1] fragmented computed data
Service
Service

[SOA v2] Computed data as modular platform
Superhost example
Client
Computed data
platform service
Reservations
data service
Data source config
Data attribute Source
A Reservations
B Reviews
C Cancellations
@jessicamtai
Reviews data
service
Cancellations
data service

[SOA v2] Computed data as modular platform
Superhost example
Computed data
platform service
Data source config
Business logic lambda
boolean isSuperhost(A, B, C) {
return A && B && C;
}
Reservations
data service
@jessicamtai
Reviews data
service
Cancellations
data service
Client

Computed data as modular platform
Superhost example
Computed data
platform service
Offline pipeline &
index framework
A
B
C
Eventual
consistency
@jessicamtai
Client

Migration to computed data
Computed data
service
DataDataDataData
Compare responses
from codepaths
Service
Presentation
@jessicamtai
Offline
comparison
framework

Difficult to onboard verticals:
Custom APIs slow to navigate
@jessicamtai

SOA v2 patterns:
Service blocks
@jessicamtai

[SOA v1] Service blocks
@jessicamtai

Computed
Logical grouping
of services
Presentation
UI logic
@jessicamtai

Presentation
UI logic
Logical grouping
of services
@jessicamtai
Computed

DataData
Facade service
Computed
@jessicamtai
Computed

Facade service
Unified API and schema
@jessicamtai

Facade service
Service
Service
No internal block calls to services outside of the block
@jessicamtai

Facade service Facade service
Presentation
UI logic
Service
Service
Service
@jessicamtai

[SOA v2] Facade + computed data
Unified service
Computed data
platform
Facade service &
Presentation
UI logic
@jessicamtai

Migration to service blocks
Facade service
DataDataData
Computed
Data aggregation serviceClient service
@jessicamtai

@jessicamtai
Users
Homes
Pricing
Initial service blocks
Core domain entities

SOA V2: Read path
computedFacade +
Presentation
UI logic
Service
Service
Service
computedFacade +
DataDataDataData
@jessicamtai

Pilot results
•Data aggregation service: 35% lower latency
[v1] Presentation service calls to data services
[v2] Single GraphQL query to aggregator
•Computed data platform: 30% faster engineer velocity
[v1] Implement in own service
[v2] Add to computed data platform
@jessicamtai

Design service architecture to
enable current growth needs.

[ScaleConf 2020] How to Tame Your Microservices: Evolving Airbnb's Architecture

More Related Content

What's hot (17)

Similar to [ScaleConf 2020] How to Tame Your Microservices: Evolving Airbnb's Architecture (20)

Recently uploaded (20)

[ScaleConf 2020] How to Tame Your Microservices: Evolving Airbnb's Architecture