SlideShare a Scribd company logo

Os Solomon

oscon2007, profile picture
oscon2007

This document summarizes Mike Solomon's experience scaling YouTube's architecture using Python. It discusses: 1) Starting simply with all services on the same boxes before gradually separating services like search and thumbnails onto specialized machines. 2) Systematically removing bottlenecks revealed by user demand growth and replacing components to continuously improve. 3) Using Python's flexibility to enable migrating databases and services with minimal disruption. 4) Balancing machine resources by overlaying orthogonal tasks on the same hardware.

Technology
1 of 25
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
Os Solomon
Os Solomon
Os Solomon
Os Solomon
Os Solomon
Os Solomon
Os Solomon
Os Solomon
Os Solomon
Os Solomon
Os Solomon
Os Solomon
Os Solomon
Os Solomon
Os Solomon
Os Solomon
Os Solomon
Os Solomon
Os Solomon
Os Solomon
Os Solomon
Os Solomon
Os Solomon
Os Solomon
Os Solomon

More Related Content

PDF
Super Sizing Youtube with Python
didip
 
PPTX
Natural Laws of Software Performance
Gibraltar Software
 
PPTX
Scaling Systems: Architectures that Grow
Gibraltar Software
 
PPT
Vertical vs Horizontal Scaling
Mark Myers
 
PDF
Agile experiments in Machine Learning with F#
J On The Beach
 
PDF
Modern web dev_taxonomy
kevin_donovan
 
PPTX
Scaling Systems: Architectures that grow
Gibraltar Software
 
PPTX
CQRS
Fabian Vilers
 
Super Sizing Youtube with Python
didip
 
Natural Laws of Software Performance
Gibraltar Software
 
Scaling Systems: Architectures that Grow
Gibraltar Software
 
Vertical vs Horizontal Scaling
Mark Myers
 
Agile experiments in Machine Learning with F#
J On The Beach
 
Modern web dev_taxonomy
kevin_donovan
 
Scaling Systems: Architectures that grow
Gibraltar Software
 
CQRS
Fabian Vilers
 

What's hot (20)

PDF
Server’s variations bsw2015
Laurent Cerveau
 
PPTX
Modern Cloud Fundamentals: Misconceptions and Industry Trends
Christopher Bennage
 
PPT
Presentation on Large Scale Data Management
Chris Bunch
 
PPTX
Using a gateway to leverage on premises data in Power BI
Adam Saxton
 
PPTX
2016 Mastering SAP Tech - 2 Speed IT and lessons from an Agile Waterfall eCom...
Eneko Jon Bilbao
 
PDF
From Obvious to Ingenius: Incrementally Scaling Web Apps on PostgreSQL
Konstantin Gredeskoul
 
PPT
Tips & Tricks On Architecting Windows Azure For Costs
Nuno Godinho
 
PPTX
Deep Learning Frameworks Using Spark on YARN by Vartika Singh
Data Con LA
 
PPTX
Java Enterprise Performance - Unburdended Applications
Lucas Jellema
 
PPTX
Aws 12 Month Free Tier for Web Designers and Developers
Dylan Burris
 
PPTX
How to Build High Performance : WordPress
Dylan Burris
 
PDF
Trends in DNN compression
Kaushalya Madhawa
 
PDF
Metaflow: The ML Infrastructure at Netflix
Bill Liu
 
PDF
What's New in H2O Driverless AI? - Arno Candel - H2O AI World London 2018
Sri Ambati
 
PPT
Stat 5.4 Pre Sales Demo Master
reachtimsq
 
PPTX
Blockchain for the DBA and Data Professional
Karen Lopez
 
PDF
System Update 2010 CrossRef Workshops
Crossref
 
PDF
Re invent 2018 meetup presentation
Eliran Yamin
 
PPTX
Flink Forward SF 2017: Trevor Grant - Introduction to Online Machine Learning...
Flink Forward
 
PDF
Deep Learning Computer Build
PetteriTeikariPhD
 
Server’s variations bsw2015
Laurent Cerveau
 
Modern Cloud Fundamentals: Misconceptions and Industry Trends
Christopher Bennage
 
Presentation on Large Scale Data Management
Chris Bunch
 
Using a gateway to leverage on premises data in Power BI
Adam Saxton
 
2016 Mastering SAP Tech - 2 Speed IT and lessons from an Agile Waterfall eCom...
Eneko Jon Bilbao
 
From Obvious to Ingenius: Incrementally Scaling Web Apps on PostgreSQL
Konstantin Gredeskoul
 
Tips & Tricks On Architecting Windows Azure For Costs
Nuno Godinho
 
Deep Learning Frameworks Using Spark on YARN by Vartika Singh
Data Con LA
 
Java Enterprise Performance - Unburdended Applications
Lucas Jellema
 
Aws 12 Month Free Tier for Web Designers and Developers
Dylan Burris
 
How to Build High Performance : WordPress
Dylan Burris
 
Trends in DNN compression
Kaushalya Madhawa
 
Metaflow: The ML Infrastructure at Netflix
Bill Liu
 
What's New in H2O Driverless AI? - Arno Candel - H2O AI World London 2018
Sri Ambati
 
Stat 5.4 Pre Sales Demo Master
reachtimsq
 
Blockchain for the DBA and Data Professional
Karen Lopez
 
System Update 2010 CrossRef Workshops
Crossref
 
Re invent 2018 meetup presentation
Eliran Yamin
 
Flink Forward SF 2017: Trevor Grant - Introduction to Online Machine Learning...
Flink Forward
 
Deep Learning Computer Build
PetteriTeikariPhD
 
Ad

Similar to Os Solomon (20)

PDF
Scalable, good, cheap
Marc Cluet
 
PPS
Web20expo Scalable Web Arch
mclee
 
PPS
Web20expo Scalable Web Arch
guest18a0f1
 
PPS
Web20expo Scalable Web Arch
royans
 
PDF
My Sql And Search At Craigslist
MySQLConference
 
PPS
Scalable Web Architectures: Common Patterns and Approaches - Web 2.0 Expo NYC
Cal Henderson
 
PDF
Improving The Performance of Your Web App
Joe Stump
 
PDF
The Web Scale
Guille -bisho-
 
PPT
Scaling 101 test
Rashmi Sinha
 
PPT
Scaling 101
Chris Finne
 
PDF
Your backend architecture is what matters slideshare
Colin Charles
 
PPS
Scalable Web Arch
royans
 
PPS
Scalable Web Architectures - Common Patterns & Approaches
Cal Henderson
 
PPT
Planning for-high-performance-web-application
Nguyễn Duy Nhân
 
PPTX
Case Study with Answers.com on Scaling with Memcached and MySQL
answers
 
ODP
Scalable Architecture 101
Mike Willbanks
 
PPTX
Clustrix Database Percona Ruby on Rails benchmark
Clustrix
 
PDF
Flickr Architecture Presentation
web25
 
PPT
Apache Con 2008 Top 10 Mistakes
John Coggeshall
 
PPT
FOWA Scaling The Lamp Stack Workshop
dlieberman
 
Scalable, good, cheap
Marc Cluet
 
Web20expo Scalable Web Arch
mclee
 
Web20expo Scalable Web Arch
guest18a0f1
 
Web20expo Scalable Web Arch
royans
 
My Sql And Search At Craigslist
MySQLConference
 
Scalable Web Architectures: Common Patterns and Approaches - Web 2.0 Expo NYC
Cal Henderson
 
Improving The Performance of Your Web App
Joe Stump
 
The Web Scale
Guille -bisho-
 
Scaling 101 test
Rashmi Sinha
 
Scaling 101
Chris Finne
 
Your backend architecture is what matters slideshare
Colin Charles
 
Scalable Web Arch
royans
 
Scalable Web Architectures - Common Patterns & Approaches
Cal Henderson
 
Planning for-high-performance-web-application
Nguyễn Duy Nhân
 
Case Study with Answers.com on Scaling with Memcached and MySQL
answers
 
Scalable Architecture 101
Mike Willbanks
 
Clustrix Database Percona Ruby on Rails benchmark
Clustrix
 
Flickr Architecture Presentation
web25
 
Apache Con 2008 Top 10 Mistakes
John Coggeshall
 
FOWA Scaling The Lamp Stack Workshop
dlieberman
 
Ad

More from oscon2007 (20)

PDF
J Ruby Whirlwind Tour
oscon2007
 
ODP
Solr Presentation5
oscon2007
 
PDF
Os Borger
oscon2007
 
PDF
Os Harkins
oscon2007
 
PDF
Os Fitzpatrick Sussman Wiifm
oscon2007
 
PDF
Os Bunce
oscon2007
 
PDF
Yuicss R7
oscon2007
 
PDF
Performance Whack A Mole
oscon2007
 
ODP
Os Fogel
oscon2007
 
PDF
Os Lanphier Brashears
oscon2007
 
PPT
Os Tucker
oscon2007
 
PDF
Os Fitzpatrick Sussman Swp
oscon2007
 
PDF
Os Furlong
oscon2007
 
PDF
Os Berlin Dispelling Myths
oscon2007
 
PDF
Os Kimsal
oscon2007
 
PDF
Os Pruett
oscon2007
 
PDF
Os Alrubaie
oscon2007
 
PDF
Os Keysholistic
oscon2007
 
ODP
Os Jonphillips
oscon2007
 
PDF
Os Urnerupdated
oscon2007
 
J Ruby Whirlwind Tour
oscon2007
 
Solr Presentation5
oscon2007
 
Os Borger
oscon2007
 
Os Harkins
oscon2007
 
Os Fitzpatrick Sussman Wiifm
oscon2007
 
Os Bunce
oscon2007
 
Yuicss R7
oscon2007
 
Performance Whack A Mole
oscon2007
 
Os Fogel
oscon2007
 
Os Lanphier Brashears
oscon2007
 
Os Tucker
oscon2007
 
Os Fitzpatrick Sussman Swp
oscon2007
 
Os Furlong
oscon2007
 
Os Berlin Dispelling Myths
oscon2007
 
Os Kimsal
oscon2007
 
Os Pruett
oscon2007
 
Os Alrubaie
oscon2007
 
Os Keysholistic
oscon2007
 
Os Jonphillips
oscon2007
 
Os Urnerupdated
oscon2007
 

Recently uploaded (20)

PDF
KodekX | Application Modernization Development
KodekX
 
PPT
“AI and Expert System Decision Support & Business Intelligence Systems”
ZubinRadhakrishnan
 
PPTX
Big Data Technologies - Introduction.pptx
kuthubussaman1
 
PDF
NewMind AI Monthly Chronicles - July 2025
NewMind AI
 
PDF
How UI/UX Design Impacts User Retention in Mobile Apps.pdf
SynapseIndia
 
PDF
NewMind AI Weekly Chronicles - August'25 Week I
NewMind AI
 
PDF
Peak of Data & AI Encore- AI for Metadata and Smarter Workflows
Safe Software
 
PPTX
A Presentation on Artificial Intelligence
memembruh336
 
PDF
Network Security Unit 5.pdf for BCA BBA.
Serpent6
 
PDF
Reach Out and Touch Someone: Haptics and Empathic Computing
Mark Billinghurst
 
PDF
Approach and Philosophy of On baking technology
30anthuong17
 
PPTX
Effective Security Operations Center (SOC) A Modern, Strategic, and Threat-In...
ReZa AdineH
 
PDF
Shreyas Phanse Resume: Experienced Backend Engineer | Java • Spring Boot • Ka...
SHREYAS PHANSE
 
PDF
Chapter 3 Spatial Domain Image Processing.pdf
Getnet Tigabie Askale -(GM)
 
PDF
Review of recent advances in non-invasive hemoglobin estimation
IAESIJAI
 
PDF
Spectral efficient network and resource selection model in 5G networks
IAESIJAI
 
PDF
Per capita expenditure prediction using model stacking based on satellite ima...
IAESIJAI
 
PDF
Architecting across the Boundaries of two Complex Domains - Healthcare & Tech...
Peter Tan
 
PDF
Bridging biosciences and deep learning for revolutionary discoveries: a compr...
IAESIJAI
 
PDF
Agricultural_Statistics_at_a_Glance_2022_0.pdf
SandeepSingh286037
 
KodekX | Application Modernization Development
KodekX
 
“AI and Expert System Decision Support & Business Intelligence Systems”
ZubinRadhakrishnan
 
Big Data Technologies - Introduction.pptx
kuthubussaman1
 
NewMind AI Monthly Chronicles - July 2025
NewMind AI
 
How UI/UX Design Impacts User Retention in Mobile Apps.pdf
SynapseIndia
 
NewMind AI Weekly Chronicles - August'25 Week I
NewMind AI
 
Peak of Data & AI Encore- AI for Metadata and Smarter Workflows
Safe Software
 
A Presentation on Artificial Intelligence
memembruh336
 
Network Security Unit 5.pdf for BCA BBA.
Serpent6
 
Reach Out and Touch Someone: Haptics and Empathic Computing
Mark Billinghurst
 
Approach and Philosophy of On baking technology
30anthuong17
 
Effective Security Operations Center (SOC) A Modern, Strategic, and Threat-In...
ReZa AdineH
 
Shreyas Phanse Resume: Experienced Backend Engineer | Java • Spring Boot • Ka...
SHREYAS PHANSE
 
Chapter 3 Spatial Domain Image Processing.pdf
Getnet Tigabie Askale -(GM)
 
Review of recent advances in non-invasive hemoglobin estimation
IAESIJAI
 
Spectral efficient network and resource selection model in 5G networks
IAESIJAI
 
Per capita expenditure prediction using model stacking based on satellite ima...
IAESIJAI
 
Architecting across the Boundaries of two Complex Domains - Healthcare & Tech...
Peter Tan
 
Bridging biosciences and deep learning for revolutionary discoveries: a compr...
IAESIJAI
 
Agricultural_Statistics_at_a_Glance_2022_0.pdf
SandeepSingh286037