SlideShare a Scribd company logo
PAINFUL SUCCESS
Jesper Richter-Reichhelm (@jrirei) / session 4853
Monday, 18 March 13
InfoQ.com: News & Community Site
• 750,000 unique visitors/month
• Published in 4 languages (English, Chinese, Japanese and Brazilian
Portuguese)
• Post content from our QCon conferences
• News 15-20 / week
• Articles 3-4 / week
• Presentations (videos) 12-15 / week
• Interviews 2-3 / week
• Books 1 / month
Watch the video with slide
synchronization on InfoQ.com!
http://www.infoq.com/presentations
/scalability-case-study
Presented at QCon London
www.qconlondon.com
Purpose of QCon
- to empower software development by facilitating the spread of
knowledge and innovation
Strategy
- practitioner-driven conference designed for YOU: influencers of
change and innovation in your teams
- speakers and topics driving the evolution and innovation
- connecting and catalyzing the influencers and innovators
Highlights
- attended by more than 12,000 delegates since 2007
- held in 9 cities worldwide
Pain
Monday, 18 March 13
Monday, 18 March 13
Monday, 18 March 13
Monday, 18 March 13
Flash client Backend
Monday, 18 March 13
7M players / month
Monday, 18 March 13
7M players / month
10K API calls / second
Monday, 18 March 13
7M players / month
10K API calls / second
200K DB operations / second
Monday, 18 March 13
7M players / month
10K API calls / second
200K DB operations / second
100K DB writes / second
Monday, 18 March 13
7M players / month
10K API calls / second
200K DB operations / second
100K DB writes / second
2 devops since 2009
Monday, 18 March 13
Success
Monday, 18 March 13
Monday, 18 March 13
Monday, 18 March 13
Painful Success
Monday, 18 March 13
Painful Success
Monday, 18 March 13
2009
Decisions
Monday, 18 March 13
Monday, 18 March 13
Monday, 18 March 13
What is the right
software stack?
Monday, 18 March 13
Monday, 18 March 13
Monday, 18 March 13
Monday, 18 March 13
SQL
ap
lb
SQL
Browser
sql sql
apap
Monday, 18 March 13
2010
Scaling Up
Monday, 18 March 13
Monday, 18 March 13
Monday, 18 March 13
0%#
20%#
40%#
60%#
80%#
100%#
0# 5# 10# 15# 20# 25# 30# 35# 40#
Monday, 18 March 13
0%#
20%#
40%#
60%#
80%#
100%#
0# 5# 10# 15# 20# 25# 30# 35# 40#
Monday, 18 March 13
0%#
20%#
40%#
60%#
80%#
100%#
0# 5# 10# 15# 20# 25# 30# 35# 40#
Monday, 18 March 13
Small things...
Monday, 18 March 13
AMF responses
Monday, 18 March 13
Checking connection
‘status’
Monday, 18 March 13
ActiveRecord caching
Monday, 18 March 13
Always check back
on reality!
Lesson 1:
Monday, 18 March 13
Big Things...
Monday, 18 March 13
Lot’s of tuning
Monday, 18 March 13
Lot’s of tuning
and more memory
Monday, 18 March 13
app
slave
master
Monday, 18 March 13
app
slave
master master
slave
1. create new DBs
Monday, 18 March 13
app
slave
master master
slave
1. create new DBs
2. setup replication
Monday, 18 March 13
app
slave
master master
slave
1. create new DBs
2. setup replication
Monday, 18 March 13
app
slave
master master
slave
1. create new DBs
2. setup replication
3. start using
masters
Monday, 18 March 13
app
slave
master master
slave
1. create new DBs
2. setup replication
3. start using
masters
4. cut replication
Monday, 18 March 13
app
master
slave
1. create new DBs
2. setup replication
3. start using
masters
4. cut replication
Monday, 18 March 13
app
slave
master master
slave
1. create new DBs
2. setup replication
3. start using
masters
Monday, 18 March 13
app
slave
master master
slave
1. create new DBs
2. setup replication
3. start using
masters
4. remove old DBs
Monday, 18 March 13
app
master
slave
1. create new DBs
2. setup replication
3. start using
masters
4. remove old DBs
Monday, 18 March 13
app
slave
master master
slave
1. create new DBs
2. setup replication
3. start using
masters
Monday, 18 March 13
app
slave
master master
slave
1. create new DBs
2. setup replication
3. start using
masters
4. drop database;
Monday, 18 March 13
app
slave
master master
slave
1. create new DBs
2. setup replication
3. start using
masters
4. drop database;
Monday, 18 March 13
app
slave
master master
slave
1. create new DBs
2. setup replication
3. start using
masters
4. drop database;
Monday, 18 March 13
app
slave
master master
slave
1. create new DBs
2. setup replication
3. start using
masters
4. drop database;
Monday, 18 March 13
app
slave
master master
slave
1. create new DBs
2. setup replication
3. start using
masters
4. drop database;
Monday, 18 March 13
app
1. create new DBs
2. setup replication
3. start using
masters
4. drop database;
Monday, 18 March 13
app
1. create new DBs
2. setup replication
3. start using
masters
4. drop database;
Monday, 18 March 13
You will make
mistakes!
Lesson 2:
Monday, 18 March 13
More things...
Monday, 18 March 13
app
slave
master master
slave
1. create new DBs
2. setup replication
3. start using
masters
4. cut replication
Monday, 18 March 13
app
slave
master master
slave
1. create new DBs
2. setup replication
3. start using
masters
4. cut replication
AB
AB
AB
AB
Monday, 18 March 13
app
slave
master master
slave
1. create new DBs
2. setup replication
3. start using
masters
BA
AB
AB
AB
AB
Monday, 18 March 13
app
slave
master master
slave
1. create new DBs
2. setup replication
3. start using
masters
4. cut replication
BA
AB
AB
AB
AB
Monday, 18 March 13
app
slave
master master
slave
1. create new DBs
2. setup replication
3. start using
masters
4. cut replication
BA
AB
AB
AB
AB
Monday, 18 March 13
app
slave
master master
slave slave
master master
slave
AC
AC
BD
BD
Monday, 18 March 13
app
slave
master master
slave slave
master master
slave
AC
AC
BD
BD
AC
AC
BD
BD
Monday, 18 March 13
app
slave
master master
slave slave
master master
slave
AC
AC
BD
BD
AC
AC
BD
BD
Monday, 18 March 13
app
slave
master master
slave slave
master master
slave
AC
AC
BD
BD
AC
AC
BD
BD
Monday, 18 March 13
The right thing...
Monday, 18 March 13
Monday, 18 March 13
Monday, 18 March 13
Monday, 18 March 13
Software is easy,
data is hard!
Lesson 3:
Monday, 18 March 13
24 GB
44 GB
in 8 days
Monday, 18 March 13
Not enough memory
Monday, 18 March 13
Not enough memory
=> no backups
Monday, 18 March 13
Not enough memory
=> no backups
=> no new slaves
Monday, 18 March 13
Not enough memory
=> no backups
=> no new slaves
=> big problem
Monday, 18 March 13
24 GB
38 GB
in 3 days
Monday, 18 March 13
24 GB
38 GB
in 3 days
Fixed in v2.2
Monday, 18 March 13
2011
Operation Hell
Monday, 18 March 13
SQL SQL
sql sql
SQL SQL
sql sql
SQL SQL
sql
Rd Rd
rd rd
Rd Rd
rd rd
Rd Rd
rd rd
Rd Rd
rd rd
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
lb lblblb
Monday, 18 March 13
SQL SQL
sql sql
SQL SQL
sql sql
SQL SQL
sql
Rd Rd
rd rd
Rd Rd
rd rd
Rd Rd
rd rd
Rd Rd
rd rd
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
lb lblblb
Monday, 18 March 13
SQL SQL
sql sql
SQL SQL
sql sql
SQL SQL
sql
Rd Rd
rd rd
Rd Rd
rd rd
Rd Rd
rd rd
Rd Rd
rd rd
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
SQL
EBS EBS EBS EBS
lb lblblb
Monday, 18 March 13
EBSSQLBrowser
lb
lb
lb
lb
ap
ap
ap
ap
ap
ap
ap ap
Monday, 18 March 13
EBSSQLBrowser
lb
lb
lb
lb
ap
ap
ap
ap
ap
ap
ap ap
Monday, 18 March 13
EBSSQLBrowser
lb
lb
lb
lb
ap
ap
ap
ap
ap
ap
ap ap
Monday, 18 March 13
EBSSQLBrowser
lb
lb
lb
lb
ap
ap
ap
ap
ap
ap
ap ap
Monday, 18 March 13
EBSSQLBrowser
lb
lb
lb
lb
ap
ap
ap
ap
ap
ap
ap ap
Monday, 18 March 13
EBSSQLBrowser
lb
lb
lb
lb
ap
ap
ap
ap
ap
ap
ap ap
Monday, 18 March 13
EBSSQLBrowser
lb
lb
lb
lb
ap
ap
ap
ap
ap
ap
ap ap
Monday, 18 March 13
EBSSQLBrowser
lb
lb
lb
lb
ap
ap
ap
ap
ap
ap
ap ap
Monday, 18 March 13
0"
500,000"
1,000,000"
1,500,000"
2,000,000"
Apr*10" Jul*10" Oct*10" Jan*11" Apr*11" Jul*11" Oct*11"
Monday, 18 March 13
0"
500,000"
1,000,000"
1,500,000"
2,000,000"
Apr*10" Jul*10" Oct*10" Jan*11" Apr*11" Jul*11" Oct*11"
AWS outage
in Ireland
Monday, 18 March 13
http://en.wikipedia.org/wiki/File:St%C3%B6wer_Titanic.jpg
Monday, 18 March 13
SQL SQL
sql sql
SQL SQL
sql sql
SQL SQL
sql
Rd Rd
rd rd
Rd Rd
rd rd
Rd Rd
rd rd
Rd Rd
rd rd
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
lb lblblb
Monday, 18 March 13
SQL SQL
sql sql
SQL SQL
sql sql
SQL SQL
sql
Rd Rd
rd rd
Rd Rd
rd rd
Rd Rd
rd rd
Rd Rd
rd rd
SQL SQL SQL SQL SQL
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
lb lblblb
Monday, 18 March 13
Monday, 18 March 13
Installed in 3 years:
Monday, 18 March 13
Installed in 3 years:
98 MySQL machines
Monday, 18 March 13
Installed in 3 years:
98 MySQL machines
195 Redis machines
Monday, 18 March 13
Monday, 18 March 13
Monday, 18 March 13
Monday, 18 March 13
Monday, 18 March 13
2012/2013
Refactoring
Monday, 18 March 13
Lessons learned
Monday, 18 March 13
Server Database
Monday, 18 March 13
Server Database
Monday, 18 March 13
Server Database
Monday, 18 March 13
Server Database
Monday, 18 March 13
Server Database
Monday, 18 March 13
Server Database
Monday, 18 March 13
Server Database
Monday, 18 March 13
Server Database
Monday, 18 March 13
Server Database
Monday, 18 March 13
Server Database
One Game Session
Monday, 18 March 13
Server Database
One Game Session
Monday, 18 March 13
Rd
rd
SQL SQL
sql sql
SQL SQL
sql sql
SQL SQL
sql
Rd Rd
rd rd
Rd Rd
rd rd
Rd
rd
Rd Rd
rd rd
SQL SQL SQL SQL SQL
ap apap
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap apap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
lb lblblb
Monday, 18 March 13
Rd
rd
ap apap
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap apap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
lb lblblb
Monday, 18 March 13
Rd
rd
ap apap
lb lblblb
Monday, 18 March 13
Monday, 18 March 13
1. Separate user and world DBs
Monday, 18 March 13
1. Separate user and world DBs
2. Migrate to JRuby
Monday, 18 March 13
1. Separate user and world DBs
2. Migrate to JRuby
3. Introduce life cycle
Monday, 18 March 13
1. Separate user and world DBs
2. Migrate to JRuby
3. Introduce life cycle
4. Remove (user) DBs
Monday, 18 March 13
1. Separate user and world DBs
2. Migrate to JRuby
3. Introduce life cycle
4. Remove (user) DBs
5. Enjoy!
Monday, 18 March 13
Separate user DBs
Monday, 18 March 13
SQL SQL
sql sql
SQL SQL
sql sql
SQL SQL
sql
Rd Rd
rd rd
Rd Rd
rd rd
Rd Rd
rd rd
Rd Rd
rd rd
SQL SQL SQL SQL
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
Rd
rd
lb lblblb
Monday, 18 March 13
SQL SQL
sql
Rd Rd
rd rd
Rd Rd
rd rd
Rd Rd
rd rd
Rd Rd
rd rd
SQL SQL SQL SQL
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
Rd
rd
SQL SQL
sql sql
SQL SQL
sql sql
SQL
sql
lb lblblb
Monday, 18 March 13
SQL SQL
sql sql
SQL SQL
sql sql
SQL SQL
sql
Rd Rd
rd rd
Rd Rd
rd rd
Rd Rd
rd rd
Rd Rd
rd rd
SQL SQL SQL SQL
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
Rd
rd
SQL
sql
SQL
lb lblblb
Monday, 18 March 13
SQL SQL
sql sql
SQL SQL
sql sql
SQL SQL
sql
Rd Rd
rd rd
Rd Rd
rd rd
Rd Rd
rd rd
Rd Rd
rd rd
SQL SQL SQL SQL
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
Rd
rd
SQL
sql
SQL
Rd
rd
Rd
rd
lb lblblb
Monday, 18 March 13
JRuby
Monday, 18 March 13
http://www.flickr.com/photos/biblicone/3425903181/sizes/l/in/photostream/
Monday, 18 March 13
JRuby v2
Monday, 18 March 13
0
50
100
150
200
Rails 2.3.11
MRI 1.8.7
Throughput
Monday, 18 March 13
0
50
100
150
200
Rails 2.3.11
MRI 1.8.7
Rails 3.2.8
MRI 1.8.7
Throughput
Monday, 18 March 13
0
50
100
150
200
Rails 2.3.11
MRI 1.8.7
Rails 3.2.8
MRI 1.8.7
Rails 3.2.8
MRI 1.9.3
Throughput
Monday, 18 March 13
0
50
100
150
200
Rails 2.3.11
MRI 1.8.7
Rails 3.2.8
MRI 1.8.7
Rails 3.2.8
MRI 1.9.3
Rails 3.2.8
JRuby 1.7.2
Throughput
Monday, 18 March 13
0
50
100
150
200
Rails 2.3.11
MRI 1.8.7
Rails 3.2.8
MRI 1.8.7
Rails 3.2.8
MRI 1.9.3
Rails 3.2.8
JRuby 1.7.2
Throughput
no benchmark
Monday, 18 March 13
0
50
100
150
200
Rails 2.3.11
MRI 1.8.7
Rails 3.2.8
MRI 1.8.7
Rails 3.2.8
MRI 1.9.3
Rails 3.2.8
JRuby 1.7.2
Throughput
no benchmark
Monday, 18 March 13
0
50
100
150
200
Rails 2.3.11
MRI 1.8.7
Rails 3.2.8
MRI 1.8.7
Rails 3.2.8
MRI 1.9.3
Rails 3.2.8
JRuby 1.7.2
Throughput
no benchmark
multi
threaded
Monday, 18 March 13
0
50
100
150
200
Rails 2.3.11
MRI 1.8.7
Rails 3.2.8
MRI 1.8.7
Rails 3.2.8
MRI 1.9.3
Rails 3.2.8
JRuby 1.7.2
Throughput
no benchmark
multi
threaded
tuning
Monday, 18 March 13
0
50
100
150
200
Rails 2.3.11
MRI 1.8.7
Rails 3.2.8
MRI 1.8.7
Rails 3.2.8
MRI 1.9.3
Rails 3.2.8
JRuby 1.7.2
Throughput
no benchmark
multi
threaded
tuning
cpu
saturation
Monday, 18 March 13
SQL SQL
sql sql
SQL SQL
sql sql
SQL SQL
sql
Rd Rd
rd rd
Rd Rd
rd rd
Rd Rd
rd rd
Rd Rd
rd rd
SQL SQL SQL SQL SQL
ap apap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap apap ap ap
ap ap
ap ap
ap ap
ap ap
ap ap
ap ap
ap ap
ap ap ap ap ap ap ap ap ap apap ap ap
lb lblblb
Monday, 18 March 13
SQL SQL
sql sql
SQL SQL
sql sql
SQL SQL
sql
Rd Rd
rd rd
Rd Rd
rd rd
Rd Rd
rd rd
Rd Rd
rd rd
SQL SQL SQL SQL SQL
ap apap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap apap ap ap
lb lblblb
ap
ap
ap
ap
ap
ap
ap
ap
ap
ap
ap
ap
ap
ap
Monday, 18 March 13
Monday, 18 March 13
Monday, 18 March 13
Monday, 18 March 13
NEW OLD
Monday, 18 March 13
NEW OLD
Monday, 18 March 13
NEW OLD
Monday, 18 March 13
NEW OLD
Monday, 18 March 13
Life cycle
Monday, 18 March 13
app
Monday, 18 March 13
app
Monday, 18 March 13
app
servlet
Monday, 18 March 13
app
servlet archiver
Monday, 18 March 13
User
Registry
app
servlet archiver
Monday, 18 March 13
User
Registry
S3
app
servlet archiver
Monday, 18 March 13
User
Registry
SQL Rd
S3
app
servlet archiver
Monday, 18 March 13
User
Registry
SQL Rd
S3
app
servlet archiver
Monday, 18 March 13
World
Data
User
Registry
SQL Rd
S3
app
servlet archiver
Monday, 18 March 13
World
Data
User
Registry
SQL Rd
S3
Load Balancer
app
servlet archiver
Monday, 18 March 13
SQL SQL
sql sql
SQL SQL
sql sql
SQL SQL
sql
Rd Rd
rd rd
Rd Rd
rd rd
Rd Rd
rd rd
Rd
rd
Rd
rd
SQL SQL SQL SQL
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
Rd
rd
SQL
sql
SQL
Rd
rd
Rd
rd
lb lblblb
Monday, 18 March 13
SQL SQL
sql sql
SQL SQL
sql sql
SQL SQL
sql
Rd Rd
rd rd
Rd Rd
rd rd
Rd Rd
rd rd
Rd
rd
Rd
rd
SQL SQL SQL SQL
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
Rd
rd
SQL
sql
SQL
Rd
rd
Rd
rd
Rd
rd
lb lblblb
Monday, 18 March 13
Removing DBs
Monday, 18 March 13
SQL SQL
sql
Rd
rd
Rd
rd
Rd
rd
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
Rd
rd
SQL
sql
SQLSQL SQL
sql sql
SQL SQL
sql sql
Rd Rd
rd rd
Rd Rd
rd rd
Rd
rd
SQL SQL SQL SQL
Rd
rd
Rd
rd
Rd
rd
lb lblblb
Monday, 18 March 13
SQL SQL
sql
Rd
rd
Rd
rd
Rd
rd
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
Rd
rd
SQL
sql
SQL
Rd
rd
Rd
rd
lb lblblb
Monday, 18 March 13
SQL
sql
Rd
rd
Rd
rd
Rd
rd
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
Rd
rd
SQL
sql
SQL
Rd
rd
Rd
rd
lb lblblb
Monday, 18 March 13
Enjoy
Monday, 18 March 13
SQL
sql
Rd
rd
Rd
rd
Rd
rd
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
Rd
rd
SQL
sql
SQL
Rd
rd
Rd
rd
lb lblblb
Monday, 18 March 13
Rd
rd
Rd
rd
Rd
rd
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
Rd
rd
SQL
sql
SQL
Rd
rd
Rd
rd
lb lblblb
Monday, 18 March 13
Rd
rd
Rd
rd
Rd
rd
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
ap apap ap ap ap ap ap ap ap ap apap ap ap
Rd
rd
Rd
rd
Rd
rd
lb lblblb
Monday, 18 March 13
http://www.flickr.com/photos/aigle_dore/
Monday, 18 March 13
http://www.flickr.com/photos/aigle_dore/
Monday, 18 March 13
Looking Back
Monday, 18 March 13
What is the right
software stack?
Monday, 18 March 13
Monday, 18 March 13
Monday, 18 March 13
What is the right
software stack?
Monday, 18 March 13
What is the right
software stack?
Monday, 18 March 13
What kind of application
are we building?
Monday, 18 March 13
User
Monday, 18 March 13
User
Avatar
Monday, 18 March 13
User
Avatar
Cus-
tomer
Monday, 18 March 13
User
GardenAvatar
Cus-
tomer
Monday, 18 March 13
User
GardenAvatar
Cus-
tomer
Tiles
Deco-
rations
Monday, 18 March 13
User
GardenAvatar
Cus-
tomer
Tiles
Deco-
rations
Monday, 18 March 13
User
GardenAvatar
Cus-
tomer
Tiles
Deco-
rations
User
GardenAvatar
Cus-
tomer
Tiles
Deco-
rations
Monday, 18 March 13
User
GardenAvatar
Cus-
tomer
Tiles
Deco-
rations
User
GardenAvatar
Cus-
tomer
Tiles
Deco-
rations
Monday, 18 March 13
http://en.wikipedia.org/wiki/File:Columbus_Breaking_the_Egg%27_(Christopher_Columbus)_by_William_Hogarth.jpg
Monday, 18 March 13
http://en.wikipedia.org/wiki/File:Columbus_Breaking_the_Egg%27_(Christopher_Columbus)_by_William_Hogarth.jpg
Monday, 18 March 13
http://en.wikipedia.org/wiki/File:Columbus_Breaking_the_Egg%27_(Christopher_Columbus)_by_William_Hogarth.jpg
Monday, 18 March 13
http://en.wikipedia.org/wiki/File:Columbus_Breaking_the_Egg%27_(Christopher_Columbus)_by_William_Hogarth.jpg
Monday, 18 March 13
http://en.wikipedia.org/wiki/File:Columbus_Breaking_the_Egg%27_(Christopher_Columbus)_by_William_Hogarth.jpg
Monday, 18 March 13
What kind of application
are we building?
Monday, 18 March 13
Monday, 18 March 13
Monday, 18 March 13
Monday, 18 March 13
Monday, 18 March 13
Monday, 18 March 13
Plain File System
Monday, 18 March 13
Monday, 18 March 13
Handle state
the right way!
Monday, 18 March 13
Software is easy,
data is hard!
Lesson 3:
Monday, 18 March 13
You will make
mistakes!
Lesson 2:
Monday, 18 March 13
Always check back
on reality!
Lesson 1:
Monday, 18 March 13
Always check back
on your assumptions!
Lesson 1:
Monday, 18 March 13
Thank you
Monday, 18 March 13
Jesper Richter-Reichhelm
@jrirei
woo.ga/backend
wooga.com/jobs
Monday, 18 March 13
Jesper Richter-
Reichhelm
@jrirei
wooga.com/jobs
woo.ga/backend
Monday, 18 March 13
Watch the video with slide synchronization on
InfoQ.com!
http://www.infoq.com/presentations/scalability
-case-study

More Related Content

PDF
BigData and Algorithms - LA Algorithmic Trading
PDF
Who's afraid of the big bad loop?
PDF
Containerization Is More than the New Virtualization
PDF
Games for the Masses (QCon London 2012)
PDF
javazone 2014
PDF
Becoming a Rock Star DBA
PDF
KYSUC - Keep Your Schema Under Control
PDF
Growing up new PostgreSQL developers (pgcon.org 2018)
BigData and Algorithms - LA Algorithmic Trading
Who's afraid of the big bad loop?
Containerization Is More than the New Virtualization
Games for the Masses (QCon London 2012)
javazone 2014
Becoming a Rock Star DBA
KYSUC - Keep Your Schema Under Control
Growing up new PostgreSQL developers (pgcon.org 2018)

Similar to Painful Success - Lessons Learned while Scaling Up (20)

PDF
Devoxx UK: Reliability & Scale in AWS while letting you sleep through the night
PPTX
Defy the Defaults
PPT
Db trends final
PDF
10x Performance Improvements - A Case Study
PDF
Agile Operations or How to sleep better at night
PDF
Games for the Masses (Jax)
PDF
Qcon talk
PDF
Database Management Systems 3rd Edition Raghu Ramakrishnan
PDF
Advanced Deployment
PPTX
Sql server infernals
PDF
Data antipatterns NYC Devops - 2014
PPTX
PPTX
Database Industry perspective
KEY
10x improvement-mysql-100419105218-phpapp02
KEY
10x Performance Improvements
PDF
PostgreSQL worst practices, version FOSDEM PGDay 2017 by Ilya Kosmodemiansky
PDF
When Devs Do Ops
PDF
Couche Base par Tugdual Grall
PPT
Lecture-1.ppt
PPTX
SQLBits VI - Improving database performance by removing the database
Devoxx UK: Reliability & Scale in AWS while letting you sleep through the night
Defy the Defaults
Db trends final
10x Performance Improvements - A Case Study
Agile Operations or How to sleep better at night
Games for the Masses (Jax)
Qcon talk
Database Management Systems 3rd Edition Raghu Ramakrishnan
Advanced Deployment
Sql server infernals
Data antipatterns NYC Devops - 2014
Database Industry perspective
10x improvement-mysql-100419105218-phpapp02
10x Performance Improvements
PostgreSQL worst practices, version FOSDEM PGDay 2017 by Ilya Kosmodemiansky
When Devs Do Ops
Couche Base par Tugdual Grall
Lecture-1.ppt
SQLBits VI - Improving database performance by removing the database
Ad

More from C4Media (20)

PDF
Streaming a Million Likes/Second: Real-Time Interactions on Live Video
PDF
Next Generation Client APIs in Envoy Mobile
PDF
Software Teams and Teamwork Trends Report Q1 2020
PDF
Understand the Trade-offs Using Compilers for Java Applications
PDF
Kafka Needs No Keeper
PDF
High Performing Teams Act Like Owners
PDF
Does Java Need Inline Types? What Project Valhalla Can Bring to Java
PDF
Service Meshes- The Ultimate Guide
PDF
Shifting Left with Cloud Native CI/CD
PDF
CI/CD for Machine Learning
PDF
Fault Tolerance at Speed
PDF
Architectures That Scale Deep - Regaining Control in Deep Systems
PDF
ML in the Browser: Interactive Experiences with Tensorflow.js
PDF
Build Your Own WebAssembly Compiler
PDF
User & Device Identity for Microservices @ Netflix Scale
PDF
Scaling Patterns for Netflix's Edge
PDF
Make Your Electron App Feel at Home Everywhere
PDF
The Talk You've Been Await-ing For
PDF
Future of Data Engineering
PDF
Automated Testing for Terraform, Docker, Packer, Kubernetes, and More
Streaming a Million Likes/Second: Real-Time Interactions on Live Video
Next Generation Client APIs in Envoy Mobile
Software Teams and Teamwork Trends Report Q1 2020
Understand the Trade-offs Using Compilers for Java Applications
Kafka Needs No Keeper
High Performing Teams Act Like Owners
Does Java Need Inline Types? What Project Valhalla Can Bring to Java
Service Meshes- The Ultimate Guide
Shifting Left with Cloud Native CI/CD
CI/CD for Machine Learning
Fault Tolerance at Speed
Architectures That Scale Deep - Regaining Control in Deep Systems
ML in the Browser: Interactive Experiences with Tensorflow.js
Build Your Own WebAssembly Compiler
User & Device Identity for Microservices @ Netflix Scale
Scaling Patterns for Netflix's Edge
Make Your Electron App Feel at Home Everywhere
The Talk You've Been Await-ing For
Future of Data Engineering
Automated Testing for Terraform, Docker, Packer, Kubernetes, and More
Ad

Recently uploaded (20)

PPTX
KOM of Painting work and Equipment Insulation REV00 update 25-dec.pptx
PDF
Encapsulation_ Review paper, used for researhc scholars
PPTX
Effective Security Operations Center (SOC) A Modern, Strategic, and Threat-In...
PDF
cuic standard and advanced reporting.pdf
PDF
7 ChatGPT Prompts to Help You Define Your Ideal Customer Profile.pdf
PPTX
ACSFv1EN-58255 AWS Academy Cloud Security Foundations.pptx
PPTX
Detection-First SIEM: Rule Types, Dashboards, and Threat-Informed Strategy
PDF
Electronic commerce courselecture one. Pdf
PDF
Agricultural_Statistics_at_a_Glance_2022_0.pdf
PDF
KodekX | Application Modernization Development
PDF
Unlocking AI with Model Context Protocol (MCP)
PPTX
VMware vSphere Foundation How to Sell Presentation-Ver1.4-2-14-2024.pptx
PDF
Dropbox Q2 2025 Financial Results & Investor Presentation
PDF
Network Security Unit 5.pdf for BCA BBA.
PPTX
Spectroscopy.pptx food analysis technology
PPTX
MYSQL Presentation for SQL database connectivity
PDF
Empathic Computing: Creating Shared Understanding
PPTX
Understanding_Digital_Forensics_Presentation.pptx
PDF
How UI/UX Design Impacts User Retention in Mobile Apps.pdf
PDF
Build a system with the filesystem maintained by OSTree @ COSCUP 2025
KOM of Painting work and Equipment Insulation REV00 update 25-dec.pptx
Encapsulation_ Review paper, used for researhc scholars
Effective Security Operations Center (SOC) A Modern, Strategic, and Threat-In...
cuic standard and advanced reporting.pdf
7 ChatGPT Prompts to Help You Define Your Ideal Customer Profile.pdf
ACSFv1EN-58255 AWS Academy Cloud Security Foundations.pptx
Detection-First SIEM: Rule Types, Dashboards, and Threat-Informed Strategy
Electronic commerce courselecture one. Pdf
Agricultural_Statistics_at_a_Glance_2022_0.pdf
KodekX | Application Modernization Development
Unlocking AI with Model Context Protocol (MCP)
VMware vSphere Foundation How to Sell Presentation-Ver1.4-2-14-2024.pptx
Dropbox Q2 2025 Financial Results & Investor Presentation
Network Security Unit 5.pdf for BCA BBA.
Spectroscopy.pptx food analysis technology
MYSQL Presentation for SQL database connectivity
Empathic Computing: Creating Shared Understanding
Understanding_Digital_Forensics_Presentation.pptx
How UI/UX Design Impacts User Retention in Mobile Apps.pdf
Build a system with the filesystem maintained by OSTree @ COSCUP 2025

Painful Success - Lessons Learned while Scaling Up