SlideShare a Scribd company logo
An Approach to
Automated Application Tuning
Mike Huang, Sr. Performance Architect
Performance and Reliability Engineering
UNNECESSARY EXPOSITION
Prologue.
Copy What
Someone
Else Is
Doing
Tweak
Until
Things
“Work”
Forget
About It
Something
Changes
Something
Breaks
Tuning. Ugh.
Why?
● Insufficient defaults
● One-size-fits-all myth
● Unfamiliar technology
● Complex interactions
● Time
Nirvana
●Optimal
●Adaptive
●Automatic
Construct a Model
1. Frame the problem
2. Understand interactions
3. Define success
4. Narrow focus
5. Validate data and behavior
6. Refine
ORIGIN STORY
Scene
one.
An Approach to Automated Application Tuning
Spilled Milk
Goal
Improve resiliency to downstream failures
UNLOCKING MYSTERIES
Scene
two.
Dependency Framework
Timeouts
Fallbacks
Rejections
Short-circuits
Timeouts
Retries
Pooling
Blacklists
Understanding Interactions
Hystrix
Service B
EVCache
EVC Client
Ribbon
Ribbon
/all/the/movies
Service A
Hystrix
EVC Client
/moar/movies
Hystrix
Hystrix
Hystrix
Understanding Interactions
Hystrix
Service B
EVCache
EVC Client
Ribbon
Ribbon
/all/the/movies
Service A
Hystrix
EVC Client
/moar/movies
Hystrix Workflow
Define Success
Under normal operations:
● Expire only the most egregious requests
● No concurrency constraint
● No premature fallback
Under duress:
● Fail fast!
Assumptions
● Spectrum of desirable behavior
● You can’t win them all
● It’s okay to fail
Key Tunables
● Timeout
● Error Threshold
● Isolation Strategy
● Thread Pool Size
● Queue Size
● Batch Size
THE TWIST
Scene
three.
True Lies
● Recommendations are only
as good as the data
● Data is only good if you
understand it
Things Are Not Always As They Seem
Pick a Number, Any Number
How Low Can You Go?
THE FINAL ACT
Scene
four.
Automation? Piece of Cake
Evaluate
RecommendApply
The Cake Is a Lie!
Questions?
mhuang@netflix.com
@gnauHekiM
GO HOME – THERE ARE NO STINGERS HERE
Credits.
Acknowldgements (Please Don’t Sue Me)
Slide:
 4 – Parks and Recreation (Season 2, Episode 4)
 5 – http://www.musiclipse.com/wp-content/uploads/2013/09/nirvana-hd-photo-band-members.jpg
 6 (C) – https://rtfitch.files.wordpress.com/2014/08/bojack-horseman.jpg
 6 (BoJack) – http://www.serieslyawesome.tv/wp-content/uploads/2015/05/BoJack-Horseman-Season-2.jpg
 9 (Drew Barrymore) – The Wedding Singer
 9 (Angel of Grief) – https://en.wikipedia.org/wiki/Angel_of_Grief#/media/File:Emelyn_Story_Tomba_(Cimitero_Acattolico_Roma).jpg
 9 (Angry Cat) – http://theeconomiccollapseblog.com/wp-content/uploads/2011/01/Angry-Cat.jpg
 12 (Ribbon logo) – http://3.bp.blogspot.com/-8OQzOCw1rXY/VdE2mxvnybI/AAAAAAAABoE/ARq3Pa4v9yI/s1600/ribbon.png
 12, 18 (Hystrix logo) – https://raw.githubusercontent.com/wiki/Netflix/Hystrix/images/hystrix-logo-tagline-640.png
 15 – https://github.com/Netflix/Hystrix/wiki/How-it-Works/images/hystrix-flow-chart-original.png
 20 – https://fanart.tv/fanart/movies/36955/moviebackground/true-lies-51036cf5a0251.jpg
 25 – http://www.fmvmagazine.com/wp-content/uploads/2012/07/Portal-cake.jpg
 26 – http://theportalwiki.com/wiki/File:The_bots.png
 28 – Mike Huang

More Related Content

PDF
5-Whys Method
KEY
Agile xp crash_course_2010_05_21
PDF
Please don't test your product - Agile Testing
PPT
The Smells Of Bad Design
PDF
Automated Agility?! Let's Talk Truly Agile Testing - Adam Howard - AgileNZ 2017
PPT
Scrum And The Enterprise
PDF
Test Automation in Agile: A Successful Implementation
PPTX
Why unvalidated assumption is the enemy of good product
5-Whys Method
Agile xp crash_course_2010_05_21
Please don't test your product - Agile Testing
The Smells Of Bad Design
Automated Agility?! Let's Talk Truly Agile Testing - Adam Howard - AgileNZ 2017
Scrum And The Enterprise
Test Automation in Agile: A Successful Implementation
Why unvalidated assumption is the enemy of good product

What's hot (20)

PDF
Test Driven Development
PPT
Kens Scrum Presentation
PPTX
What if you could eliminate the hidden costs of development?
PDF
ONBOARDING AT SCALE AT BOOKING.COM
ODP
Automating good coding practices
PPTX
Unit testing for project managers
PDF
Failfast
PPT
Agile testing
PPT
User Stories
PDF
STLDODN - Agile Testing in a Waterfall World
PPT
'The Real Agile Testing Quadrants' with Michael Bolton
PPTX
One trunk one pipeline one truth
PPTX
Performance testing mistakes newbies make
PDF
Chicago Code Camp 2014 - Agile Testing in a waterfall world
PDF
Building Better Products (for SpeedUp! Europe)
PDF
“High performance teams" - Marco Cecconi
PPTX
Quality Assurance: What is it and what are the Business Benefits?
PPTX
Quality Assurance - SQLSatBR presentation
PDF
Seven Bad Habits to Avoid As a QA Engineer
PDF
Scaling CTO / On Freund
Test Driven Development
Kens Scrum Presentation
What if you could eliminate the hidden costs of development?
ONBOARDING AT SCALE AT BOOKING.COM
Automating good coding practices
Unit testing for project managers
Failfast
Agile testing
User Stories
STLDODN - Agile Testing in a Waterfall World
'The Real Agile Testing Quadrants' with Michael Bolton
One trunk one pipeline one truth
Performance testing mistakes newbies make
Chicago Code Camp 2014 - Agile Testing in a waterfall world
Building Better Products (for SpeedUp! Europe)
“High performance teams" - Marco Cecconi
Quality Assurance: What is it and what are the Business Benefits?
Quality Assurance - SQLSatBR presentation
Seven Bad Habits to Avoid As a QA Engineer
Scaling CTO / On Freund
Ad

Viewers also liked (20)

PPT
IUA 2001 Creative Techniques for Application Tuning
PPTX
Profiling and Tuning a Web Application - The Dirty Details
PDF
Application Performance Tuning Basic (Paul S.J.Kim)
PPTX
PDF
บริการ
PPTX
The HathiTrust Research Center: Enabling New Knowledge Through Shared Infras...
PDF
Delivering scalable and high performance BI with least IT effort
PDF
Presentation1
PPTX
How Social Turns Your Employees Into A Secret PR Army
PPT
Panel Session: VIVO and the data culture of universities-VIVO@IU
PPTX
Produire un webdoc et respecter le droit
 
PPTX
PDF
WE ARE YOUR FUTURE -by @agalorda
PPT
Herpes zoster infection in neurology: management updates and prevention
PDF
The Cloud Native Journey
PDF
What makes professors inspiring
PPTX
Hemodialysis procedure
PPTX
Congenital anomalies of upper limb
PPTX
Using Hystrix to Build Resilient Distributed Systems
PDF
Young Dreamer
IUA 2001 Creative Techniques for Application Tuning
Profiling and Tuning a Web Application - The Dirty Details
Application Performance Tuning Basic (Paul S.J.Kim)
บริการ
The HathiTrust Research Center: Enabling New Knowledge Through Shared Infras...
Delivering scalable and high performance BI with least IT effort
Presentation1
How Social Turns Your Employees Into A Secret PR Army
Panel Session: VIVO and the data culture of universities-VIVO@IU
Produire un webdoc et respecter le droit
 
WE ARE YOUR FUTURE -by @agalorda
Herpes zoster infection in neurology: management updates and prevention
The Cloud Native Journey
What makes professors inspiring
Hemodialysis procedure
Congenital anomalies of upper limb
Using Hystrix to Build Resilient Distributed Systems
Young Dreamer
Ad

Similar to An Approach to Automated Application Tuning (20)

PDF
Chaos Engineering
PDF
Chaos Engineering: Injecting Failure for Building Resilience in Systems
PDF
Chaos Engineering - The Art of Breaking Things in Production
PDF
DevOps in Practice: When does "Practice" Become "Doing"?
PDF
Reliability as a Discipline
PPTX
WinOps Conf 2016 - Gael Colas - Configuration Management Theory: Why Idempote...
PPTX
Agile Testing Dilemmas
PDF
Architectural Patterns of Resilient Distributed Systems
PPTX
Dependency Injection in .NET applications
PPTX
Things To Unlearn In Software Development
PPTX
SF Chaos Engineering Meetup: Building Disaster Recovery via Resilience Engine...
PDF
Comments for Simulations Part III.pdf
PDF
Chaos Engineering – why we should all practice breaking things on purpose by ...
PPTX
Carrenza at Government ICT 2016 / The advantages of highly automated clouds
PDF
From Monoliths to Microservices at Realestate.com.au
PPTX
Cloud design pattern using azure
PPTX
JavaOne 2016 "Java, Microservices, Cloud and Containers"
PPTX
Simplexity Smart Goods workshop presentation rev3
PDF
Faster apps. faster time to market. faster mean time to repair
PDF
EclipseCon Eu 2015 - Breathe life into your Designer!
Chaos Engineering
Chaos Engineering: Injecting Failure for Building Resilience in Systems
Chaos Engineering - The Art of Breaking Things in Production
DevOps in Practice: When does "Practice" Become "Doing"?
Reliability as a Discipline
WinOps Conf 2016 - Gael Colas - Configuration Management Theory: Why Idempote...
Agile Testing Dilemmas
Architectural Patterns of Resilient Distributed Systems
Dependency Injection in .NET applications
Things To Unlearn In Software Development
SF Chaos Engineering Meetup: Building Disaster Recovery via Resilience Engine...
Comments for Simulations Part III.pdf
Chaos Engineering – why we should all practice breaking things on purpose by ...
Carrenza at Government ICT 2016 / The advantages of highly automated clouds
From Monoliths to Microservices at Realestate.com.au
Cloud design pattern using azure
JavaOne 2016 "Java, Microservices, Cloud and Containers"
Simplexity Smart Goods workshop presentation rev3
Faster apps. faster time to market. faster mean time to repair
EclipseCon Eu 2015 - Breathe life into your Designer!

Recently uploaded (20)

PDF
Design an Analysis of Algorithms II-SECS-1021-03
PDF
How to Choose the Right IT Partner for Your Business in Malaysia
PPTX
Reimagine Home Health with the Power of Agentic AI​
PPTX
Introduction to Artificial Intelligence
PDF
System and Network Administraation Chapter 3
PPTX
VVF-Customer-Presentation2025-Ver1.9.pptx
PDF
Digital Systems & Binary Numbers (comprehensive )
PDF
PTS Company Brochure 2025 (1).pdf.......
PDF
Raksha Bandhan Grocery Pricing Trends in India 2025.pdf
PDF
wealthsignaloriginal-com-DS-text-... (1).pdf
PDF
Softaken Excel to vCard Converter Software.pdf
PDF
Claude Code: Everyone is a 10x Developer - A Comprehensive AI-Powered CLI Tool
PDF
2025 Textile ERP Trends: SAP, Odoo & Oracle
PDF
EN-Survey-Report-SAP-LeanIX-EA-Insights-2025.pdf
PDF
Designing Intelligence for the Shop Floor.pdf
PDF
top salesforce developer skills in 2025.pdf
PDF
SAP S4 Hana Brochure 3 (PTS SYSTEMS AND SOLUTIONS)
PDF
Adobe Illustrator 28.6 Crack My Vision of Vector Design
PPTX
Operating system designcfffgfgggggggvggggggggg
PDF
medical staffing services at VALiNTRY
Design an Analysis of Algorithms II-SECS-1021-03
How to Choose the Right IT Partner for Your Business in Malaysia
Reimagine Home Health with the Power of Agentic AI​
Introduction to Artificial Intelligence
System and Network Administraation Chapter 3
VVF-Customer-Presentation2025-Ver1.9.pptx
Digital Systems & Binary Numbers (comprehensive )
PTS Company Brochure 2025 (1).pdf.......
Raksha Bandhan Grocery Pricing Trends in India 2025.pdf
wealthsignaloriginal-com-DS-text-... (1).pdf
Softaken Excel to vCard Converter Software.pdf
Claude Code: Everyone is a 10x Developer - A Comprehensive AI-Powered CLI Tool
2025 Textile ERP Trends: SAP, Odoo & Oracle
EN-Survey-Report-SAP-LeanIX-EA-Insights-2025.pdf
Designing Intelligence for the Shop Floor.pdf
top salesforce developer skills in 2025.pdf
SAP S4 Hana Brochure 3 (PTS SYSTEMS AND SOLUTIONS)
Adobe Illustrator 28.6 Crack My Vision of Vector Design
Operating system designcfffgfgggggggvggggggggg
medical staffing services at VALiNTRY

An Approach to Automated Application Tuning