SlideShare a Scribd company logo
Production
Model
Deployment
Juliet Hougland
@j_houg
April 2018
@j_houg 2
whoami
And other,
failed startups
Agenda
@j_houg 3
MODEL LIFECYCLE
DEPLOYMENT CHALLENGES
SOLUTIONS
THAT THEMSELVES ARE CHALLENGES
THERE ARE SOLUTIONS TO THOSE TOO!
THEY ARE ALSO CHALLENGING
@j_houg 4
Model Lifecycle
@j_houg
Black Box Data Scientists
@j_houg
Black Box Data Scientists
@j_houg
The Black Box
@j_houg 8
Data
Warehouse
Featurization Training ValidationApplication
@j_houg 9
Deploying a Model
@j_houg 10
Share!
@j_houg 11
3 Modes of Deployment
Data
Warehouse
@j_houg 12
What is a model?
Data
Warehouse
Featurization Training Validation
Application
New
Observation Featurization
Application
Learned
Prediction
@j_houg 13
Moving Models
@j_houg 14
Data
Warehouse
Featurization Training Validation
Application
New
Observation
Featurization
Application
Learned
Prediction
Serialized
Model
Server
@j_houg 15
Deployment
Challenges &
Solutions
@j_houg 16
3 Questions
Does your model do what you need?
Does it meet your engineering
requirements?
Is your team organized to build and
support it?
@j_houg 17
Function?
@j_houg 18
Useful?
George Box
(kind of) said
“All models are
wrong, some are
useful.”
@j_houg 19
Useful?
@j_houg 20
Continues to be useful?
Forever?
Distribution of features
Distribution of predictions
@j_houg 21
Continues to be useful?
Forever?
Distribution of features
Distribution of predictions
@j_houg 22
Pipelines
We don’t deploy
one model, we
deploy the
process for
repeatedly
making more
@j_houg 23
Data
Warehouse
Featurization Training Validation
Application
New
Observation
Featurization
Application
Learned
Prediction
Serialized
Model
Featurization Training ValidationApplication
Serialized
Model
@j_houg
When?
Nightly
Continuously
@j_houg
Scheduled Model Training
Cron
@j_houg
Continuous Training
oryx.io and https://www2007.org/papers/paper570.pdf
@j_houg 27
Meets Requirements?
@j_houg 28
Throughput?
Client
Load Balancer
Model
Server
Model
Server
Model
Server
State
Store
@j_houg
Lambda & Throughput?
oryx.io and https://www2007.org/papers/paper570.pdf
@j_houg 30
Fast enough?
@j_houg 31
Fast enough?
Materialize model features
Application
New
Observation
Featurization
Learned
Prediction
Feature
Store
@j_houg 32
Fast enough?
Materialize model output
Application
New
Observation
Featurization
Learned
Prediction
Model
Output
@j_houg 33
Fast enough?
@j_houg 34
Model Handoff
#WOCTechChat
@j_houg 35
Model Handoff
@j_houg 36
Error Prone
@j_houg 37
Conway’s Law
"organizations which design
systems ... are constrained
to produce designs which
are copies of the
communication structures of
these organizations."
— M. Conway
@j_houg
Typical Data Science Department
Data
Engineers
Data
Scientists
Software
Engineers
Data Infrastructure Engineers
provide
resources &
support
@j_houg
Typical Data Science Department
Data Driven Capability
Data Engineers
Data Scientists
Software Engineers
Infrastructure Engineers
@j_houg
Lambda & Team Structure
oryx.io and https://www2007.org/papers/paper570.pdf
@j_houg 41
Data
Warehouse
Featurization Training Validation
Application
New
Observation
Featurization
Application
Learned
Prediction
Serialized
Model
@j_houg 42
Fix Model Handoff?
#WOCTechChat
@j_houg
Capabilities
Data Platform Engineers
specialized by function
Capability A Capability B Capability C Capability D
DataScientists
specialized by capability
DataScientists
DataScientists
DataScientists
@j_houg 44
Data
Warehouse
Featurization Training Validation
Application
New
Observation
Featurization
Application
Learned
Prediction
Serialized
Model
@j_houg
Open Standards
Open Scoring
@j_houg 46
Data
Warehouse
Featurization Training Validation
Application
New
Observation
Featurization
Application
Learned
Prediction
Serialized
Model
Open Scoring
@j_houg
Open Standards,
Limited Choices
http://dmg.org/pmml/products.html
@j_houg 48
Data
Warehouse
Featurization Training Validation
Application
New
Observation
Featurization
Application
Learned
Prediction
Serialized
Model
@j_houg 49
Does the model…
Do the thing you want it to?
- Functionality
- Usefulness
- Both continually
Meet your requirements?
- Throughput
- Latency
- Freshness
@j_houg 50
Is your team…
Organized in a way
that supports your
system and its
requirements?
@j_houg
Thanks!
@j_houg
Questions?

More Related Content

PPTX
A Survey on Software Release Planning Models - Slides for the Presentation @ ...
PDF
Preso #5 Recycle.it
PDF
Recycle.it 441 presentation 4
PDF
Software product line with IBEA
PDF
Build the Right Product Right: Transitioning Test from Critiquing to Defining
PDF
Conversion Hotel 2018 Keynote: Aleksander Fabijan
PDF
Production Model Deployment - StitchFix - 2018
PPTX
Production and Beyond: Deploying and Managing Machine Learning Models
A Survey on Software Release Planning Models - Slides for the Presentation @ ...
Preso #5 Recycle.it
Recycle.it 441 presentation 4
Software product line with IBEA
Build the Right Product Right: Transitioning Test from Critiquing to Defining
Conversion Hotel 2018 Keynote: Aleksander Fabijan
Production Model Deployment - StitchFix - 2018
Production and Beyond: Deploying and Managing Machine Learning Models

Similar to Production model deployment (20)

PDF
From Zero to Production Dataiku Meetup Berlin
PPTX
Comcast Labs Connect - PHLAI Conference Philadelphia 2018
PDF
Productionising Machine Learning Models
PDF
Bridging the Gap: from Data Science to Production
PDF
Tecton - The state of applied ML 2023.pdf
PDF
Rsqrd AI: From R&D to ROI of AI
PPTX
Deploying ML models to production (frequently and safely) - PYCON 2018
PPTX
Scaling & Managing Production Deployments with H2O ModelOps
PPTX
Why do the majority of Data Science projects never make it to production?
PPTX
Deploying ML models in the enterprise
PPTX
Lifecycle of a Data Science Project
PPTX
The Other 99% of a Data Science Project
PPTX
Deploying ML models to production (frequently and safely) - PYCON 2018
PPTX
Productionalizing Machine Learning Models: The Good, the Bad, and the Ugly
PDF
Productionalizing Machine Learning Models: The Good, The Bad and The Ugly
PDF
Apache ® Spark™ MLlib 2.x: How to Productionize your Machine Learning Models
PDF
From Lab to Factory: Creating value with data
PDF
Data Science: Good, Bad and Ugly by Irina Kukuyeva
PPTX
“Machine Learning in Production + Case Studies” by Dmitrijs Lvovs from Epista...
PDF
Fundamental MLOps
From Zero to Production Dataiku Meetup Berlin
Comcast Labs Connect - PHLAI Conference Philadelphia 2018
Productionising Machine Learning Models
Bridging the Gap: from Data Science to Production
Tecton - The state of applied ML 2023.pdf
Rsqrd AI: From R&D to ROI of AI
Deploying ML models to production (frequently and safely) - PYCON 2018
Scaling & Managing Production Deployments with H2O ModelOps
Why do the majority of Data Science projects never make it to production?
Deploying ML models in the enterprise
Lifecycle of a Data Science Project
The Other 99% of a Data Science Project
Deploying ML models to production (frequently and safely) - PYCON 2018
Productionalizing Machine Learning Models: The Good, the Bad, and the Ugly
Productionalizing Machine Learning Models: The Good, The Bad and The Ugly
Apache ® Spark™ MLlib 2.x: How to Productionize your Machine Learning Models
From Lab to Factory: Creating value with data
Data Science: Good, Bad and Ugly by Irina Kukuyeva
“Machine Learning in Production + Case Studies” by Dmitrijs Lvovs from Epista...
Fundamental MLOps
Ad

More from Stitch Fix Algorithms (11)

PPTX
Progression by Regression: How to increase your A/B Test Velocity
PPTX
Deep recommendations in PyTorch
PDF
Tracking data lineage at Stitch Fix
PDF
Improving ad hoc and production workflows at Stitch Fix
PDF
A compute infrastructure for data scientists
PPTX
Moment-based estimation for hierarchical models in Apache Spark
PPTX
Optimizing Spark
PPTX
When We Spark and When We Don’t: Developing Data and ML Pipelines
PPTX
Incrementality
PDF
Apache Spark & ML Workflows
PDF
Enabling full stack data scientists
Progression by Regression: How to increase your A/B Test Velocity
Deep recommendations in PyTorch
Tracking data lineage at Stitch Fix
Improving ad hoc and production workflows at Stitch Fix
A compute infrastructure for data scientists
Moment-based estimation for hierarchical models in Apache Spark
Optimizing Spark
When We Spark and When We Don’t: Developing Data and ML Pipelines
Incrementality
Apache Spark & ML Workflows
Enabling full stack data scientists
Ad

Recently uploaded (20)

PPTX
MYSQL Presentation for SQL database connectivity
PDF
Profit Center Accounting in SAP S/4HANA, S4F28 Col11
PDF
MIND Revenue Release Quarter 2 2025 Press Release
PDF
How UI/UX Design Impacts User Retention in Mobile Apps.pdf
PDF
Electronic commerce courselecture one. Pdf
PDF
Approach and Philosophy of On baking technology
PPTX
Detection-First SIEM: Rule Types, Dashboards, and Threat-Informed Strategy
PDF
Encapsulation_ Review paper, used for researhc scholars
PPTX
Effective Security Operations Center (SOC) A Modern, Strategic, and Threat-In...
PDF
Build a system with the filesystem maintained by OSTree @ COSCUP 2025
PDF
Dropbox Q2 2025 Financial Results & Investor Presentation
PDF
Blue Purple Modern Animated Computer Science Presentation.pdf.pdf
PDF
Encapsulation theory and applications.pdf
PPTX
ACSFv1EN-58255 AWS Academy Cloud Security Foundations.pptx
PPTX
Digital-Transformation-Roadmap-for-Companies.pptx
PDF
Optimiser vos workloads AI/ML sur Amazon EC2 et AWS Graviton
PPTX
sap open course for s4hana steps from ECC to s4
PPTX
Programs and apps: productivity, graphics, security and other tools
PDF
Per capita expenditure prediction using model stacking based on satellite ima...
PDF
Building Integrated photovoltaic BIPV_UPV.pdf
MYSQL Presentation for SQL database connectivity
Profit Center Accounting in SAP S/4HANA, S4F28 Col11
MIND Revenue Release Quarter 2 2025 Press Release
How UI/UX Design Impacts User Retention in Mobile Apps.pdf
Electronic commerce courselecture one. Pdf
Approach and Philosophy of On baking technology
Detection-First SIEM: Rule Types, Dashboards, and Threat-Informed Strategy
Encapsulation_ Review paper, used for researhc scholars
Effective Security Operations Center (SOC) A Modern, Strategic, and Threat-In...
Build a system with the filesystem maintained by OSTree @ COSCUP 2025
Dropbox Q2 2025 Financial Results & Investor Presentation
Blue Purple Modern Animated Computer Science Presentation.pdf.pdf
Encapsulation theory and applications.pdf
ACSFv1EN-58255 AWS Academy Cloud Security Foundations.pptx
Digital-Transformation-Roadmap-for-Companies.pptx
Optimiser vos workloads AI/ML sur Amazon EC2 et AWS Graviton
sap open course for s4hana steps from ECC to s4
Programs and apps: productivity, graphics, security and other tools
Per capita expenditure prediction using model stacking based on satellite ima...
Building Integrated photovoltaic BIPV_UPV.pdf

Production model deployment