Py data scikit-production
Download as PPTX, PDF5 likes4,125 views
The document discusses deploying machine learning models in production environments. It outlines several challenges with current approaches such as models being opaque objects and a focus on training rather than prediction. It then proposes six requirements for an architecture to handle live traffic directly from trained models: 1) easy integration, 2) high performance, 3) fault tolerance, 4) scalability, 5) maintainability, and 6) extensibility. Finally, it introduces Dato Predictive Services as a platform that meets these requirements by deploying models as low-latency REST services that can elastically scale and includes monitoring and model management capabilities.
Py data scikit-production
- 1. Deploying scikit-learn Models in Production Rajat Arya (@rajatarya) Product Manager, Dato Inc. 1
- 2. 2 Dato provides a platform for building intelligent apps Data Engineering Data Intelligence Deployment ⢠Fast & scalable ⢠Rich data type support ⢠Visualization ⢠App-oriented ML ⢠Supporting utils ⢠Extensibility ⢠Batch & always-on ⢠RESTful interface ⢠Elastic & robust Build, deploy, & manage your intelligent apps with Dato.
- 3. 3 DATA ML Algorithm How Everyone Starts with ML ⢠Running experiments ⢠Plots are the results ⢠Not clear how to get this deployed
- 4. 4 DATA ML Algorithm Deployment? ⢠Write a spec for other team to implement in âproductionâ language ⢠Translate code in 6-12 months ⢠Stale / irrelevant model implemented ⢠Two teams maintaining two systems Custom Model Data Engineers, Data Architects, DevOps, App Developers App A P I Data Scientist
- 5. 5 Current Challenges ⢠Machine Learning Models are opaque objects ⢠Export format like PMML donât support many models ⢠Focus on training, not prediction
- 6. 6 Starting from the Beginning GOAL: Handle live production traffic directly served from the trained machine learning model What are the requirements if we wanted to build a similar architecture for ML Models?
- 7. One: Easy to Integrate ⢠REST APIs for both querying and management ⢠Have client libraries in other languages (no Python lock-in) 7 App A P I
- 8. Two: High Performance ⢠Utilize Load Balancer for distributing request load ⢠Integrated distributed cache so repeated queries are only answered once 8 App A P I C A C H E A P I C A C H E Engine A P I C A C H E LB
- 9. Three: Fault Tolerant ⢠Model running on many machines ⢠System operational during node failure 9 App A P I C A C H E A P I C A C H E A P I C A C H E LB GLC Model GLC Model GLC Model Engine Engine Engine
- 10. Four: Scalable ⢠Elastic scale nodes in cluster up and down ⢠Easy to configure, cache automatically updates with cluster changes 10 App A P I C A C H E A P I C A C H E LB GLC Model GLC Model Engine Engine A P I C A C H E Engine A P I C A C H E Engine
- 11. Five: Maintainable ⢠Zero downtime during model deployment ⢠Metrics & logs ⢠Model management 11 App A P I C A C H E A P I C A C H E A P I C A C H E LB GLC Model GLC Model GLC Model Engine Engine Engine
- 12. Six: Extensible ⢠Arbitrary Python ⢠Use any set of Python packages ⢠Model ensembling 12 App A P I C A C H E A P I C A C H E A P I C A C H E LB GLC Model GLC Model GLC Model Python Python Python
- 13. 13 Requirements Recap 1. Easy to Integrate 2. High Performance 3. Fault Tolerant 4. Scalable 5. Maintainable 6. Extensible App A P I C A C H E A P I C A C H E A P I C A C H E LB GLC Model GLC Model GLC Model Python Python Python
- 14. 14 Do-It-Yourself ⢠Web Service layer: - Tornado, Flask, Keen, Django, etc ⢠Caching layer: - Redis, Cassandra, Memcached, DynamoDb, BerkeleyDb, MySQL, etc ⢠Logs: - Logback, LogStash, Splunk, Loggly ⢠Metrics: - AWS CloudWatch, Mixpanel, Librato, etc
- 15. 15 ⌠or use Dato Predictive Services We set out with this goal, and used these requirements ⌠and now I'd like to show it to you.
- 16. DEMO: Deploying a scikit-learn model using Dato Predictive Services 16
- 17. 17 Models as Services ⢠Deploy models as low-latency REST services ⢠Elastically scale up or out with one command ⢠Monitoring & Model Management ⢠Deploy existing Python models ⢠Run on AWS EC2 or Hadoop YARN Dato Predictive Services Predictive Engine REST Client Direct Model Mgmt
Editor's Notes
- #4: So I got started with ML by taking a class. Data -> to ML algo, and then generate a plot. Of course this isnât how actual applications are written, but this is often where customers are starting when approaching taking ML to production.
- #5: So I got started with ML by taking a class. Data -> to ML algo, and then generate a plot. Of course this isnât how actual applications are written, but this is often where customers are starting when approaching taking ML to production.