Machine Learning Models: From Research to Production 6.13.18
4 likes887 views
The document explains the importance of machine learning models in addressing complex prediction problems across various industries, highlighting the limitations of traditional programming methods. It introduces Cloudera Data Science Workbench as a tool that facilitates the end-to-end machine learning process, from model development and training to deployment and monitoring. Additionally, it emphasizes the need for efficient and reproducible research practices to streamline the transition of models from research to production.
Machine Learning Models: From Research to Production 6.13.18
- 1. MACHINE LEARNING MODELS: FROM RESEARCH TO PRODUCTION Tristan Zajonc | Head of Machine Learning Engineering
- 2. 2 © Cloudera, Inc. All rights reserved. WHY ARE MACHINE LEARNING MODELS IMPORTANT? PROTECT business CONNECT products & services (IoT) DRIVE customer insights The world is filled with prediction problems that require a new approach to software ● Predictive maintenance ● Logistics optimization ● Self-driving cars ● Medical diagnostics ● Marketing effectiveness ● Next best action ● Insider threat prevention ● Fraud prevention ● Payment integrity
- 3. 3 © Cloudera, Inc. All rights reserved. SOFTWARE 1.0 Traditional development relies on hand-coded programs that map inputs to outputs FUNCTION OUTPUTINPUT x f(x) y This approach is intractable or performs poorly for many problems that are easy for humans.
- 4. 4 © Cloudera, Inc. All rights reserved. SOFTWARE 2.0 - MACHINE LEARNING FUNCTION OUTPUTINPUT x f(θ,x) y Machine learning searches for programs that map inputs to outputs effectively Machine Learning Model
- 5. 5 © Cloudera, Inc. All rights reserved. THE POWER OF MACHINE LEARNING FUNCTION TEXTPIXELS Machine learning enables software to address entirely new applications Source: https://cs.stanford.edu/people/karpathy/deepimagesent/
- 6. 6 © Cloudera, Inc. All rights reserved.
- 7. 7 © Cloudera, Inc. All rights reserved. MACHINE LEARNING AT FACEBOOK, UBER, AND GOOGLE Efficient, reliable, large-scale machine learning requires new supporting tools Facebook FBLearner Uber Michelangelo Google TFX
- 8. 8 © Cloudera, Inc. All rights reserved. ACCELERATING THREE STAGES OF MACHINE LEARNING Manage models Deploy models Monitor performance DEPLOYDEVELOP Explore data Develop models Share results TRAIN Optimize parameters Track experiments Compare performance Machine learning platforms can accelerate model development, training, and deployment
- 9. 9 © Cloudera, Inc. All rights reserved. CLOUDERA DATA SCIENCE WORKBENCH Enables self-service data science at scale in secure environments For data scientists • Open data science Use R, Python, or Scala with your favorite libraries and on- demand compute • No need to sample Directly access secure data via Spark, Impala, or HDFS • Reproducible, collaborative research Share with your whole team For IT professionals • Bring analysis to the data Give data science team the freedom to work how they want, when they want • Secure by default Stay compliant with out-of-the- box Hadoop security • Flexible deployment On-premises or in the cloud
- 10. 10 © Cloudera, Inc. All rights reserved. NEW IN CLOUDERA DATA SCIENCE WORKBENCH 1.4 Accelerate machine learning from research to production DEVELOP MODELS • Explore data securely and develop models as a team TRAIN MODELS • Train, track, and compare reproducible experiments DEPLOY MODELS • Deploy and monitor models as APIs to serve predictions NEW! NEW!
- 11. 11 © Cloudera, Inc. All rights reserved. RUN AND TRACK EXPERIMENTS
- 12. 12 © Cloudera, Inc. All rights reserved. CHALLENGE: REPRODUCIBLE RESEARCH How do you know what model is better? How can you repeat a result? • Model development is iterative • Try different data, features, libraries, algorithms, hyperparameters, etc. • Reproducing a model means you need (at least)... • Training data • Data/feature pipeline code • Model training code + dependencies • Runtime environment (CPU, GPU, memory, …) • Any results or performance metrics • This is a lot to keep track of!
- 13. 13 © Cloudera, Inc. All rights reserved. HOW THIS WORKS TODAY WHY THIS IS A PROBLEM • Wasted time and effort keeping track of model/environment changes • Wasted time and effort trying to recreate a result, especially for junior data scientists / new team members • Compliance risk due to inability to explain the modeling process • Source control? • Unless you forget • And maybe the library changes • Or this… • mymodel.py • mymodel.final.py • mymodel.final.final2.py • mymodel.final.final2.noreallyfinal.py • Post-it notes or notebook • To keep track of performance metrics
- 14. 14 © Cloudera, Inc. All rights reserved. INTRODUCING EXPERIMENTS Versioned model training runs for evaluation and reproducibility Data scientists can now... • Create a snapshot of model code, dependencies, and configuration necessary to train the model • Build and execute the training run in an isolated container • Track specified model metrics, performance, and model artifacts • Inspect, compare, or deploy prior models
- 15. 15 © Cloudera, Inc. All rights reserved. DEMO
- 16. 16 © Cloudera, Inc. All rights reserved. DEPLOY AND MANAGE MODELS
- 17. 17 © Cloudera, Inc. All rights reserved. CHALLENGE: GETTING TO PRODUCTION So you’ve got a trained model. Now what? • Data scientists want to rapidly expose candidate models to serve predictions • REST APIs are the most requested approach • Development and production are very different • Owners: Data Scientists vs. Data Engineers • Languages: Python/R vs. Java/Scala/C++ • Policy Controls: Approved code, packages, etc. • Vocabulary: Data Science vs. DevOps • Data scientists do not often have the skills (or entitlements) to deploy models
- 18. 18 © Cloudera, Inc. All rights reserved. HOW THIS WORKS TODAY WHY THIS IS A PROBLEM • Can take days to months, raising the bar for what’s worth deploying, leading to stale models and reduced innovation • Many opportunities to introduce errors and compliance risk • Workarounds are difficult to maintain while increasing the skills gap • Models get handed off to an engineering team for re-coding • Can you reproduce the model? • Can you prove it’s the same? • Or data science teams try to hack it • Requires an entirely new set of tools (e.g. Flask, Docker, Kubernetes, …) • Getting an API up is different from maintaining it (e.g. managing, monitoring, testing, updating, scaling, …)
- 19. 19 © Cloudera, Inc. All rights reserved. INTRODUCING MODELS Machine learning models as one-click microservices (REST APIs) Model APIs made easy! 1. Choose Python/R file, e.g. score.py 2. Choose function, e.g. forecast f = open('model.pk', 'rb') model = pickle.load(f) def forecast(data): return model.predict(data) 3. Choose resources 4. Deploy!
- 20. 20 © Cloudera, Inc. All rights reserved. DEMO
- 21. 21 © Cloudera, Inc. All rights reserved. HOW IT WORKS Experiments and Models leverage a new way of building images from source • When running and experiment or deploying a model: • Provides declarative pathway from version control to experiment or model SOURCE • Stage 1: Git snapshot of source, respecting .gitignore (before sure to ignore any local Python/R environment) IMAGE • Stage 2: Docker build from source; cdsw-build.sh defines build steps, e.g.: RUN • Stage 3: Run versioned image as Experiment (batch) or Model (online) in Kubernetes #!/bin/bash pip3 install -r requirements.txt
- 22. 22 © Cloudera, Inc. All rights reserved. Not all machine learning applications have the same requirements AS ALWAYS, USE THE RIGHT TOOL FOR THE JOB • Not all models need online scoring -- batch scoring is simple and reliable • Latency sensitive models are often better deployed to the edge • mobile applications • autonomous vehicles • high-frequency trading • Self-service deployment is not always appropriate • Facebook scale (200 trillion predictions per day) • Prediction services with high SLAs But self-service model deployment enables rapid delivery for many ML use cases and should be an available tool in every agile enterprise.
- 23. Confidential-Restricted – For Discussion Purposes Only23 © Cloudera, Inc. All rights reserved. BENEFITS OF AN UNIFIED MACHINE LEARNING PLATFORM RESEARCH EXPERIENCE ✓ Faster iteration, with confidence ✓ Easier to identify best models ✓ Easier to reproduce/explain work ✓ Easier to onboard team members DEPLOYMENT EXPERIENCE ✓ Faster business impact ✓ Easier to deploy more models ✓ Easier to reproduce/explain work ✓ Easier to access CDH data/compute OPERATIONAL EXPERIENCE ✓ Lower cost and risk for managing models ✓ Easier to support through self-service ✓ Easier to scale a shared environment ✓ Easier to control access
- 24. THANK YOU