IBM Developer Model Asset eXchange
- 1. IBM Developer Model Asset eXchange Nick Pentreath Principal Engineer @MLnick DOC ID / Month XX, 2018 / © 2018 IBM Corporation
- 2. About @MLnick on Twitter & Github Principal Engineer, IBM CODAIT - Center for Open-Source Data & AI Technologies Machine Learning & AI Apache Spark committer & PMC Author of Machine Learning with Spark Various conferences & meetups
- 3. Center for Open Source Data and AI Technologies CODAIT codait.org DBG / Oct 4, 2018 / © 2018 IBM Corporation CODAIT aims to make AI solutions dramatically easier to create, deploy, and manage in the enterprise Relaunch of the Spark Technology Center (STC) to reflect expanded mission Improving Enterprise AI Lifecycle in Open Source Gather Data Analyze Data Machine Learning Deep Learning Deploy Model Maintain Model Python Data Science Stack Fabric for Deep Learning (FfDL) Mleap + PFA Scikit-LearnPandas Apache Spark Apache Spark Jupyter Model Asset eXchange Keras + Tensorflow
- 4. Applying Deep Learning: Perception Data ??? Train ??? $$$ Training – Data Scientist, Researcher Model ??? Deploy ??? $$$ Consumption – App Developer, Domain Expert
- 5. Applying Deep Learning: Reality Discovery Execution Consumption Find model Get code Test, verify, fix Train model Deploy model Use model
- 6. Find model Get code Test, verify, fix Train model Deploy model Use model Does a model exist? • … that does what you need • … that is free to use • … that is performant enough
- 7. Is there a good implementation or pre- trained model available? • … that does what you need • … that is free to use • … that is performant enough Find model Get code Test, verify, fix Train model Deploy model Use model
- 8. Verify the model you found • Model matches problem • Code quality & correctness • Licensing • Code, weights, dataset • Performance • Computation • Accuracy Find model Get code Test, verify, fix Train model Deploy model Use model
- 9. Find model Get code Test, verify, fix Train model Deploy model Use model Data Model Compute resources Expertise Trained model
- 10. • Develop inference code (or customize existing code) • Package inference code and model artifacts • Deploy to serving environment Find model Get code Test, verify, fix Train model Deploy model Use model
- 11. • Integrate with your application • … which does not know (or care) about tensors Find model Get code Test, verify, fix Train model Deploy model Use model
- 12. Applying Deep Learning: Challenges Discovery Execution Consumption Find model Get code Test, verify, fix Train model Deploy model Use model
- 13. 13 Discovery
- 16. 16 Training
- 17. * Logos trademarks of their respective projects
- 18. 18 Deployment
- 19. Containers for ML Deployment • But … • What goes in the container is most important • Performance can be highly variable across language, framework, version • Requires devops knowledge, CI / deployment pipelines, good practices • Does not solve the issue of standardization • Formats • APIs exposed • A serving framework is still required on top • Container-based deployment has significant benefits • Repeatability • Ease of configuration • Separation of concerns – focus on what, not how • Allow data scientists & researchers to use their language / framework of choice • Container frameworks take care of (certain) monitoring, fault tolerance, HA, etc.
- 20. 20 Consumability
- 21. 21
- 22. IBM Developer Model Asset eXchange Free, open-source deep learning models. Wide variety of domains. Multiple deep learning frameworks. Vetted and tested code and IP. Build and deploy a web service in 30 seconds. Start training on Fabric for Deep Learning (FfDL) or Watson Machine Learning in minutes. DBG / Oct 4, 2018 / © 2018 IBM Corporation http://ibm.biz/model- exchange
- 23. Fabric for Deep Learning https://github.com/IBM/FfDL FfDL Github Page https://github.com/IBM/FfDL FfDL dwOpen Page https://developer.ibm.com/code/open/projects/fabri c-for-deep-learning-ffdl/ FfDL Announcement Blog http://developer.ibm.com/code/2018/03/20/fabric- for-deep-learning FfDL Technical Architecture Blog http://developer.ibm.com/code/2018/03/20/democr atize-ai-with-fabric-for-deep-learning Deep Learning as a Service within Watson Studio https://www.ibm.com/cloud/deep-learning Research paper: “Scalable Multi-Framework Management of Deep Learning Training Jobs” http://learningsys.org/nips17/assets/papers/paper_ 29.pdf IBM Corporation FfDL provides a scalable, resilient, and fault tolerant deep-learning framework • Fabric for Deep Learning or FfDL (pronounced as ‘fiddle’) is an open source project which aims at making Deep Learning easily accessible to the people it matters the most i.e. Data Scientists, and AI developers. • FfDL provides a consistent way to deploy, train and visualize Deep Learning jobs across multiple frameworks like TensorFlow, Caffe, PyTorch, Keras etc. • FfDL is being developed in close collaboration with IBM Research and IBM Watson. It forms the core of Watson`s Deep Learning service in open source. FfDL
- 24. Fabric for Deep Learning https://github.com/IBM/FfDL DBG / Oct 4, 2018 / © 2018 IBM Corporation Just announced: Support for PyTorch 1.0 – including distributed training and ONNX! Supports distributed training via Horovod
- 25. 25 Training Data Training Code Training Definition Standardized Script Choose trainable model Train model Deploy model Use model
- 26. 26 Choose deployable model Deploy model Use model 26 Data Model Compute resources Expertise Input/output processing REST APIPre-trained model Deep Learning asset on Model Asset Exchange (ibm.biz/model-exchange)
- 27. Choose deployable model Deploy model Use model Deep Learning asset on Model Asset Exchange (MAX) Deploy Swagger specification Inference endpoint Metadata endpoint Microservice Input pre-processing, model execution, and output post-processing
- 28. 28 Demo Demo
- 29. Summary Current status • 22 models (4 trainable) • Image, audio, text, healthcare, time-series and more • 3 Code Patterns demonstrating how to consume MAX models in a web app • Code Pattern on training an audio classifier using Watson Machine Learning • One-line deployment via Docker and on a Kubernetes cluster Potential Future • New MAX web portal launching soon • More deployable models – breadth and depth • More trainable models - transfer learning in particular • More MAX-related content: • Code Patterns • Conference talks, meetups • Workshops • Enhance production-readiness of MAX models • Improve MAX API framework
- 30. IBM Developer Model Asset eXchange Free, open-source deep learning models. Wide variety of domains. Multiple deep learning frameworks. Vetted and tested code and IP. Build and deploy a web service in 30 seconds. Start training on FfDL or Watson Machine Learning in minutes. DBG / Oct 4, 2018 / © 2018 IBM Corporation http://ibm.biz/model- exchange
- 31. 31 Thank you! codait.org twitter.com/MLnick github.com/MLnick developer.ibm.com http://ibm.biz/model-exchange http://ibm.biz/max-developers FfDL Sign up for IBM Cloud and try Watson Studio! https://ibm.biz/BdY89J https://datascience.ibm.com/ MAX
- 32. 32
Editor's Notes
- #7: E.g. research paper?
- #10: Load, analyze, and prepare training data Train, analyze, and tune model
- #23: One example of IBM open source making it easier to consume AI technology in the enterprise is the IBM Code Model Asset eXchange. We’ve taken a collection of best-of-breed open source deep learning models for a wide variety of domains, carefully vetted all the IP, and packaged them all in a consistent, easy to use format. You can deploy our pretrained models in 30 seconds with Docker and Kubernetes. And with our trainable models, you can start training a custom model in minutes using the Fabric for Deep Learning. Speaking of which…
- #24: The Fabric for Deep Learning, or “fiddle” for short, is the open-source core of IBM Watson Machine Learning’s distributed deep learning training service. The FfDL technology is in active use within IBM’s offerings, as we use it heavily for training models in IBM Research. And speaking of IBM Research…
- #29: Explore Model Asset Exchange Deploy model asset as a microservice on Docker Explore microservice Consume / Test microservice (Optional) Show Code Pattern / Web App linked to the model