MLOps - Build pipelines with Tensor Flow Extended & Kubeflow
0 likes621 views
The document discusses Machine Learning Operations (MLOps) and the importance of using pipelines throughout the machine learning lifecycle, particularly with tools like TensorFlow Extended (TFX) and Kubeflow. It highlights challenges in scaling AI initiatives, including low success rates and inefficiencies in transitioning from proof of concept (POC) to production. The document also emphasizes the role of automation in enhancing ML workflows, data processing, and model management to drive efficiency in AI development.
MLOps - Build pipelines with Tensor Flow Extended & Kubeflow
- 1. Prof. Dr. Jan Kirenz Machine Learning Operations (MLOps) Usage of Pipelines in the ML Lifecycle with Tensor Flow Extended (TFX) and Kubeflow Prof. Dr. Jan Kirenz HdM Stuttgart
- 2. Prof. Dr. Jan Kirenz 80-85% PoC Factory The Proof of Concept Factory Most companies... ● … conduct AI experiments and pilots but achieve a low scaling success rate ● … have significant under investment, yielding low returns Source: Accenture (2019) https://www.accenture.com/us-en/insights/artificial-intelligence/ai-investments
- 3. Prof. Dr. Jan Kirenz https://www.gartner.com/smarterwithgartner/gartner-top-10-data-and-analytics-trends-for-2021/ Scalable AI
- 4. Prof. Dr. Jan Kirenz ML Project Code The problem with scaling AI: ML code is only a fraction of a production-ready ML project code ML Code 5-10%
- 5. Prof. Dr. Jan Kirenz Monitoring Hidden technical debt in machine learning systems Sculley, D. et al. (2015). Hidden technical debt in machine learning systems. Advances in neural information processing systems, 28, pp. 2503-2511 Data Collection Configuration Feature Engineering Data Verification Metadata Management Model Analysis Serving Infra- structure Automation Process Management Machine Resource Management Testing and Debugging ML Code
- 6. Prof. Dr. Jan Kirenz Machine learning operations (MLOps) ● ML Engineering culture and practice that aims at unifying ML System development (Dev) and ML system operations (Ops) ● Tools and principles to support workflow standardization and automation through the ML system lifecycle (e.g. with pipelines)
- 7. Prof. Dr. Jan Kirenz Prof. Dr. Jan Kirenz Machine learning lifecycle
- 8. Prof. Dr. Jan Kirenz Plan Model Deployment Data Business Analyst Data Engineer Software Developer Data Scientist Lifecycle of an ML System Plan | Data | Model | Deployment
- 9. Prof. Dr. Jan Kirenz Plan Model Deployment Data Identify use case Frame problem Identify variables Define metrics Business Analyst Data Engineer Software Developer Data Scientist Lifecycle of an ML System Plan | Data | Model | Deployment
- 10. Prof. Dr. Jan Kirenz Plan Model Deployment Data Identify use case Frame problem Identify variables Define metrics Business Analyst Data Engineer Software Developer Data Scientist Data ingestion Analyze & clean data Define schema Feature engineering Data splitting Anomaly detection Data preprocessing Lifecycle of an ML System Plan | Data | Model | Deployment
- 11. Prof. Dr. Jan Kirenz Plan Model Deployment Data Identify use case Frame problem Identify variables Define metrics Business Analyst Data Engineer Software Developer Data Scientist Data ingestion Analyze & clean data Define schema Feature engineering Evaluate model Model Training & tuning Select algorithm Data splitting Anomaly detection Data preprocessing Lifecycle of an ML System Plan | Data | Model | Deployment
- 12. Prof. Dr. Jan Kirenz Plan Model Deployment Data Identify use case Frame problem Identify variables Define metrics Business Analyst Data Engineer Software Developer Data Scientist Data ingestion Analyze & clean data Define schema Feature engineering Validate model Deploy model Serve model Retrain triggers Evaluate model Model Training & tuning Monitor model Select algorithm Data splitting Anomaly detection Data preprocessing Lifecycle of an ML System Plan | Data | Model | Deployment
- 13. Prof. Dr. Jan Kirenz Plan Model Deployment Data Identify use case Frame problem Identify variables Define metrics Business Analyst Data Engineer Software Developer Data Scientist Data ingestion Analyze & clean data Define schema Feature engineering Validate model Deploy model Serve model Retrain triggers Evaluate model Model Training & tuning Monitor model Select algorithm Data splitting Anomaly detection Data preprocessing Lifecycle of an ML System Plan | Data | Model | Deployment Common issues which lead to a PoC to production gap ● Lack of reuse and duplication ● Inconsistency (data, code, models) ● Manual and slow transition from PoC to production
- 14. Prof. Dr. Jan Kirenz Plan Model Deployment Data Identify use case Frame problem Identify variables Define metrics Business Analyst Data Engineer Software Developer Data Scientist Data ingestion Analyze & clean data Define schema Feature engineering Validate model Deploy model Serve model Retrain triggers Evaluate model Model Training & tuning Monitor model Select algorithm Data splitting Anomaly detection Data preprocessing Model management Model registry Data and feature management Feature store Pipeline management Pipeline orchestration Metadata management Metadata store Lifecycle of an ML System Plan | Data | Model | Deployment MLOps components
- 15. Prof. Dr. Jan Kirenz What is a pipeline? ● Description of an ML workflow ● A pipeline component is a self-contained set of user code that performs one step in the pipeline ● Includes the definition of the configuration and inputs required to run the pipeline (e.g. model hyperparameters) … do this … than that Start ... … the end The workflow is also called directed acyclic graph (DAG) This is a component Complete workflow of the ML system lifecycle
- 16. Prof. Dr. Jan Kirenz Source: Baer & Ngahane (2019) … do this … than that Start … the end
- 17. Prof. Dr. Jan Kirenz TensorFlow Extended (TFX) ● Google-production-scale machine learning (ML) platform based on TensorFlow ● Portable to multiple environments (Azure, AWS, Google Cloud, IBM, ...) ● Python based toolkit; can be used with notebooks ● Helps you orchestrate your ML process: Apache Airflow, Apache Beam or Kubeflow pipelines Source: TensorFlow (2021)
- 18. Prof. Dr. Jan Kirenz TFX 1.0 (19.05.21) ● Enterprise-grade support ● Security patches and select bug fixes for up to three years ● Guaranteed API & Artifact backward compatibility Source: Google (2021)
- 19. Prof. Dr. Jan Kirenz Plan Model Deployment Data Identify use case Frame problem Identify variables Define metrics Business Analyst Data Engineer Software Developer Data Scientist Data ingestion Analyze & clean data Define schema Feature engineering Validate model Deploy model Serve model Evaluate model Model Training & tuning Monitor model ExampleGen Select algorithm StatisticsGen SchemaGen Example Validator Transform Data splitting Trainer Tuner Evaluator InfraValidator Anomaly detection Pusher HUB / JS / LITE / SERVING Model Server TF Data Validation (TFDV) TFT TF KerasTuner TensorFlow Model Analysis (TFMA) BulkInferrer Data preprocessing Metadata Store: ML Metadata (MLMD) TFX Options for Pipeline Orchestration
- 20. Prof. Dr. Jan Kirenz Plan Model Deployment Data Identify use case Frame problem Identify variables Define metrics Business Analyst Data Engineer Software Developer Data Scientist Data ingestion Analyze & clean data Define schema Feature engineering Validate model Deploy model Serve model Evaluate model Model Training & tuning Monitor model ExampleGen Select algorithm StatisticsGen SchemaGen Example Validator Transform Data splitting Trainer Tuner Evaluator InfraValidator Anomaly detection Pusher HUB / JS / LITE / SERVING Model Server TF Data Validation (TFDV) TFT TF KerasTuner TensorFlow Model Analysis (TFMA) BulkInferrer Metadata Store (ML Metadata) TFX Options for Pipeline Orchestration Data preprocessing
- 21. Prof. Dr. Jan Kirenz Plan Model Deployment Data Identify use case Frame problem Identify variables Define metrics Business Analyst Data Engineer Software Developer Data Scientist Data ingestion Analyze & clean data Define schema Feature engineering Validate model Deploy model Serve model Evaluate model Model Training & tuning Monitor model ExampleGen Select algorithm StatisticsGen SchemaGen Example Validator Transform Data splitting Trainer Tuner Evaluator InfraValidator Anomaly detection Pusher HUB / JS / LITE / SERVING Model Server TF Data Validation (TFDV) TFT TF KerasTuner TensorFlow Model Analysis (TFMA) BulkInferrer Data preprocessing Metadata Store (ML Metadata) TFX Options for Pipeline Orchestration
- 22. Prof. Dr. Jan Kirenz Plan Model Deployment Data Identify use case Frame problem Identify variables Define metrics Business Analyst Data Engineer Software Developer Data Scientist Data ingestion Analyze & clean data Define schema Feature engineering Validate model Deploy model Serve model Evaluate model Model Training & tuning Monitor model ExampleGen Select algorithm StatisticsGen SchemaGen Example Validator Transform Data splitting Trainer Tuner Evaluator InfraValidator Anomaly detection Pusher HUB / JS / LITE / SERVING Model Server TF Data Validation (TFDV) TFT TF TensorFlow Model Analysis (TFMA) BulkInferrer Data preprocessing Metadata Store (ML Metadata) TFX Options for Pipeline Orchestration
- 23. Prof. Dr. Jan Kirenz Plan Model Deployment Data Identify use case Frame problem Identify variables Define metrics Business Analyst Data Engineer Software Developer Data Scientist Data ingestion Analyze & clean data Define schema Feature engineering Validate model Deploy model Serve model Evaluate model Model Training & tuning Monitor model ExampleGen Select algorithm StatisticsGen SchemaGen Example Validator Transform Data splitting Trainer Tuner Evaluator InfraValidator Anomaly detection Pusher HUB / JS / LITE / SERVING Model Server TF Data Validation (TFDV) TFT TF TensorFlow Model Analysis (TFMA) BulkInferrer Data preprocessing Metadata Store (ML Metadata) TFX Options for Pipeline Orchestration
- 24. Prof. Dr. Jan Kirenz Plan Model Deployment Data Identify use case Frame problem Identify variables Define metrics Business Analyst Data Engineer Software Developer Data Scientist Data ingestion Analyze & clean data Define schema Feature engineering Validate model Deploy model Serve model Evaluate model Model Training & tuning Monitor model ExampleGen Select algorithm StatisticsGen SchemaGen Example Validator Transform Data splitting Trainer Tuner Evaluator InfraValidator Anomaly detection Pusher HUB / JS / LITE / SERVING Model Server TF Data Validation (TFDV) TFT TF KerasTuner TensorFlow Model Analysis (TFMA) BulkInferrer Data preprocessing Metadata Store (ML Metadata) TFX Options for Pipeline Orchestration
- 25. Prof. Dr. Jan Kirenz Tuner Evaluator InfraValidator ExampleGen StatisticsGen SchemaGen Example Validator Transform Trainer Pusher HUB / JS / LITE / SERVING Model Server KerasTuner BulkInferrer Metadata Store (ML Metadata) TF Data Validation (TFDV) TFT TF TFX Options for Pipeline Orchestration TensorFlow Model Analysis (TFMA) TensorFlow Lite is a set of tools that enables on-device machine learning by helping developers run their models on mobile, embedded, and IoT devices.
- 26. Prof. Dr. Jan Kirenz Plan Model Deployment Data Identify use case Frame problem Identify variables Define metrics Business Analyst Data Engineer Software Developer Data Scientist Data ingestion Analyze & clean data Define schema Feature engineering Validate model Deploy model Serve model Retrain model Evaluate model Model Training & tuning Monitor model ExampleGen Select algorithm StatisticsGen SchemaGen Example Validator Transform Data splitting Trainer Tuner Evaluator InfraValidator Anomaly detection Pusher HUB / JS / LITE / SERVING Model Server TF Data Validation (TFDV) TFT TF KerasTuner TensorFlow Model Analysis (TFMA) BulkInferrer Data preprocessing Metadata Store (ML Metadata) TFX Options for Pipeline Orchestration Production phase: automate the execution of the ML pipeline based on a schedule or certain triggering conditions. Development phase: run the ML experiment, instead of manually executing each step. Data preparation phase: automatically ingest, validate and transform data and provide features to models
- 27. Prof. Dr. Jan Kirenz Prof. Dr. Jan Kirenz Pipeline orchestration
- 28. Prof. Dr. Jan Kirenz TFX & Apache Airflow ● Programmatically author, schedule and monitor workflows with Python code. ● User interface to visualize pipelines running in production, monitor progress, and troubleshoot issues.
- 29. Prof. Dr. Jan Kirenz TFX & Apache Beam ● Provides a framework for running batch and streaming data processing jobs that run on a variety of runners (Spark, Flink, ...). ● Beam provides an abstraction layer which enables TFX to run on any supported runner without code modifications ● TFX only uses the Beam Python API
- 30. Prof. Dr. Jan Kirenz TFX & Kubeflow pipelines The Kubeflow Pipelines platform consists of: ● An engine for scheduling multi-step ML workflows (using Kubernetes). ● User interface (UI) for managing and tracking experiments, jobs, and runs. ● Python SDK for defining and manipulating pipelines and components. ● Notebooks for interacting with the system using the SDK Kubeflow Pipelines is available as a core component of Kubeflow or as a standalone installation.
- 31. Prof. Dr. Jan Kirenz
- 32. Prof. Dr. Jan Kirenz
- 33. Prof. Dr. Jan Kirenz
- 34. Prof. Dr. Jan Kirenz KubeFlow
- 35. Prof. Dr. Jan Kirenz ML toolkit for Kubernetes
- 36. Prof. Dr. Jan Kirenz Google’s Vertex AI Launched in May 2021
- 37. Prof. Dr. Jan Kirenz ML Pipelines | wrap-up Source: TensorFlow (2021) By using a ML pipeline, you can: ● Automate your ML process, which lets you regularly retrain, evaluate, and deploy your model. ● Utilize distributed compute resources for processing large datasets and workloads. ● Increase the velocity of experimentation by running a pipeline with different sets of hyperparameters. To learn more visit the following tutorials @: https://kirenz.github.io/ MLOps tutorials on how to: ● Install TF and TFX ● Build your first TFX pipeline ● Install Kubeflow ● Build your first Kubeflow pipeline Jan Kirenz www.kirenz.com