MLOps for Compositional AI
Download as PPTX, PDF0 likes92 views
The document discusses the development of compositional AI within the context of MLOps, emphasizing the integration of machine learning models across various enterprise applications such as product repair assessment and recommendation systems. It highlights the necessity for an integrated data operations (DataOps) and ML operations (MLOps) pipeline to utilize model inferences effectively and create new training data. Ultimately, it asserts that compositional AI can enhance the efficiency and agility of ML model development in enterprises.
![Compositional AI Scenario (2)
In future, when the enterprise is looking for models
to develop a Product Recommendation service; the
Repair Service is considered.
The data gathered by the Repair Service: state of
products owned by the users (gathered by CV
assessment model) together with their demographics
(gathered by the Ordering Chatbot) - provides
additional training data for the Recommender Service.
Privacy policies may prevent their data from being
combined, such that, they cannot be used to profile
customers – “data used for a different purpose than
originally intended”. Product Repair
Assessment CV Model Chatbot Ordering App
Repair Ordering
Service
[Damaged product
images + Text
description, Customer
demographics ]
Product
Recommendation
Service
[Products
purchased +
Demographics]](https://image.slidesharecdn.com/mlopsdmml-221111184841-00d7ab2e/85/MLOps-for-Compositional-AI-4-320.jpg)
![Compositional AI Scenario (3)
Enterprise further wants to develop a CV
App to detect Defective products during
Manufacturing.
The Repair Service can help here as it has
labeled images of damaged products (with
the product damage descriptions provided
to the Chatbot acting as ‘labels’).
Product Repair
Assessment CV Model Chatbot Ordering App
Repair Ordering
Service
[Damaged product
images + Text
description, Customer
demographics ]
Manufacturing Defect
Detection App
[Damaged Product
images + Text
description]](https://image.slidesharecdn.com/mlopsdmml-221111184841-00d7ab2e/85/MLOps-for-Compositional-AI-5-320.jpg)
MLOps for Compositional AI
- 1. MLOps for Compositional AI Debmalya Biswas NeurIPS 2022 Workshop: Challenges in Deploying and Monitoring Machine Learning Systems (DMML)
- 2. Enterprise AI Enterprise AI/ML use-cases are pervasive. 4 Broadly categorized by the three core AI/ML capabilities enabling them: Natural Language Processing (NLP), Computer Vision and Predictive Analytics Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor Majority of AI/ML models are still developed with the goal of solving a single task, e.g., prediction, classification.
- 3. Compositional AI Scenario Consider the online Repair Service of a luxury goods vendor. The service consists of a Computer Vision (CV) model capable of assessing the repairs needed, given a picture of the product uploaded by the customer. Product Repair Assessment CV Model Chatbot Ordering App Repair Ordering Service The assessment is followed by an Ordering Chatbot conversation that captures additional details required to process the user’s repair request, e.g., damage details, username, contact details, etc.
- 4. Compositional AI Scenario (2) In future, when the enterprise is looking for models to develop a Product Recommendation service; the Repair Service is considered. The data gathered by the Repair Service: state of products owned by the users (gathered by CV assessment model) together with their demographics (gathered by the Ordering Chatbot) - provides additional training data for the Recommender Service. Privacy policies may prevent their data from being combined, such that, they cannot be used to profile customers – “data used for a different purpose than originally intended”. Product Repair Assessment CV Model Chatbot Ordering App Repair Ordering Service [Damaged product images + Text description, Customer demographics ] Product Recommendation Service [Products purchased + Demographics]
- 5. Compositional AI Scenario (3) Enterprise further wants to develop a CV App to detect Defective products during Manufacturing. The Repair Service can help here as it has labeled images of damaged products (with the product damage descriptions provided to the Chatbot acting as ‘labels’). Product Repair Assessment CV Model Chatbot Ordering App Repair Ordering Service [Damaged product images + Text description, Customer demographics ] Manufacturing Defect Detection App [Damaged Product images + Text description]
- 6. Compositional AI (Labeled) Data (Train) ML Model API Endpoint (New) Composite Service Arbitrary composition of Data, Model, API Compositional AI envisions seamless composition of existing AI/ML services, to provide a new (composite) AI/ML service, capable of addressing complex multi-domain use-cases.
- 7. MLOps Manages model versions and parameters, however model composition aspect is missing. * D. Sculley, et. al. Hidden Technical Debt in Machine Learning Systems. NIPS 2015: 2503-2511 MLOps, also known as ModelOps, combines DevOps with ML to manage ML models in production. End-to-end ML lifecycle: Data and (Serving) API aspects are also considered.
- 8. DataOps Reality: this curated / processed data is moved to another location, e.g., cloud storage buckets, or another data lake, where it is further transformed as part of ML training and deployment. “DataOps is an automated, process- oriented methodology, used by analytic and data teams, to improve the quality and reduce the cycle time of data analytics.” - Wikipedia Source data, both structured and unstructured, is ingested into the Bronze layer, where it is cleansed and standardized into the Sliver layer, with further modeling and transformation into the Gold layer. The data is now ready for consumption by both BI — Reporting tools & ML pipelines. This results in redundancy and a fragmentation of the DataOps and MLOps pipelines
- 9. Bridging DataOps & MLOps - Challenges The data (pre-)processing part of MLOps includes a series of transformations that support a learning algorithm – which are more complex than those supported by traditional ETL tools.
- 10. Bridging DataOps & MLOps - Solutions Snowflake recently announced Snowpark Python API that allows ML models to be trained and deployed within Snowflake, with Snowpark allowing data scientists to use Python (rather than writing code in SQL) Google Cloud Platform (GCP) provides BigQuery ML, a GCP tool that allows ML models to be trained purely using SQL within GCP’s Data Warehouse environment. AWS Redshift Data API makes it easy for any application written in Python to interact with Redshift. This allows a SageMaker notebook to connect to the Redshift cluster and run Data API commands in Python.
- 11. ML Model Inferences as a new Data Source Inferences made by a deployed ML model can be provided as a feedback loop to augment the existing training dataset of the deployed model, or as training dataset for a new model. This leads to the scenario where a deployed ML model generates new data — acts as a Data Source for the DataOps pipeline. Synthetic data can also be considered as an additional data source here. Structured Raw / Staging (Bronze) Cleansed / Standardized (Silver) Transformed / Modeled (Gold) Unstructured BI / Reporting AI/ML Feature extraction Training dataset Test dataset Model Training Exploratory Data Analysis Model Serving (Inference) Model Monitoring DataOps DQ/Validation Filtering Historization Aggregation Data pre-processing within MLOps DQ/Cleaning Encoding Selection Normalization ML Outputs (Inferences, Predictions )
- 12. Conclusion In this paper, we highlighted two aspects of MLOps needed to enable Compositional AI, primarily an integrated DataOps-MLOps pipeline that leverages model inferences to augment/generate new training data (for new models). As ML models proliferate in the enterprise, Compositional AI has the potential to enable reuse of (training) data and models, improving agility and efficiency in ML model development.
- 13. 13 Thanks for your attention Contact: Debmalya Biswas (LinkedIn) (Medium)