MLOps - Getting Machine Learning Into Production

1
© 2021 Peak AI Ltd. All Rights Reserved Confidential
AltitudeX Workshop
MLOps
Getting Machine Learning into Production

2
© 2021 Peak AI Ltd. All Rights Reserved Confidential 2
Confidential
© 2021 Peak AI Ltd. All Rights Reserved
A snippet from a popular
online training course,
showing how much can
be involved in MLOps!
Disclaimer

3
© 2021 Peak AI Ltd. All Rights Reserved Confidential 3
Confidential
A basic example of a
Machine Learning (ML) Model
Ref: https://towardsdatascience.com/

4
Introduction
Agenda
Why we need MLOps
What is MLOps
How do we achieve MLOps

5
The average time it takes an organization
to get a single ML model into production
is anywhere between 31 and 90 days —
with some companies spending over a
year on productionizing.
Ref: Wallaroo AI article on Why ML Models Rarely Reaches Production and What You Can
Do About it
Ref: Algorithmia 2021 Enterprise trends in Machine Learning

6
What challenges
do we face?
ⓘ Start presenting to display the poll results on this slide.

7
Long waiting times for
deployment - Sitting far
from the final solution.
Monitoring models and
managing them
Provability
Inconsistent data
availability
Testing
Updating ML models as
new data comes in
Versioning data
Evolving requirements
Multiple technologies
needed
development of new
technologies
Resistance from
Engineering
Large complex problems
that take a lot of
computational power
Explainability
Training on your own
cluster
How do we protect ML
related intellectual
property?
Not knowing what tools are
best
Customers unwilling to
Data cleansing
Na
Insufficient data
Getting data reliably
Managing stakeholder
requests and expectations
Size of the datasets
Testing
Getting Multiple functions
on board with data prep,
creation and gap analysis
of results
Ethics
Time to deployment...
Measuring value
Explainability - eg why was
person X denied a loan,
Java
Data drift
The alignment problem
changing environments
When to retrain a model
Fear and misunderstanding
Bias
Data quality
What challenges do we face?
Participant Responses

8
human-in-the-loop
feedback
Identifying relevant
variables.
User acceptance
Not enough data
Major changes in the
supply chain making
models obsolete- eg
supplier strike
Learning the right tools
Traceability
The definition of
"production ready"
Customers unwilling to
change the way they
operate
Ethics
Multiple technologies
needed
What to have for breakfast
Data problems
Confidence in the accuracy
of the predictions.
Live model updating /
integrating
Long waiting times for
deployment - Sitting far
from the final solution.
Monitoring models and
managing them
Provability
Inconsistent data
availability
Testing
Updating ML models as
new data comes in
Versioning data
Lazy secops who would
rather lock things down
than do their job
Non linear scaling
Productionizing ML
From experiments to
deployment and integrating
into our tech stack
Data quality, data storage,
stakeholder communication

9
What to have for breakfast
Data problems
Confidence in the accuracy
of the predictions.
Live model updating /
integrating human-in-the-
loop feedback
Identifying relevant
variables.
User acceptance
Not enough data
Major changes in the
supply chain making
models obsolete- eg
supplier strike
Learning the right tools
Traceability
The definition of
"production ready"
Lazy secops who would
rather lock things down
than do their job
Non linear scaling
Productionizing ML
From experiments to
deployment and integrating
into our tech stack
Data quality, data storage,
stakeholder communication

10
Confidential 10
What other challenges
do we face?
1. Deploying models
2. Monitoring model performance
3. Testing and redeploying improved models
4. Scaling their AI/ML operations
5. Lack of ROI

11
Why do these
challenges exist?

12
Limited talent pool for
complex AI, particularly
affordability
Dependent on the
customers existing data
Misunderstanding of what
data science is/can do
from non-data scientists
Legacy stuff?
Dynamic environments -
more dynamic than the
solution
Insufficient data upkeep
and storage
Siloed data management
systems
Complexity of the
problems
Inexperience
Lack of VC in UK
Difficult to execute it
Lack of knowledge and/or
control over the source of
data
People expect data
scientists to do mlops but
they may not have been
trained
Lack of integrated cross-
functional teams across
the DS project lifecycle
Not enough time
None existence of AI
ready data
Lack of experience
Existence of off-the-shelf
tools being more
economical than investing
in an in-house solution
Backlog and prioritisation
issues
Dependence on aws aka
gcp
Inaccurate data
Difficult stakeholders
Time to mature
Why do we face these challenges?

13
Emerging areas are
chaotic
Different mindsets
Habits of a lifetime
Model and infrastructure
complexity
Communication
breakdown
Lack of understanding of
the requirements for data
science teams
Business related Silos
cause friction
Continuous changes in
dataset and environment
AI is a nascent field. None
of us are 100% sure what
we're doing yet!
big, fast, global changes
It's a new field
Hard work isn’t easy
Lack of understanding of
ML across the business
Misunderstanding
Immature toolset
ROI - Expectation
Lack of knowledge about
MLOps and fitting with
other DevOps practices
Businesses are often slow
to accept decisions made
by people other than
themselves
High expectations
Human errors
Challenging objectives
Lots of potential solutions.
Which one to choose?
Why do we face these challenges?
Misunderstanding/poor
communication
Skills gap
Computers
Time

14
Confidential 14
1. “Sign off”, red tape, approval processes
2. Siloed teams
3. Complicated Tech Stack
4. Skills
5. Money
6. No MLOps!
What makes it so hard?

15
Confidential 15
1. Bridge the gap between DS and Ops
2. Make use of managed services
3. Introduce MLOps practices
What can we do about it?

16
What is MLOps?
How do we get started?

17
A continuous flow...
Source: Neal Analytics

18
● Iterative-Incremental Development
● Automation
● Continuous Deployment
● Versioning
● Testing
● Reproducibility
● Monitoring
Important
Concepts

19
A typical ML Pipeline
Source: Gartner

20
Technologies
Off the shelf platforms
AI Platform
Google Cloud
AzureML
Microsoft Azure
SageMaker
Amazon Web Services Kubeflow
Open-source tools
for MLOps assembled
on Kubernetes
Peak platform
by Peak 😉
Peak

21
The Peak Platform
Today we announce the general availability of our platform
(called Peak) in January 2022.
It is a cloud based, multi-tenant platform to give you everything
you need to build and deploy Decision Intelligence Solutions at
pace and scale.
We can build solutions for you, build it with you or you can
build it yourself.
We are announcing our data science community (a waiting list
to find out more) - this will include events and early access to
the platform.
Dock - data management - everything you need
to make your data AI ready - includes data
connectors and data bridge.
Factory - an ML workbench designed by data
scientists, for data scientists to create a
centralised intelligence for companies.
Work - a way for commercial leaders to interact
with the intelligence created in Factory - used to
power great decisions.

22
What Open Source tools
do you use for MLOps?

23

24
Getting starting with and
utilising Kubernetes (K8s):
● Kubeflow
● Seldon Core
Technologies
Open-source
Tools & Libraries
Model Tracking:
● MLFlow
● Metaflow
● MLRun
Pipelines:
● Kedro
● ZenML
Automation:
● Flyte
Version Control:
● Data Version
Control (DVC)

25
Confidential 25
1. Tell your team!
2. Identify the stakeholders
3. Decide your priorities
4. Ask some questions
5. Implement some things
6. Iterate, iterate, iterate!
What’s next?

MLOps - Getting Machine Learning Into Production

More Related Content

What's hot (20)

Similar to MLOps - Getting Machine Learning Into Production (20)

More from Michael Pearce (13)

Recently uploaded (20)

MLOps - Getting Machine Learning Into Production

Editor's Notes