Managing the AI process: putting humans (back) in the loop

HUAWEI TECHNOLOGIES CO., LTD.
www.huawei.com
Managing the AI process:
putting humans (back) in the loop
— Balazs Kegl for Noah's Ark Research Lab, Paris

HUAWEI TECHNOLOGIES CO., LTD. Page 2
 AI research veteran (25 years)
› recently crossing over from academic research to industry
 Leading a team of 15 at Huawei Noah's Ark Lab
in Paris
 Main objective: optimizing engineering systems
› Better
› Cheaper
› More reliable
› Safer
› More energy efficient
Who am I?

AI: Highly visible recent breakthroughs

Why these advances
are not already
in engineering systems?

Engineering systems = ~$10s of trillions per year

A typical engineering control system
EngineerSystem
𝒂 𝒕
𝒐 𝒕, 𝒓𝒕
Engineer observes
system states and performance indicators,
tunes some parameters time to time

Automated control, if exists, is based on
deep understanding of the physics
of the system.

Sometimes it goes wrong

But mostly it works (it just doesn't learn)

What is AI (in this context)?
Learn the system behavior
based on historical data
and use it for better control

Meet the data scientist
 Highly trained scientific minds
› Iterating the scientific method, collecting data, running
a lot of experiments, trial and error
› Work on (predictive) "models"
› Work alone or in small teams
› "Full stack" - little specialization
› Build their sandbox
› Hate operational constraints imposed by
standardized tools

Meet the system engineer
 Highly trained operators of systems
› Responsible for safe operation
› Performance is important but secondary (within
limits)
› Little incentive of collecting data and let the data
scientist experiment

Challenge
The system engineer is the
consumer of AI
(so interfaces should be built between AI and engineer)
And also the
crucial collaborator of the data scientist

 SE:“I would like you to use AI to control my engineering system.”
 DS: “Ok, can I access your system with an algorithm which takes control of
the system, possibly breaking it sometimes in order to learn?”
 SE: “Over my dead body.”
 DS: “OK, do you have a simulator which I can use to learn a control policy?”
 SE: “We are working on it. But in any case, it will never be good enough to
be trusted.”
 DS: “Can you execute a new control policy, after thorough checking and with
human safeguards, time to time and log the system variables and KPIs?
 SE: “Maybe.”
A typical conversation between data scientists
(DS) and system engineer (SE)

 Strong top-down mandate
› Don't just create a powerless CDO/CDSO position
 Form "commandos"
› The 70-20-10 rule: 70% of an AI project is change management
 Build trust through iterative pilots
› We need data to start the AI process  we need trust to engage the resources to collect data
 Design and enforce the data science process
› https://towardsdatascience.com/how-to-build-a-data-science-pipeline-f24341848045
 Start building standard operational tools for data science
› Save experience in reusable code
› https://paris-saclay-cds.github.io/ramp-docs/ramp-workflow
Recommendations

Managing the AI process: putting humans (back) in the loop

More Related Content

Similar to Managing the AI process: putting humans (back) in the loop (20)

More from Balázs Kégl (10)

Recently uploaded (20)

Managing the AI process: putting humans (back) in the loop