Edge AI Miramond technical seminCERN.pdf

Edge AI
How to bring AI at the edge of the physical world
Benoît Miramond / LEAT

 LEAT research lab
 Edge AI
 Different Edge lines
 Edge lines and properties
 Some examples studied at LEAT
 Smart sensors: When AI touches the physical world
 A matter of energy
 The bio-inspired approach
 Conclusion
Outline
2

Laboratoire d’Electronique,
Antennes et Télécommunications
UnitéMixtede Recherche UMR7248
UniversitéCôted’Azur et CNRS

Location
4
Université Côte d’Azur | CNRS | LEAT
Campus SophiaTech
LEAT building

Composition of the laboratory
5
Université Côte d’Azur, CNRS, LEAT
Professors
22
Interns 15
Visitors
PostDoc/
Engineers
PhD students
36 Administration/Technical
staff
3,3+2
Members : ~80 (June 2022)

Activities
6
Research
Teaching
Dissemination Transfert
3 teams
Publications / Conferences
Electronics/ Computer Sciences
Patents / R&D

Lab. Com. UCA/CNRS-Orange Labs
7
Université Côte d’Azur, CNRS, LEAT
• Co-directors : Ph. Ratajczak (Orange Labs)
F. Ferrero (UCA-CNRS)
• Joint research center
• Orange Labs :
Unité de recherche ANT: Antennes (Orange Labs Sophia)
Unité de recherche WAVE: Interactions Ondes-corps humain(Orange Labs Paris)
• 2012-2022 Subjects of research
•Integrated Antennas
• Communications from 60 to 120 GHz
• Sensors and sensor networks
• New materials, electromagnetic modeling and applications

Academic Collaborations
8

Industrial Collaborations
9

Research Teams
10
• ISA: Imaging and Associated Antennas Systems
Imagerie microondes et Systèmes d’Antennes
• CMA: Antenna Design and Modeling
Conception et Modélisation d’Antennes
• EDGE: Edge computing & DiGital systEms
Systèmes Numériques et Calcul embarqué

Edge computing & DiGital Electronics
EDGE Team

EDGE research axis
12
1. eBrain - embedded Bio-inspiRed
Artificial Intelligence and
Neuromorphic Architectures
2. eWISE - energy-aware
WIreless Sensor nEtworks
3. eSoC - energy efficiency
of SoC
E-Health,
Smart
City
IoT,
wearables
Autonomous
cars

14
From embedded systems to Edge Intelligence
Data volume explodes with AI, 5G, IoT
• ONLY 25% of usable data reach a datacenter
• 75% of data must be analyzed on site immediately
The impact in France and Europe will be immense in Aerospace,
Automotive, Defense, Telecom,...
AI / Edge processors market has important growth
GPUs and FPGAs should not dominate this market.
(1) McKinsey – Jan. 2019 – AI-related semiconductor market

15
A contrasted picture on Cloud Artificial Intelligence
Transformer / GPT3

16
Digital Neural Network Accelerators
https://nicsefc.ee.tsinghua.edu.cn/projects/neural-network-accelerator/
GPU
FPGA
CPU
MCU

17
Digital Neural Network Accelerators
https://nicsefc.ee.tsinghua.edu.cn/projects/neural-network-accelerator/
• Specialized chips for AI
calculation in the cloud
• Nvidia GPU, US
• Google TPU, US
• Baidu Kunlun, CH
• GraphCore, EN
• Intel Movidius, US
• Cerebras, US =>
300.000 cores per
wafer, 15kW
• At the Edge
• NVIDIA Jetson can
provide 11 T FLOPs,
dissipating up to 15
W
• Myriad X 4TOPS
dissipating up to 1,5
W
• Google Coral = 4
TOPS for 2W
• …
GPU
FPGA
CPU
MCU
Sensor
MCU
NN
Accelerator
MP SoC
HPC
GPU

Memory Computation Power Efficiency
Edge
Servers
GB 1 Tops 100 W 10
Gops/W
Gateway MB 100 Gops 1 W 100
Gops/W
IoT
Nodes
Hundreds
of kB
1 Gops 1 mW 1 000
Gops/W
Edge Lines and their specific constraints
19

Key elements of IoT sensors
Sensors Connectivity Persons & process
Captures a discrete
representation of the dynamics
of the physical world
Transmits the sensors data
through wireless communication
Provides the information to
people or process the raw data
into more abstract information 20

When EdgeAI enables smart sensing
Fusion of AI, embedded sensors and connectivity
21

 Edge AI also offers the possibility to embed near-sensor processing
 By bringing AI closer to the sensor, the goal is
 To reduce the amount of data to communicate
 To lower the global energy consumption of the digital infrastructure
 To reduce latency for decising making (close or open loop)
 Integrating AI into (near to) the sensor needs to specifically work at different
scales
 Algorithm/training: explore neural architecture that reduce parameters/computation
 Embedded preparation: compression, quantization of the network
 Electronic hardware: design and optimize the electronic architecture to support the
neural network => Hw/Sw Codesign
Smart sensors
22

 Complete Solution: from Training to Edge
 Training of networks
(frameworks PyTorch, Keras, N2D2)
 Embedded preparation of ANN with
MicroAI
• Quantification des SNN
• Automatic code generation
• Open-source:
https://bitbucket.org/edge-team-leat/microai_public
 Hardware accelerator: next gen AI
• Convolutionnal networks
• Reprogrammable Architecture
• Signal and Image processing applications
The LEAT codesign flow for Edge AI
Training
framework
• PyTorch
• N2D2
• Keras
MicroAI
• Compression
• Quantificarion
• Code generation
Edge
Deployment
• MCU
• MCU +
SPLEAT
Quantization and deployment of deep neural networks on microcontrollers, PE Novac, GB Hacene, A Pegatoquet, B Miramond, V Gripon, Sensors 21 (9), 2984, 2021
SPLEAT: SPiking Low-power Event-based ArchiTecture for in-orbit processing of satellite imagery,, N. Abderrahmane B. Miramond, IJCNN 2022
23

24
Example of near-sensor classification
1
Send the entire
image
Send only the
images without
clouds, fire, …
2
VS.
CIAR

What is the on-board scientific experience ?
FPGA Electronic device
1. Artificial Neural Network
2. Bio-inspired Spiking Neural Network
More details in the publication:
“An Hybrid Neural Network on FPGA for
Embedded Satellite Image Classification“, Edgar
Lemaire et al., IEEE International Symposium on
Circuits and Systems (ISCAS), 2020
25
CIAR

MitySOM FPGA board
E Lemaire, M Moretti, L Daniel, B Miramond, P Millet, An FPGA-based Hybrid Neural Network accelerator for embedded satellite image classification, IEEE International Symposium on
Circuits and Systems 2020
OPS-SAT: an ESA CubeSat
26
From Spiking Neural Networks
to bio-inspired machine-learning CIAR
Next step: full spike architecture with SPLEAT (SPiking Low-energy Event-based ArchiTecture) 1k -> 500k synapses
Rate coding + CNN to SNN Conversion
[L. Khacef, N. Abderrahmane and B. Miramond. Confronting machine-learning with neuroscience for
neuromorphic architectures design. In International Joint Conference on Neural Networks (IJCNN). 2018]

27
Example of distributed AI with Satellite IoT
Satellite LoRa
Gateway
Embedded AI
Ns: Nodes
CH: Cluster Head
AI: Artificial Intelligence
MicroAI + SPLEAT
DIRECT ACCESS
INDIRECT ACCESS
Sensor Node designed at LEAT
Ground sensors connected to a satellite.
End nodes embed sensors, MCU, battery and LoRA connectivity.
Each node embeds EdgeAI and has to be autonomous in energy
I. Abdoulaye, L. Rodriguez, C. Beleudy, B. Miramond, Embedded Artificial Neural Network for Data Prediction in Efficient Wireless Sensors Networks, ASPAI 2022

The bio-inspired approach at LEAT
31

32
Electronics Cognitive Neurosciences
ebrAIn
Embedded Bio-inspiRed Artificial Intelligence and Neuromorphic systems
Spiking Networks
Brain plasticity
Self-Organization
Neuromorphic
Embedded AI
Smart IoT
Bio-inspired Artificial Intelligence

 Spiking neural networks are the main subject of exploration in the domain of
bio-inspired computing.
 Main technical reasons:
• Impulsion coding
• Temporal integration operations
• Asynchronous behaviour
• Decentralized learning rules
• Bio-mimetic approach
• Main scientific questions:
1. How to code efficiently information in spikes ?
2. Define new neural models: How to train those networks ?
3. How to capture event-based data ?
Bio-inspired computing with Spiking Neural Networks
33
CNN vs SNN with Leaky Integrate and Fire neurons

34
 Rate coding
 Rate coding find the average spiking frequency of a neuron over a certain timeframe
 Time coding
 the neuron output is encoded in the temporal information of individual spikes.
 time to first spike – TTFS (A),
 rank order coding – ROC (B),
 latency coding (C)
Question 1: Spike coding
Ponulak, Filip & Kasiński, Andrzej. (2011). Introduction to spiking neural networks: Information processing, learning and applications. Acta
neurobiologiae experimentalis. 71. 409-33.

35
Question 2: Training spiking neural networks
G. Srinivasa, et. Al, TRAINING DEEP SPIKING NEURAL NETWORKS FOR ENERGY-EFFICIENT NEUROMORPHIC COMPUTING, ICASSP, 2020
1
STDP
2
Conversion
3
Spike
Backprop

36
Question 3. How to capture event-based data
DeepSee: industrial ANR Project
[2] Learning from event cameras with sparse spiking convolutional networks, Loïc
CORDONE, Sonia FERRANTE, Benoît Miramond; IJCNN 2021
Event-based cameras (EBC)
Main technical reasons
• Event-based representation
• Sparse inputs
• High temporal sensibility (μs)
• High Dynamic Range (HDR)
Main scientific questions
• How to train SNN from event-based data ?
• How to take advantage of input sparsity ?
Application in image processing
Classification / Object Detection / Optical flow …
Main scientific results
SNN with sparse convolutions [2]
First Spiking network for Object Detection on EBC [3]
[3] Object Detection with Spiking Neural Networks on Automotive Event
Data, Loïc CORDONE, Benoît Miramond; IJCNN 2022

Comparison between CNN and SNN
An Analytical Estimation of Spiking Neural Networks Energy Efficiency , Edgar Lemaire, Loïc Cordone, Andrea Castagnetti, Pierre-Emmanuel
Novac, Jonathan Courtois and Benoît Miramond, 29th International Conference on Neural Information Processing (ICONIP 2022)
CIFAR 10 GSC NCARS
Energy consumption
reduction
(ASIC 45 nm)
7.85 x 6.25 x 8.02 x
Spike Rate (vs. CNN) 0.1 0.14 0.08
37

39
 The combination of Edge AI and sensors
 makes AI to the contact of the physics of the real world
 Addresses the question of the energy consuption reduction of AI
 By bringing AI closer to the sensor, the goal is
 To reduce the amount of data to communicate
 To lower the global energy consumption of the digital infrastructure
 To reduce latency for decising making (close or open loop)
 Original approach and promising results on bio-inspired AI thanks to
 Greater sparsity
 Event-based processing (specific neuromorphic hardware)
 Reduced power consumption
 And a large amount of unexplored features in the brain
 Remaining challenges for
 EdgeAI training
 Neuromorphic architectures
 Realistic application demonstrations
Conclusion

40
The field of possibilities is only limited by your imagination
EdgeAI, let’s play !
IDEX Sith project, F. Ferrero, L. Rodriguez, B. Miramond
Plug
Train
Embed
Play
Repeat …

« l'organisation, la chose organisée, l'action d'organiser, et le résultat sont inséparables ».
Paul Valéry
Questions ?
LEAT Lab, eBrain group:
https://leat.univ-cotedazur.fr/ebrain/
41

Edge AI Miramond technical seminCERN.pdf

More Related Content

Similar to Edge AI Miramond technical seminCERN.pdf (20)

Recently uploaded (20)

Edge AI Miramond technical seminCERN.pdf