SlideShare a Scribd company logo
uTensor
Neural Network Inference Library for MCU
github.com/uTensor
uTensor Team
FOSDEM presentation by Neil Tan
About Me
● Data Scientist @ Pinkoi
● Python Programmer
○ utensor_cgen
Agenda
● Tensorflow Toward the Edge
● Automatic Code Generator
○ CNN demo
Tensorflow v.s uTensor
Tensorflow uTensor
We Are All Set!
From This
To This
[COSCUP 2018] uTensor C++ Code Generator
[COSCUP 2018] uTensor C++ Code Generator
[COSCUP 2018] uTensor C++ Code Generator
[COSCUP 2018] uTensor C++ Code Generator
[COSCUP 2018] uTensor C++ Code Generator
[COSCUP 2018] uTensor C++ Code Generator
[COSCUP 2018] uTensor C++ Code Generator
我
Automatic Code Generator (utensor-cli)
github.com/uTensor/utensor_cgen
utensor-cli simple_mnist.pb --output-nodes=y_pred
[COSCUP 2018] uTensor C++ Code Generator
The Code
[COSCUP 2018] uTensor C++ Code Generator
End-to-End Example
1. MLP: end_to_end.ipynb
2. CNN: deep_cnn.ipynb
Basic Pipeline
1. Training with Tensorflow
2. Save trained model (.pb file)
3. uTensor cli
a. model.pb ---> uTensor cpp/hpp files
Status Quo
● Cli support
○ Multi layer Perceptron (MLP)
○ Convolution Neural Network (CNN)
● Performance/Memory Optimization
○ Memory Optimization: Reference Counting
○ Graph Optimization
■ Dropout layer removal
?
CNN Demo
[COSCUP 2018] uTensor C++ Code Generator
Guide
Q & A

More Related Content

PDF
tokyotalk
PDF
Torch7 and ConvNet
PDF
Torch intro
PDF
Using R in remote computer clusters
PDF
Buzzwords Numba Presentation
PDF
Parallel computing with GPars
PDF
Be a Zen monk, the Python way
ODP
Event Driven with LibUV and ZeroMQ
tokyotalk
Torch7 and ConvNet
Torch intro
Using R in remote computer clusters
Buzzwords Numba Presentation
Parallel computing with GPars
Be a Zen monk, the Python way
Event Driven with LibUV and ZeroMQ

What's hot (20)

PDF
Learning Erlang (from a Prolog dropout's perspective)
PDF
Python lecture 02
PDF
memcached Binary Protocol in a Nutshell
PDF
Concurrency
PDF
Golang design4concurrency
ODP
bup backup system (2011-04)
PDF
Yufeng Guo - Tensor Processing Units: how TPUs enable the next generation of ...
PDF
Go on!
PDF
Golang concurrency design
PDF
It's just a Web server - a plea for simplicity
PPTX
#2 (UDP)
PDF
Automatically Fusing Functions on CuPy
PDF
Introduction to nand2 tetris
PDF
Logging kernel oops and panic
PDF
IFB cloud: Integration of snakemake workflows in an appliance designed for Ch...
PDF
Gameboy emulator in rust and web assembly
PDF
FlowSim_presentation
PDF
2010 08-19-30 minutes of python
PPTX
Your data isn't that big @ Big Things Meetup 2016-05-16
PDF
Some Tricks in Using Terminal - KienDT
Learning Erlang (from a Prolog dropout's perspective)
Python lecture 02
memcached Binary Protocol in a Nutshell
Concurrency
Golang design4concurrency
bup backup system (2011-04)
Yufeng Guo - Tensor Processing Units: how TPUs enable the next generation of ...
Go on!
Golang concurrency design
It's just a Web server - a plea for simplicity
#2 (UDP)
Automatically Fusing Functions on CuPy
Introduction to nand2 tetris
Logging kernel oops and panic
IFB cloud: Integration of snakemake workflows in an appliance designed for Ch...
Gameboy emulator in rust and web assembly
FlowSim_presentation
2010 08-19-30 minutes of python
Your data isn't that big @ Big Things Meetup 2016-05-16
Some Tricks in Using Terminal - KienDT
Ad

Similar to [COSCUP 2018] uTensor C++ Code Generator (20)

PPTX
uTensor COSCUP
PPTX
Machine Learning on Microcontroller for Lora using uTensor
PPTX
GPU and Deep learning best practices
PDF
Deep learning with C++ - an introduction to tiny-dnn
PDF
Introduction To Using TensorFlow & Deep Learning
PDF
High Performance Distributed TensorFlow with GPUs - TensorFlow Chicago Meetup...
PPTX
Dp2 ppt by_bikramjit_chowdhury_final
PDF
OpenPOWER Workshop in Silicon Valley
PDF
NeuralProcessingofGeneralPurposeApproximatePrograms
PDF
Towards neuralprocessingofgeneralpurposeapproximateprograms
PDF
Optimize + Deploy Distributed Tensorflow, Spark, and Scikit-Learn Models on GPUs
PDF
Anirudh Koul. 30 Golden Rules of Deep Learning Performance
PDF
ML in Android
PDF
06.09.2017 Computer Science, Machine Learning & Statistiks Meetup - MULTI-GPU...
PDF
Neural Networks from Scratch - TensorFlow 101
PDF
Netflix machine learning
PPTX
TPU paper slide
PPTX
Tensor Processing Unit (TPU)
PPTX
Automatic Attendace using convolutional neural network Face Recognition
PDF
Icpp power ai-workshop 2018
uTensor COSCUP
Machine Learning on Microcontroller for Lora using uTensor
GPU and Deep learning best practices
Deep learning with C++ - an introduction to tiny-dnn
Introduction To Using TensorFlow & Deep Learning
High Performance Distributed TensorFlow with GPUs - TensorFlow Chicago Meetup...
Dp2 ppt by_bikramjit_chowdhury_final
OpenPOWER Workshop in Silicon Valley
NeuralProcessingofGeneralPurposeApproximatePrograms
Towards neuralprocessingofgeneralpurposeapproximateprograms
Optimize + Deploy Distributed Tensorflow, Spark, and Scikit-Learn Models on GPUs
Anirudh Koul. 30 Golden Rules of Deep Learning Performance
ML in Android
06.09.2017 Computer Science, Machine Learning & Statistiks Meetup - MULTI-GPU...
Neural Networks from Scratch - TensorFlow 101
Netflix machine learning
TPU paper slide
Tensor Processing Unit (TPU)
Automatic Attendace using convolutional neural network Face Recognition
Icpp power ai-workshop 2018
Ad

Recently uploaded (20)

PPTX
Sustainable Sites - Green Building Construction
PPTX
M Tech Sem 1 Civil Engineering Environmental Sciences.pptx
PPTX
additive manufacturing of ss316l using mig welding
PPTX
CH1 Production IntroductoryConcepts.pptx
PPTX
Welding lecture in detail for understanding
PPTX
OOP with Java - Java Introduction (Basics)
PPTX
Engineering Ethics, Safety and Environment [Autosaved] (1).pptx
PPT
Mechanical Engineering MATERIALS Selection
PPTX
CYBER-CRIMES AND SECURITY A guide to understanding
PDF
July 2025 - Top 10 Read Articles in International Journal of Software Enginee...
PDF
Enhancing Cyber Defense Against Zero-Day Attacks using Ensemble Neural Networks
PDF
The CXO Playbook 2025 – Future-Ready Strategies for C-Suite Leaders Cerebrai...
PDF
Model Code of Practice - Construction Work - 21102022 .pdf
DOCX
573137875-Attendance-Management-System-original
DOCX
ASol_English-Language-Literature-Set-1-27-02-2023-converted.docx
PPTX
KTU 2019 -S7-MCN 401 MODULE 2-VINAY.pptx
PDF
BMEC211 - INTRODUCTION TO MECHATRONICS-1.pdf
PDF
Operating System & Kernel Study Guide-1 - converted.pdf
PPTX
Recipes for Real Time Voice AI WebRTC, SLMs and Open Source Software.pptx
PDF
Mohammad Mahdi Farshadian CV - Prospective PhD Student 2026
Sustainable Sites - Green Building Construction
M Tech Sem 1 Civil Engineering Environmental Sciences.pptx
additive manufacturing of ss316l using mig welding
CH1 Production IntroductoryConcepts.pptx
Welding lecture in detail for understanding
OOP with Java - Java Introduction (Basics)
Engineering Ethics, Safety and Environment [Autosaved] (1).pptx
Mechanical Engineering MATERIALS Selection
CYBER-CRIMES AND SECURITY A guide to understanding
July 2025 - Top 10 Read Articles in International Journal of Software Enginee...
Enhancing Cyber Defense Against Zero-Day Attacks using Ensemble Neural Networks
The CXO Playbook 2025 – Future-Ready Strategies for C-Suite Leaders Cerebrai...
Model Code of Practice - Construction Work - 21102022 .pdf
573137875-Attendance-Management-System-original
ASol_English-Language-Literature-Set-1-27-02-2023-converted.docx
KTU 2019 -S7-MCN 401 MODULE 2-VINAY.pptx
BMEC211 - INTRODUCTION TO MECHATRONICS-1.pdf
Operating System & Kernel Study Guide-1 - converted.pdf
Recipes for Real Time Voice AI WebRTC, SLMs and Open Source Software.pptx
Mohammad Mahdi Farshadian CV - Prospective PhD Student 2026

[COSCUP 2018] uTensor C++ Code Generator