“The OpenVX Standard API: Computer Vision for the Masses,” a Presentation from the Khronos Group

The OpenVX Standard API:
Computer Vision for the Masses
Kiriti Nagesh Gowda
OpenVX Chair | The Khronos Group
Sr. Staff Engineer | AMD

© 2023 The Khronos Group
• OpenVX - Open, Royalty-free Standard for Computer Vision
• OpenVX 1.3.1 - An Overview
• Conformance
• Vendor Applications - Case Study
• Open-Source Cross-Platform Application - Case Study
• OpenVX - Future Work
• Summary
Agenda
2

© 2023 The Khronos Group 3
OpenVX - Open, Royalty-free Standard
for Computer Vision

Open, Royalty-free Standard for Computer Vision
4
OpenVX™ is an open, royalty-free API standard for a cross-platform acceleration
of computer vision applications
High-level graph-based abstraction for portable, efficient vision processing

5
API
What is an API?
Application Programming Interface
Why APIs are important
• Building Blocks • Speeds up development • Portability • Innovation
OpenVX enables portable, performance, and power-optimized computer vision
processing, especially important in embedded and real-time use cases

6
CPU / GPU / DSP / Any Vendor Hardware

7
OpenVX™ Graph Framework
• Fuse nodes when possible to achieve better overall performance
• Auto graph-level memory optimizations to achieve a low memory footprint
• Deployed on a wide range of computer hardware, including small embedded CPUs, ASICs,
APUs, discrete GPUs, and heterogeneous servers

8
OS Support: Linux, Windows, Android, iOS, Raspbian, Embedded OS, no OS
Vision
Node
Vision
Node
Vision
Node
Downstream
Application
Processing
CNN Nodes
NNEF Import converts a trained
neural network into OpenVX graph
Layers are represented as OpenVX nodes
OpenVX Graph
Open Source
Convertors
https://github.com/KhronosGroup/NNE
F-Tools
Stable Specification

OpenVX 1.3.1 - An Overview

OpenVX 1.3.1 - An Overview
10
The defined OpenVX 1.3.1 feature sets include:
• Graph Infrastructure - baseline for other feature sets
• Vision - core vision functionality
• Enhanced Vision - functions introduced in OpenVX 1.2
• Neural Network Inferencing - including tensor objects
• NNEF Kernel Import - including tensor objects
• Binary Images – one bit image processing
• Safety Critical - reduced features to enable easier safety certification

OpenVX 1.3.1 - Highlights
11
• Three Conformance Feature Sets:
• Vision - OpenVX 1.1 equivalent vision functions
• Neural Network - OpenVX 1.2 equivalent neural-network functions, plus the neural network extension,
and the tensor object
• NNEF Import - kernel import plus the tensor object
• Two Optional Feature Sets:
• U1 - binary image support
• Enhanced Vision - vision functions introduced in OpenVX 1.2
• One Organizational Feature Set:
• Base Feature Set - basic graph infrastructure
• One Informational Feature Set:
• Deployment Feature Set - for safety critical usage

OpenVX - Conformance

13
• OpenVX Work Group provides comprehensive conformance test suite
• Implementations must pass exhaustive conformance test suite to be conformant
• Hardware vendors provide optimized Conformant Implementation of OpenVX drivers
• Architected to get the best performance from their silicon architecture
• Ready for developers to use

14
Conformant Implementations of OpenVX from the following vendors

OpenVX - Newest Contributing Member
15
Khronos Group Welcomes Robert Bosch GmbH as Contributor Member
to OpenVX Working Group
• BOSCH working with the OpenVX Group to add multiple extensions to OpenVX

OpenVX - Vendor Applications: A Case Study

OpenVX - Vendor Applications Case Study
17
Application: Object detection with neural networks
YoloV2
Graph
Pre-Processing
Classification
Localization
Annotation

18
Application: Reliable motion detection
• Uses OpenVX API internally for accelerating algorithm
on custom HW blocks
• Compute heavy algorithm for reliable motion
detection
Before OpenVX:
• Hand optimized custom assembler by algorithm
developers
After OpenVX:
• Algorithm developers “draw” algorithms as graphs
• Driver developers implement the needed graph API

19
Graph

20
Application: Background subtraction for video security
0%
15%
30%
45%
60%
75%
0 150 300 450 600
Memory latency (cycles)
Graph Speed-up
AccumWt
AbsDiff
Dilate
Erode
Thresh
BG
Resize

21
Application: Front camera ADAS
CNN
inference
Graph

An OpenVX Cross-Platform Application:
A Case Study

OpenVX Cross-Platform Application: A Case Study
23
Open-Source OpenVX Samples
Open-Source OpenVX sample applications, to use with any conformant
implementation of OpenVX available on GitHub

24
On Raspberry Pi 4 Model B Rev 1.2
SkinTone Detector Sample
On X86 Processor Windows
On X86 Processor Linux On MacOS
* using AMDs open-sourced MIVisionX for OpenVX Libraries
* using open-source OpenVX Raspberry Pi Implementation for OpenVX Libraries

25
Performance
* using AMDs open-sourced MIVisionX for OpenVX Libraries
0.
0.7
1.3
2.
2.6
OpenVX Graph Unoptimized OpenVX Graph Optimized
1X
2.54 X
Memory Footprint
SkinTone Detector Sample
0.
0.3
0.5
0.8
1.
OpenVX Graph Unoptimized OpenVX Graph Optimized
1X
0.234X

OpenVX - Future Work

27
• OpenVX relies on external libraries for input
• may not be supported across required operating systems
• may be too large for embedded systems
• have different functionality gaps
• OpenVX can use a simple camera capture API library that works everywhere
• Linux, Windows, Android, iOS, Raspbian, Embedded OS, no OS
• initialize()/reinitialize(), capture(), release(), getAttributes(): size, format, etc.
• Once the image is captured, OpenVX will wrap it (not copy it) into an vx_image object
OpenVX Camera Capture Extension

28
OpenVX Camera Capture Extension

OpenVX - Summary

Summary
30
• OpenVX is unique in being the only vision API shipped as an optimized driver
• OpenVX delivers performance comparable to hand-optimized, non-portable code
• Acceleration on a wide range of vision hardware architectures
• OpenVX provides a high-level graph-based abstraction
• Enables graph-level optimizations
• Can be implemented on almost any hardware or processor
• Portable, Efficient Vision Processing!

Acknowledgement
Thanks To
• Mike Schmit – Director of Software Engineering, AMD
• Neil Trevett – President, The Khronos Group
• AMD’s MIVisionX Team
• OpenVX Working Group
• Embedded Vision Summit 2023 Organizers
31

Resource Slide
32
OpenVX Resources - Khronos
Sample Implementation:
https://github.com/KhronosGroup/OpenVX-
sample-impl
Sample Applications:
https://github.com/KhronosGroup/openvx-
samples
Tutorial Material:
https://github.com/rgiduthuri/openvx_tutorial
Conformant Implementations
https://www.khronos.org/conformance/adopt
ers/conformant-products/openvx
Khronos OpenVX API Registry
https://www.khronos.org/registry/OpenVX/
OpenVX for Raspberry Pi
https://www.raspberrypi.org/blog/openvx-api-
for-raspberry-pi/
AMD ROCm MIVisionX - OpenVX
https://gpuopen-professionalcompute-
libraries.github.io/MIVisionX/

Disclaimers and attributions
33
The information presented in this document is for informational purposes only and may contain technical inaccuracies, omissions,
and typographical errors. The information contained herein is subject to change and may be rendered inaccurate for many
reasons, including but not limited to product and roadmap changes, component and motherboard version changes, new model
and/or product releases, product differences between differing manufacturers, software changes, BIOS flashes, firmware
upgrades, or the like. Any computer system has risks of security vulnerabilities that cannot be completely prevented or
mitigated. AMD assumes no obligation to update or otherwise correct or revise this information. However, AMD reserves the right
to revise this information and to make changes from time to time to the content hereof without obligation of AMD to notify any
person of such revisions or changes.
THIS INFORMATION IS PROVIDED ‘AS IS.” AMD MAKES NO REPRESENTATIONS OR WARRANTIES WITH RESPECT TO THE
CONTENTS HEREOF AND ASSUMES NO RESPONSIBILITY FOR ANY INACCURACIES, ERRORS, OR OMISSIONS THAT MAY APPEAR
IN THIS INFORMATION. AMD SPECIFICALLY DISCLAIMS ANY IMPLIED WARRANTIES OF NON-INFRINGEMENT,
MERCHANTABILITY, OR FITNESS FOR ANY PARTICULAR PURPOSE. IN NO EVENT WILL AMD BE LIABLE TO ANY PERSON FOR ANY
RELIANCE, DIRECT, INDIRECT, SPECIAL, OR OTHER CONSEQUENTIAL DAMAGES ARISING FROM THE USE OF ANY INFORMATION
CONTAINED HEREIN, EVEN IF AMD IS EXPRESSLY ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.
© 2023 Advanced Micro Devices, Inc. All rights reserved.
AMD, the AMD Arrow logo, EPYC, Radeon, ROCm and combinations thereof are trademarks of Advanced Micro Devices, Inc. Other
product names used in this publication are for identification purposes only and may be trademarks of their respective companies.

BACKUP SLIDES

OpenVX - Open-Source Tools

37
Scripting Tool - RunVX
• RunVX is a command-line tool to execute OpenVX graphs
• It encapsulates most of the routine OpenVX calls, thus speeding up development and enabling rapid prototyping.
• As input, RunVX takes a GDF (Graph Description Format) file, a simple and intuitive syntax to describe the various
data, nodes, and dependencies.
• The tool has other useful features, such as, file read/write, data compares, image and keypoint data visualization, etc.
Open-Sourced on GitHub - https://github.com/GPUOpen-ProfessionalCompute-Libraries/MIVisionX

38
Graph Tool - openvx-graph-parser
Tool for OpenVX C code generation from graphical calculation graph definitions. Includes a Python based
parser which takes graphs defined in graphml format and generates valid OpenVX C code.
Open-Sourced on GitHub - https://github.com/AxisCommunications/openvx-graph-parser

39
Python Binding - PyVX
pycbindgen is a python based frontend tool for generating python module for a given C
library. It uses pycaparser and CFFI python modules underneath.
OpenVX functionality in Python
Will be Open-Sourced on GitHub - https://github.com/KhronosGroup

“The OpenVX Standard API: Computer Vision for the Masses,” a Presentation from the Khronos Group

More Related Content

Similar to “The OpenVX Standard API: Computer Vision for the Masses,” a Presentation from the Khronos Group (20)

More from Edge AI and Vision Alliance (20)

Recently uploaded (20)

“The OpenVX Standard API: Computer Vision for the Masses,” a Presentation from the Khronos Group