"Accessing Advanced Image Processing Feature Sets with Alvium Cameras Using a V4L2/GenICam Hybrid Driver," a Presentation from Allied Vision
0 likes678 views
The document discusses accessing advanced image processing with Alvium cameras using a V4L2/Genicam hybrid driver, highlighting the benefits of combining both interfaces for flexibility and efficiency. It covers the challenges faced in embedded vision systems and presents the Alvium camera series, which features integrated on-board processing and various connectivity options. The conclusion emphasizes the freedom of choice in utilizing popular APIs for advanced image processing in new applications.
"Accessing Advanced Image Processing Feature Sets with Alvium Cameras Using a V4L2/GenICam Hybrid Driver," a Presentation from Allied Vision
- 1. © 2019 Allied Vision Technologies GmbH Accessing Advanced Image Processing Feature Sets with Alvium Cameras Using a V4L2/GenICam Hybrid Driver Sebastian Guenther May 2019
- 2. © 2019 Allied Vision Technologies GmbH Corporate Information 2 • Industrial and Machine Vision camera supplier since 1989 • >300 employees • Headquarters in Germany • Worldwide sales and distribution
- 3. © 2019 Allied Vision Technologies GmbH 3 Embedded Vision Systems and their pain points
- 4. © 2019 Allied Vision Technologies GmbH Typical Embedded Vision System 4 SW Framework V4L2 interface User libs BSPDriver Application Use Case Kernel // Image sensor // MIPI CSI-2 interface // Host System // Computer on Module
- 5. © 2019 Allied Vision Technologies GmbH Key challenges CSI-2 driver development 5 Integration efforts Expertise Functionality and Registers Timing / Behavior Image pipeline Components Image Sensor Host HW (SoC, ISP, etc.) Image processing User interfaceCamera API SW Framework Algorithms / calculations • Sensor specific design • Incomplete data sheets • Redesign exchanging sensor • Know-how of SoC internals required • Lack of information • Limited ISP and its accessibility • Camera interface (V4L2) might be limited and custom controls required • Lack of convenience for complex devices 1 2 3 Pain points
- 6. © 2019 Allied Vision Technologies GmbH Alvium camera series 6 Integrated on-board image processing Multiple housing and mount options Cost optimized design Different options of integrated sensors Easy hardware and software integration MIPI CSI-2 and USB-3 interface Intelligent power management Industrial grade 1 2
- 7. © 2019 Allied Vision Technologies GmbH 7 Video for Linux interface and GeniCam standards
- 8. © 2019 Allied Vision Technologies GmbH V4L2 for CSI-2 cameras 8 Video for Linux User interface exposed by controls via the kernel driver Basic image processing functions (e.g. contrast, sharpness) Camera controls for focus, zoom or flash Codec support for MPEG and JPEG image compression Complex devices (sensor + SoC ISP) handled by Media Control API
- 9. © 2019 Allied Vision Technologies GmbH EMVA standards overview 9 Embedded Vision Interface EMVA 1288 GenAPI GenCP SFNC GenTL • EMVA - European Machine Vision Association • EMVA 1288: sensor/camera specification and measurement • GenICam: Generic Camera Interface • Embedded Vision Interface: new EMVA standard initiative* *note: name to be changed
- 10. © 2019 Allied Vision Technologies GmbH GenICam GenAPI standard • Mechanism to provide Generic API • Node map with defined interfaces, describing interdependencies • Camera provides self-describing XML • Describe camera's capabilities and features 10
- 11. © 2019 Allied Vision Technologies GmbH GenICam GenTL standard • GenTL: Generic Transport Layer • Low-level API to provide standard access to cameras • Independent from camera's features • Responsibilities: • Enumerate devices • Access device registers • Stream image data • Asynchronous events 11 SW Framework GenTL Producer Consumer BSPInterface driver Application Use Case Kernel
- 12. © 2019 Allied Vision Technologies GmbH V4L2 vs. GenICam interface 12 V4L2 GenICam Ease of use • Lean interface, easy to handle, but low level programming • With Media Control and Request API handling becomes more complex • Low level interface with GenTL producer and convenience level with consumer implementation • Complex software stack with many standards and definitions Capabilities • Controls for basic image processing features • Camera controls for focus, zoom, flash and mechanical control • Support for MPEG codec control and JPEG compression • SFNC defines ~800 features • High variety of camera controls to do image processing available • Features for sequencer mode, various types of trigger or control of 3D cameras available Resources • Interface exposed by drivers located within the kernel • Lean and resource friendly due to usage of kernel components • The concept of GenAPI XML file requires space of several MB either on the camera or host Standardization • Controls are not standardized, but V4L2 interface is mainlined • Cameras with additional features require custom controls • Very well established standards available, with SFNC for features, GenTL and GenAPI for camera interface star rating: the more, the better…
- 13. © 2019 Allied Vision Technologies GmbH 13 Hybrid driver approach combining V4L2 and GenICam
- 14. © 2019 Allied Vision Technologies GmbH Allied Vision's hybrid driver approach 14 Idea: combine benefits and offer both interfaces with one driver Access camera via V4L2 or GenICam based applications Existing V4L2 or GenICam code can be re-used GenICam interface enables use of Allied Vision's camera SDK Vimba Support for NXP platforms: i.MX6 and i.MX8 family (QuadMax, 8M, 8X) Support for Nvidia platforms: Jetson TX2, AGX Xavier and Nano
- 15. © 2019 Allied Vision Technologies GmbH Hybrid driver System overview with hybrid driver and Alvium camera series 15 Producer ConsumerApplication sysfs Custom IOCTLs Alvium camera series 64bit GenTL CSI lib ExtendedBasicCommon GenCP Register Map 16bit user kernel V4L2 V4L2 IOCTLs
- 16. © 2019 Allied Vision Technologies GmbH Using GenICam mode with hybrid driver and Alvium camera series 16 • Distinct camera modes for V4L2 and GenICam to enable Vimba SDK usage • Input and Output buffer register to exchange control data with camera • Handshake register to ensure camera is ready and observe status Alvium camera series Mode of operation Handshake Input / Output Buffer
- 17. © 2019 Allied Vision Technologies GmbH Example: configure Multi ROI in Vimba and stream images with V4L2 17 App Hybrid driverV4L2 Alvium Open camera Setup Exposure Time, White Balance Switch mode to GenICam Setup Multi ROI Switch mode to V4L2 Start streaming V4L2
- 18. © 2019 Allied Vision Technologies GmbH 18 Conclusion
- 19. © 2019 Allied Vision Technologies GmbH Conclusion – What's in for me? 19 V4L2: popular API for embedded Advanced image processing with GenICam Conquer new applications!Your freedom of choice!
- 20. © 2019 Allied Vision Technologies GmbH Thank you! 20 Corporate information https://www.alliedvision.com GenICam standards https://www.emva.org/standards- technology/genicam/ Visit our booth at #412 Embedded Vision Summit “Accessing Advanced Image Processing Feature Sets with Alvium Cameras Using a V4L2/GenICam Hybrid Driver” Details (Time, Date, etc.) Speaker Sebastian Guenther Sebastian.guenther@alliedvision.com
- 21. © 2019 Allied Vision Technologies GmbH 21 Back-up Slides
- 22. © 2019 Allied Vision Technologies GmbH GenICam SFNC standard • SFNC: Standard Feature Naming Convention • Vendor independent feature access • Standard behavioral model for devices • Feature control groups: • Acquisition and Trigger • Color Transformation • Sequencer • etc. 22
- 23. © 2019 Allied Vision Technologies GmbH GenICam GenCP standard • GenCP: Generic Control Protocol • Read/write register protocol • Bootstrap Register Map (BRM) • Packet based communication • Protocol mechanisms: • Command and ACK • Pending Acknowledge • Message channels • etc. 23
- 24. © 2019 Allied Vision Technologies GmbH Vimba • Allied Vision's SDK for GigE Vision, USB3 Vision, IEEE 1394, Camera Link and MIPI CSI-2 cameras • GenICam compliant • Platform independent (Windows, Linux and Linux ARM) • APIs for C, C++ and .NET • Connectivity to 3rd party software, e.g. MATLAB, OpenCV, Halcon and Common Vision Blocks 24 USB3 Vision IEEE 1394 GigE Vimba Core C API C++ API .NET API Camera Link CSI-2
- 25. © 2019 Allied Vision Technologies GmbH Alvium camera series 25 CPU GPU Image Signal Processor VPU Co Processors CSI-2 D-PHY RX HDMIEmbedded Board DSP Peripherals Image Signal Processor Flexible Sensor CSI-2 D-PHY TXCamera Module • Save host resources by shifting image processing to MIPI CSI-2 camera module
- 26. © 2019 Allied Vision Technologies GmbH Alvium camera series Example: Multi ROI Feature 26 Increased contrast Applied RGB look-up table • ROI (Region of Interest) • Less pixel transferred, FPS increased • Individual settings for each Region