Open hours home security | Episode 70
- 1. Developing with Qualcomm® Snapdragon™ Processors Matthew Kneiser, Sr. Engineer Rajan Mistry, Engineer II Qualcomm Technologies, Inc. 10 am to 1 pm, 26th July, 2017 Qualcomm Snapdragon is a product of Qualcomm Technologies, Inc. Training
- 2. Agenda • DragonBoard™ development Platforms • Security Camera applications • OpenCV • Hands on Demo • Q&A • Online Resources DragonBoard 410c is a product of Arrow Electronics.
- 3. DragonBoard 410c from Arrow Electronics With Snapdragon 410E (APQ8016E Application Processor)
- 4. DragonBoard development platform Compliant with the 96Boards.org specification, utilizing the 96Boards community DragonBoard development boards are tools for • General purpose development or vertical specific development using one of the many mezzanine boards in the 96Boards ecosystem • Evaluation, development, reference platform for end customers – Schematics public, and design files available via Arrow • Universities, hackers, hobbyists, and ISV’s enabling a broader SW ecosystem
- 5. DragonBoard 410c Form Factor and 96Boards Compliance Specifications SOC Components APQ8016E (Application Processor) PM8916 (Power Management) with integrated codec WCN3620/WCN3660B (WLAN/BT) WGR7640 (Location) Software Android, Linux, Windows 10 IoT, Ubuntu Core Memory/Storage 1GB LPDDR3 8GB eMMC 4.5 Micro SD card slot Camera support MIPI camera serial interface – up to 2 CSIs Interfaces 1x DC Jack, 1x USB 2.0 micro B 2x USB 2.0 type A (host mode only) 1x uSD slot, 1 x HDMI Type A (full) 1x 40 pin low speed (LS) expansion connector (2x UART, SPI, 2x I2C, I2S, 12x GPIOs, Voltages) 1x 60 pin high speed (HS) expansion connector USB, SPI, I2C, 2x MIPI CSI (4-lane, 2-lane)), MIPI DSI (4-lane) 1x audio expansion connectors APQ8016E, PM8916, WCN3620, WGR7640 and APQ8064E are products of Qualcomm Technologies, Inc.
- 6. Designed for performance in embedded devices Snapdragon processors designed specifically for embedded applications Snapdragon 410E CPU: 1.2 GHz quad-core ARM v8 Cortex-A53, 32/64-bit capable Connectivity: Integrated Wi-Fi, Bluetooth 4.xLE and GPS DSP: Qualcomm® Hexagon™ DSP Graphics: Qualcomm® Adreno™ 306 400MHz GPU Interfaces: 1x USB2.0, 2x MIPI-CSI, MIPI- DSI, SD3.0 & eMMC v4.5with DDR support 410E (8016E) 802.11b/g/n Up to 13MP 1920x1080 Qualcomm Hexagon and Qualcomm Adreno are products of Qualcomm Technologies, Inc.
- 7. CommercializePrototypeEvaluate and develop DragonBoard: Built with path to commercialization https://developer.qualcomm.com/hardware/snapdragon-410e • Regional & global presence • Services • Testing / certification • General support SOMs & SBCs • Off-the-shelf or custom HW solutions • Software • Design and Manufacturing
- 9. Camera Subsystem Key aspects: Quality of images Frame rate Development platform Key aspects: Processing resources Library support Connectivity to Cloud platform Core Components DragonBoard 410c Supports: • 2xMIPI-CSI interfaces • MIPI-DSI • 1920x1080 • Up to 13MP DragonBoard 410c Supports: • 1.2 GHz quad-core ARM v8 Cortex-A53 64 bit capable • 1080p at 30fps with H.264(AVC) • 720p playback with H.265(HEVC) • V4l2a driver support DragonBoard 410c Supports: • 802.11 b/g/n • Bluetooth • GPS
- 10. Confidential and Proprietary – Qualcomm Technologies, Inc. Security Camera / Home Security Demo DragonBoard 410c Sensors Mezzanine Optiona l AWS Pan/Tilt Camera Mount USB Camera OpenCV computer vision library Smart security camera application 1. Detect Faces using OpenCV 2. Train the software to detect your face 3. Recognize known faces 4. Use the dual servo mount and USB webcam to track one of the known faces.
- 11. OpenCV
- 12. Installing OpenCV libraries Option 1: Use package manager Option 2: Cmake • Easy to install • sudo apt-get install python-opencv • Less control • Not as straightforward • Needs more resources • More control • Compiled for ARM specific processor features and instruction sets
- 13. Why Optimize OpenCV? OpenCV Installation Camera Resolution Per Frame Processing Time (sec) Frames Per Second Using Package Manager 1280 x 720 ~0.12 ~8.333 Custom Install 1280 x 720 ~0.065 ~15.385 Custom Install 320x240 ~0.021 ~47.619 Custom Install 160x120 ~0.005 ~200 0 50 100 150 200 1280 x 720 1280 x 720 320x240 160x120 FPS Source: Internal QTI test results
- 14. OpenCV Face Detection Basics Face Detection basics in OpenCV Train Detect Host PC Recognize Application on Embedded System
- 15. • Cascade classifier options 1. Train with Haar features 2. Train with LBP (Local Binary Pattern) features More suited to Embedded environment as features are integers. Processing is a lot faster compared to Haar features. • Resources Tutorial for training cascades OpenCV users guide for training cascades OpenCV Face Detection Basics
- 16. OpenCV Face Recognition Types of FaceRecognizer class algorithms • Eigenfaces • Fisherfaces • Local Binary Patterns Histograms • Resources Face Recognition Reference Face Recognition Algorithms in OpenCV
- 18. Lets have some fun! 1. Run face detection with Haar cascade classifiers 2. Run face detection with LBP cascade classifiers 3. Compare the performance results 4. Quality of the classifier detection depends on the training data. It is possible to train a LBP-based classifier that can provide almost the same quality as Haar based classifier. • Resources • Security Camera Project posted on 96Boards Blog posted on 96Boards
- 22. Training Classifiers using open source datasets 1. Leverage open source training data 2. Use tools to automate the process 3. Create Positive training images a) Find images and generate training and testing dataset b) Extract the training images from the master dataset 4. Create Vec file using OpenCV 5. Train the Cascade classifier using OpenCV
- 23. Experience Detecting Custom Objects with OpenCV Matthew Kneiser, Senior Engineer
- 24. The task: detect 6 masks Seems simple, right?
- 25. Step 1: Collect a bunch of positives
- 26. Step 2: Collect a bunch of negatives
- 27. Step 2: Make the images uniform
- 28. Progress was slow at first
- 29. Improving detection was an art • Crop the positive images to just the target • Increase the number of positive images • Use tools to generate more positives • Remove large negative images Things that helped
- 30. Generate more positives - using tools to be clever
- 31. After tweaking
- 32. Online Resources
- 33. Qualcomm Development Network (QDN) in detail • Snapdragon Documentation • DragonBoard Documentation • Tools and Resources • Projects Live walkthrough on https://developer.qualcomm.com/
- 34. 96Boards.org in detail • SD Card Images • Documentation and Getting Started • Community Forums! • Mezzanine Boards! Live walkthrough on http://www.96boards.org/product/dragonboard410c/
- 35. DragonBoard 410c community today Purchase Software and support forums Documentation and tutorial videos www.arrow.com/dragonboard410c www.96boards.org developer.qualcomm.com
- 36. Thank you All data and information contained in or disclosed by this document is confidential and proprietary information of Qualcomm Technologies, Inc. and/or its affiliated companies and all rights therein are expressly reserved. By accepting this material the recipient agrees that this material and the information contained therein is to be held in confidence and in trust and will not be used, copied, reproduced in whole or in part, nor its contents revealed in any manner to others without the express written permission of Qualcomm Technologies, Inc. Nothing in these materials is an offer to sell any of the components or devices referenced herein. ©2017 Qualcomm Technologies, Inc. and/or its affiliated companies. All Rights Reserved. Qualcomm, Snapdragon and DragonBoard are trademarks of Qualcomm Incorporated, registered in the United States and other countries. Other products and brand names may be trademarks or registered trademarks of their respective owners. References in this presentation to “Qualcomm” may mean Qualcomm Incorporated, Qualcomm Technologies, Inc., and/or other subsidiaries or business units within the Qualcomm corporate structure, as applicable. Qualcomm Incorporated includes Qualcomm’s licensing business, QTL, and the vast majority of its patent portfolio. Qualcomm Technologies, Inc., a wholly-owned subsidiary of Qualcomm Incorporated, operates, along with its subsidiaries, substantially all of Qualcomm’s engineering, research and development functions, and substantially all of its product and services businesses, including its semiconductor business, QCT. Follow us on: For more information, visit us at: www.qualcomm.com & www.qualcomm.com/blog
Editor's Notes
- #7: We chose 600E and 410E to provide options for wide-ranging use cases 600E featuring 1.5GHz quad-core Qualcomm® Krait™ 300 CPU is the ideal processor for building advanced systems with multi-core performance and immersive 3D graphics, thanks to the Qualcomm® Adreno™ 320 GPU and Qualcomm® Hexagon™ DSP It supports integrated Bluetooth 4.0/LE & 3.x, 802.11 a/b/g/n/ac and GPS for connected applications. It is also highly expandable for a variety of use cases with SD3.0, DDR memory, eMMC storage, HDMI, LVDS, HSIC, and PCIe interfaces, making is especially desirable for industrial applications 410E 1.2GHz quad-core processor offers high performance, low power consumption and rich multimedia with an Adreno 306 GPU and Hexagon DSP It supports Bluetooth 4.1/LE, 802.11 b/g/n and GPS It is ideal for IoT use cases in smart homes, digital signage, medical equipment, industrial automation, digital media players and smart surveillance.
- #36: Purchase now at Arrow -- arrow.com/dragonboard410c Learn more now at 96Boards.org