Introducing the ADSP BF609 Blackfin Processors

Editor's Notes

• #2: Hello and welcome to this introductory video to Analog Devices BF609 Dual-Core Blackfin Processor series. My name is Maikel Kokaly, DSP Applications Engineer at Analog Devices.
• #3: In this session, we will go through a brief overview of the BF609 Dual-core Blackfin Processor series, we’ll explore its main features, and we’ll also spend some time on the so called Pipelined Vision Processor (also refer to as PVP). Finally, we’ll have a quick look through the available Hardware and Software development tools for the BF609 processor series.
• #5: Within the Embedded Systems Products and Technology Group, we can find our two leading DSP architectures BLACKFIN and SHARC. Blackfin being a fixed point processor, while SHARC is a floating point device, although also capable of efficiently computing fixed point operations. Each of these architectures provide a fast and efficient core, containing large on-chip memory, and a rich peripheral set, which obviously varies from part to part.Blackfin is a 16-bit fixed point DSP with hardware support for single cycle execution of dual MAC operations (Multiply-Accumulate). With a broad range of products, starting at 1.99 USD, going all the way up to the new high-performance dual core devices, capable of operating at up to 500MHz per core, i.e. 2GMACs in total.Blackfin is extremely attractive in terms of power consumption, consuming as low as 28mW @100MHz operation. The Blackfin architecture combines DSP and microcontroller functionality on a single device, being able to efficiently compute complex DSP algorithms as well as handling today’s control and connectivity system requirements.Within its portfolio of products, Blackfin includes a number of devices which incorporate attractive on-chip integrated solutions like SPI or parallel flash devices, audio CODECs or an on-chip ADC.And then, there is also SHARC. A floating point DSP, again with dual MAC support, capable of operating at up to 450MHz. With a broad range of products, starting at 5$, SHARC is available in many different flavors. Low Power SHARCs suitable for computationally demanding portable solutions requiring the floating point performance, consuming as little as 250mW. And higher performance devices providing high speed interconnects and dedicated hardware accelerators for digital filter operations. In order to support both SHARC and Blackfin architectures, Analog Devices offers a comprehensive set of development tools : CrossCore Embedded Studio, our integrated development and debugging environment, JTAG In circuit emulators, and many evaluation platforms – EZKITs, EZ-Boards, and EZ-Extenders for the various processor series.
• #6: The Blackfin processor family offers scalable performance from the 200Mhz $1.99 DOLLAR BF592 Low Power Blackfin to the Dual Core 600MHz BF561 high-performance device for computationally demanding applications. Future devices will further extend ADI's leadership for performance and power efficiency while maintaining code compatibility, providing a wide range of processor options for demanding processing challenges.CLICK!!The most recent addition to the Blackfin processor portfolio is the Dual-Core BF609 processor series, with 1GHz core performance (500 MHz per core), an enhanced peripheral set, and a new dedicated vision accelerator that enables broad adoption of multi-function analytics into embedded vision applications. This new series comes in four different flavours: The BF608 and 609 optimized for embedded vision applications, featuring a high performance video analytics accelerator, called the Pipelined Vision Processor (PVP) (we’ll go through a detailed description of this later). These two processors are ideal for many applications such as automotive advanced driver assistance systems (also refer to as ADAS), industrial machine vision, and security/surveillance systems. The main difference between the 609 and 608 is the fact that 609 is capable of handling HD images (1280x960 pixels @ 30 f/s), while the BF608 can only do VGA type resolution (640x480 @ 30fps). Then we have the BF606 and BF607 both available without the PVP and designed to provide flexibility for a wide variety of general purpose DSP applications such as wireless communications, industrial process control, and electric power grid monitoring, just to name a few. The difference between these two parts is the amount of available L2 on-chip memory: 256KBs for the 607 and 128KBs for the 606.Next, we’ll go through a more detailed overview of this new Dual-Core Processor Series.
• #7: Listed here are the main highlights of the new BF60x Blackfin processor series.The PVP has been specifically designed for embedded vision applications and to support multiple concurrent analytics functions at a low price and power point. A key goal for the BF-608 and BF-609, which the PVP is a key enabler of, is to bring a sophisticated set of vision analytics, to our customers’ products, that previously could not afford them both in terms of cost and power. The goal is to broaden the adoption of image processing in our customers’ products. These processors are the highest performance Blackfin processors that have been offered to date. This is in terms of the combination of core performance, on chip memory size, on chip memory bandwidth, and high bandwidth peripherals. 1Ghz of performance is offered across two Blackfin cores running at 500Mhz each. These processors have the largest amount of on chip memory within the family with up to 4.3Mbits of on chip SRAM. Also critical to the performance is the high bandwidth system interconnect which is based on a switched fabric bus infrastructure. We’ll see more on this shortly, but basically, the interconnect enables efficient data movement between the various memories and peripherals. It is critical to have a high bandwidth infrastructure in order to support HD video data rates.A rich set of peripherals and connectivity options are offered. Memory interfaces include DDR2, low power DDR, and a RSI (or Removable Storage Interface) for interfacing to memories such a SDIO. Connectivity includes USB, two Ethernet ports with real-time 1588 support and different types of serial interfaces such as SPIs, Serial Ports, UART or I2C compatible ports.Then we have 3 ePPIs (or enhanced Parallal Peripheral Interface)s also referred to as the video interfaces that provide direct connectivity to image sensors and displays. There are also provisions on these processors for multiprocessing point-to-point communication via the Link Ports. Link ports can also be used as high bandwidth interfaces to FPGAs. Many features have been integrated for safety oriented applications. Safety related features include memory parity in the L1 memories, error correction codes in the L2 memory, and functional protection units for detecting and recovering from faults. These features protect against events that could cause anomalous operation such as software errors. Low power consumption was also a key goal in the design of these processors. Because of self heating of devices, low power consumption is critical to being able to support high temperatures ranges such as 105 degrees C. At 25 degrees C a typical power consumption for the BF609 is 400mWatts.
• #9: The BF609 series has been designed in a way that the interchip communication ports can be layed out efficiently without incurring into very complicated hardware layout designs. So here is just an example of three Blackfin devices interconnected using the Link Ports and the Serial Ports. Then each one of the three devices has its own set of off-chip L3 DDR2 memories, and at the same time they are all capturing video data from three sets of cameras. The device on the left can be used for booting the system, while the device on the far right can be used for driving the display. Again, it’s just an example of the many available options for interprocessor communication.
• #11: As discussed, the BF608 and 609 Introduce a new Video Subsystem(VSS) composed of:3 Enhanced Parallel Peripheral Interfaces (EPPI);The PVP;The Pixel CompositorAnd the Pixel CrossbarThe Video Subsystem has been architected to allow for pipelining of pixel datapaths between various blocks within the Video Subsystem without consuming core MIPs nor memory bandwitdh.For example, the Video Subsystem can be configured such that the pixel data from a camera and sensor will be pipelined directly via the video interface or EPPI directly into the PVP through the use of the video subsystem’s local pixel crossbar. This will save considerable bandwidth by avoiding the need to send high definition high frame rate data to L3 DDR SDRAM memory and later reading back in from L3 for vision processing.Similarly, if one were to configure the pixel compositor, which performs color space conversion and alpha blending, as part of the pixel pipeline then we would again save the required bandwidth to go to and from memory once again. And, as most of you know, system power grows notably particularly with increasing DDR SDRAM switching power driven by higher bandwidth applications. This is certainly an important issue if your system requirements greatly limit overall power consumption and package cost.
• #12: Video Port receives raw RGB color data (and optionally forwards it to memory), then the Pixel compositor is used to convert the RGB color data to YUV, so that then the PVP can extract the Y component to then operate on monochrome data. This is just one option. Alternatively the PVP can extract G out of RGB itself or even extract Red or Clear out of aRed-Clear-Clear sequence.Once the PVP is done processing the incoming video data, it would then pass it onto memory for further processing by the core, which would in turn perform whichever vision analysis is required by the application on the pre-processed data, to then send the data to the next stage for further actions or to send it to a display, for example. Also, note that in this final stage, it may be required by the application to overlay the processed data on top of the original image, which we had previously stored in external memory by the way. The Pixel Compositor would perform the overlay function, again reducing overall core MIPs.
• #13: So at the heart of the Video Subsystem we have the Pipelined Vision Processor. As mentioned already, the PVP is a dedicated hardware accelerator for image processing functions that help reduce overall required bandwidth for applications like ADAS,Robotic and Machine Vision Systems including security & surveillance, gesture recognition, or medical imaging….The PVP provides hardware acceleration into the following 3 major areas:Object Detection (convolution and wavelet-based)Object Classification and TrackingAnd Object VerificationPVP works in conjunction with the Blackfin DSP core, providing additional 8GMACs performance for embedded vision applications, while operating at very low power (below the 80mW mark). Additionally, the PVP incorporates windowing and pre-filtering of incoming data, again reducing the required off-chip bandwidth and hence reducing the overall system power consumption!
• #14: Let’s now zoom into the PVP to explore the various Functional Blocks in a bit more detail. As shown, the PVP is composed of 12 functional blocks, capable of performing many different functions. Left to right, we can see Four 5x5 16-bit convolution blocks optionally followed by down scaling. An up- and down-scaling unit with independent scaling ratios for horizontal and vertical components. An arithmetic unit with 32-bit addition, multiply and divide. A 16-bit cartesian-to-polar coordinate conversion block. A pixel edge classifier that supports 1st and 2nd derivative Modes. A 32-bit threshold block with 16 thresholds, a histogram, and run-length encoding. And Two 32-bit integral blocks that support regular and diagonal integrals. Then, we can only find the Input and output formatters for compatibility with many data formats, including for exampple, Bayer input format. The PVP can form a pipe of all the algorithmic modules and is dynamically reconfigurable to form different pipeline structures.CLICK!!Furthermore, the PVP supports the simultaneous processing of up to four data streams. The memory pipe stream shown in yellow operates on data received by DMA from any L1, L2, or L3 memory. The three camera pipe streams shown in orange operate on a common input received directly from any of the three PPI inputs. Each stream has a dedicated DMA output. This preprocessing concept ensures careful use of available power and bandwidth budgets and frees up the processor cores for other tasks.Worth highlighting that the PVP has been designed for monochrome Vision Processing and up to HD performance 1280x960 pixels @ 30 f/s.
• #15: Let’s now have a look at a couple of use cases. Here is an ADAS example for Lane Departure Warning. From left to right. CLICK!!We have a monochrome image sensor capturing the road scene over the EPPI. This in turn passes the captured data to the PVP, while also passing the captured raw data to memory (remember the camera pipes we talked about on the previous slide).CLICK!!Next, the PVP is used to perform the first level of processing, which in this case is edge detection, and it automatically sends its output (the edge map of the captured image) to memory. Important to note that at this stage the Core has not been used for any image processing!CLICK!!Now, using the edge map stored in memory, the core comes into play for the first time to perform some further analysis on the stored data calculating the Contour tracing . Once completed, it stores the output (i.e the lane contour) back to memory. Remember that this is for a lane departure warning application, so the goal here is to find where the lane is to then be able to take action accordingly!CLICK!!Finally, some optional post-processing might be performed. In this case, the desire is to overlay lane contour on top of the original image to then send it to a display. Remember that the Pixel Compositor can take care of the image overlay function, while an additional ePPI (we have three ports and we’ve only used one thus far) is used to drive the image to the monitor.
• #16: And here is another example. This is for a machine vision use case: a Dice Dot Counting Demo.CLICK!!In this example, a camera is monitoring a set of dice. The captured video data is fed into the PVP via EPPI2 and then passed thru a set of functional blocks: convolution block for a Gaussian Filter, followed by a Sobel filter using two more convolution blocks, and then fed into the Polar Magnitude and Angle Block for converting coordinates from Cartesian to Polar form, to then send it to the Pixel Edge Classifier block for Edge classification.CLICK!! The edge map of the incoming input data is then fed into Core 0 for edge trace operations. Core 0 is also responsible for feeding the processed data into Core 1 as well as managing the system display. CLICK!!Next, core 1 comes into play for finding the dice contours and performing the actual dot count algorithm to then output the resulting data.CLICK!!EPPI0 is finally used to render the output data to the display, along with the original image (which was initialy forwarded by EPPI2 using the camera pipe). The output image also includes the counted number of dots.
• #17: The BF609 Blackfin Processor series has been designed in 65nm Low Power Process, which combined with the Pipelined Vision Processor, enable Low Power, Low Cost, and Low Bandwidth Constraints for Embedded Vision ApplicationsAs seen, PVP works in conjunction with the Blackfin DSP core, providing additional 8GMACs performance for embedded vision applications, while operating at very low power (below the 80mW mark). Some tests have shown that the PVP can offer a reduction of about 83% of core MIPs.Additionally, the PVP helps to greatly reduce the required off-chip bandwidth (about 74%),hence reducing the overall system power consumption!
• #19: The ADSP-BF609 Evaluation board, also referred to as EZ-KIT Lite, shown at the bottom left of the screen, provides a low cost hardware solution for users to evaluate the BF609 Blackfin processor family. In addition to the BF609 device itself, it features DDR2 memory, parallel and SPI flash devices, Ethernet, CAN, UART and USB connectivity. It also contains an external SD Card slot and an Expansion Interface to facilitate the connection to external hardware.Analog Devices also offers a set of Extender boards, referred to as EZ-Extenders (some of which are shown on the right), for expanding EZ-KIT board capabilities for more specific functions like Audio processing, video Encoding/decoding as well as connectivity to LCD displays and image sensors.
• #20: In addition to Analog Devices BF609 EZ-KIT board and extenders, Avnet has newly released the BF609 Embedded Vision Starter KIT. This is a single board solution that combines the capabilities of Analog Devices’ BF609 EZ-KIT, the Camera EZ-Extender and the Video Encoder and Decoder boards all into a single hardware development platform!And it’s not only about hardware, but it’s also meant to serve as a complete reference design, since it’s provided with out-of-the box drivers and code examples enabling customers to quickly evaluate and design vision based applications!CLICKFor more info, please refer to finboard.org
• #21: So that covers the hardware side of things… let’s now briefly explore Analog Devices Integrated Development and Debugging Environment, CrossCore Embedded Studio.
• #22: CrossCore® Embedded Studio is a world-class integrated development environment supporting Analog Devices Blackfin® and SHARC® processor families. Employing the latest generation of our mature code generations tools, this Eclipse™ based IDE provides seamless, intuitive C/C++ and assembly language editing, code-generation, and debug support.
• #23: CrossCore Embedded Studio also offers highly integrated middleware (also referred to as add-ins) for drivers, services, and software modules. These include driver support for on chip and off chip peripherals, stacks for Ethernet and USB, a popular real time operating system and file system, and more. It provides an easy to use development framework which includes exceptional integrated multicore development and debug support.
• #24: MCAPI defines an API and a semantic for communication and synchronization between processing cores in embedded systems. It’s Targeted at closely distributed cores, for both homogeneous and heterogeneous cores and it’s transport independent. It uses a Communication topology abstraction similar to what sockets do for TCP/IP. As briefly mentioned in the prior slide, MCAPI support is available under CrossCore Embedded Studio as an add-in, allowing users to easily integrate MCAPI APIs and primitives into the code, without needing to worry about the low level details of the standard implementation. For more information on MCAPI, please refer to the Multicore Association website.
• #25: Here is the Add-in selection window, available from the System Configuration overview under CrossCore Embedded Studio. As it can be seen, users can easily add MCAPI support to their multi-core projects with a simple click.
• #26: Another very valuable Add-in, particularly for vision based applications that want to benefit from the flexibility and capabilities of the PVP, is the PVP Programmer.
• #27: The available PVP function blocks can be easily selected and configured using this graphical user interface, which will then generate all of the required code to properly programe the PVP blocks.
• #28: Here you can see how easily the blocks can be selected, moved around, connected, etc…. Everything being developed as build-in blocks. Upon completion, the code corresponding to the selected sequence of function blocks will be generated and added accordingly to the project’s source code… all of it done using the PVP Programmer add-in, without having to deal with the low level details of the hardware accelerator block. Hence, greatly simplifying and accelerating development time!
• #29: By default, CrossCore Embedded Studio includes a minimal number of code examples. Additional examples can be found as part of the Board Support Packages (BSPs) which provide comprehensive support for the Analog Devices’ EZ-KIT Boards and EZ-Extender platforms. The BSPs include examples which demonstrate the drivers and services provided along with CrossCore Embedded Studio. Furthermore, Middleware and RTOS product Add-Ins include their own set of examples.After installing a BSP or Add-In, example code for the various products are searchable using the example browser accessible from the Help menu in CCES. Additionally, the site shown here (analog.com/swexamples) provides a comprehensive list of the examples that are available per product, with links to each product's page and information on which products are needed to build and run each and every example.
• #30: In addition to the development tools,boards, and software examples, Analog Devices also provide a comprehensive set of highly optimized audio and video algorithms for both Blackfin and SHARC processors to allow customers to quickly incorporate these functions into their end products. These are referred to as software modules, internally developed by ADI, and freely available for download from the Software modules websites: analog.com/Blackfinmodules and analog.com/SHARCmodules.Whereas the Software Examples include examples which demonstrate the drivers and services provided with CrossCore Embedded Studio. The Blackfin and SHARC software modules provide algorithm solutions such as audio and video codecs, encoders, decoders, post processing code, etc. to help with the evaluation process and most importantly to help speed up development time.
• #31: As part of the overall software modules offering, particularly useful for Embedded Vision Applications are the 2D Graphics Library, the Image Processing Toolbox and the Vision Analytics Toolbox. The first two are Optimized graphics and image processing primitives designed to enable faster development of complex image or video processing solutions. They Include C reference codes for the primitives and wrapper codes for OpenCV-like APIs.The Blackfin Vision Analytics Toolbox is a software toolbox for system integrators working on vision based applications. This toolbox makes use of primitives from the Blackfin Image Processing Toolbox and it includes solutions for vision analytics functions like Canny and Sobel Edge Detection, among others.These free of charge and highly optimized software modules can be integrated into many different embedded vision type applications, like Traffic Sign Recognition in the ADAS space or Iris Recognition in Medical Imaging Applications, enabling faster development cycles.
• #33: And finally don’t forget to follow us on the various social media networks. You can also find the answers to your questions on the EngineerZone Support Community or feel free to contact us via email at processor.support@analog.com Thanks for watching….