CONDOR: An automated framework to accelerate convolutional neural networks on FPGA
0 likes120 views
The document presents Condor, an automated framework designed to accelerate convolutional neural networks (CNNs) on FPGAs through a structured process involving parsing CNN architecture and creating hardware accelerators. It emphasizes cloud integration and the capacity to balance performance and power consumption, supporting major deep learning libraries. The authors discuss their methodology and architectural choices, highlighting challenges and innovations in FPGA implementations for deep learning tasks.
![Our first approach
!23
[1] Giuseppe Natale, Marco Bacis, Marco D. Santambrogio
“On how to design fpga-based accelerators for Convolutional Neural Networks”, ISVLSI 2017
DMA
in
CONV POOL CONV LINEAR
w/b w/b w/b
POOL
POOL
POOL
POOL](https://image.slidesharecdn.com/berkeleycondor-180523093914/85/CONDOR-An-automated-framework-to-accelerate-convolutional-neural-networks-on-FPGA-23-320.jpg)
CONDOR: An automated framework to accelerate convolutional neural networks on FPGA
- 1. CONDOR AN AUTOMATED FRAMEWORK TO ACCELERATE CONVOLUTIONAL NEURAL NETWORKS ON FPGA Soda-430-438 Woz Lounge Berkeley, CA May 23rd, 2018 Niccolò Raspa, Marco Bacis, Giuseppe Natale, Marco D. Santambrogio
- 6. Deep Convolutional Neural Networks !6
- 7. !7 CNN on siliconCNN on silicon GPU ASIC FPGA
- 10. Manual Design !10 Extract the parameters and the weights Write the code Synthesis Evaluate DesignPackage IP Iterate
- 12. Framework Architecture !12 Parse structure of the CNN FRONTEND Creation of HW Accelerator CORE LOGIC Deployment BACKEND
- 13. Create DAG computation !13 PROTOTXT CAFFEMODEL { Input Data Convolution Pooling Fully Connected Convolution Pooling Fully Connected Input Dimension: (28, 28, 1) Output Dimension (24, 24, 20) Kernel: 5 Padding: 0 Stride: 1 Input dimension (28, 28, 1) Input Dimension: (24, 24, 20) Output Dimension (12, 12, 20) Kernel: 2 Padding: 0 Stride: 2
- 14. Map computation in hardware !14 Area Convolution Pooling Fully ConnectedConvolution Pooling
- 17. What if I don’t have an FPGA? !17 CONDOR
- 18. Features !18 Cloud Integration via Amazon F1 Instances Automatic creation of an hardware accelerator for FPGA Tune the tradeoff between performance and power consumption Support main deep learning libraries
- 19. Roadmap !19 Automated Framework Methodology for Acceleration of CNN Integration with Caffe Cloud Integration M. Bacis, G. Natale, E. Del Sozzo, and M. D. Santambrogio. “A Pipelined and Scalable Dataflow Implementation of Convolutional Neural Networks on FPGA” In: 2017 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW) Giuseppe Natale, Marco Bacis and Marco Domenico Santambrogio. “On how to design dataflow FPGA-based accelerators for Convolutional Neural Networks” In: 2017 IEEE Computer Society Annual Symposium on VLSI (ISVLSI) * 2017 Support the new standard ONXX 2018 Open Source Release 2019? Extend Methodology XOHW Competition *
- 20. CONDOR ARCHITECTURAL CHOICES FOR THE FPGA-BASED ACCELERATION OF CNNs Soda-430-438 Woz Lounge Berkeley, CA May 23rd, 2018 Niccolò Raspa, Marco Bacis, Giuseppe Natale, Marco D. Santambrogio
- 21. Framework !21 Parse structure of the CNN FRONTEND Creation of HW Accelerator CORE LOGIC Deployment BACKEND
- 22. FPGAs and CNNs !22 Dataflow Computation Data reusability Distributed Architecture
- 23. Our first approach !23 [1] Giuseppe Natale, Marco Bacis, Marco D. Santambrogio “On how to design fpga-based accelerators for Convolutional Neural Networks”, ISVLSI 2017 DMA in CONV POOL CONV LINEAR w/b w/b w/b POOL POOL POOL POOL
- 24. Bigger networks, bigger FPGAs… or not? !24 • Weights don’t fit on the on-chip BRAMs • Unrolling leads to the explosion of DSPs (multipliers) usage
- 25. Methodology improvements !25 • No complete unrolling - partial accumulations • Generic set of “one size fits all” blocks • Semi-dataflow architecture • More complex data movement
- 27. Customizable data flow !27 ReLU Pool ReLU ReLU Conv Conv Conv Datamover/Control Conv Pooling ReLU in w/b out Datamover/Control Conv Pooling ReLU in w/b out Datamover/Control Conv Pooling ReLU in w/b out Datamover/Control Conv Pooling ReLU in w/b out
- 28. MAC weights input result Dataflow Blocks !28 • Convolution, Pooling, ReLU etc… • Non-uniform memory partitioning • Streaming pattern • Optimal full buffering • Concurrent accesses
- 29. Partial accumulations approach !29 • Custom level of parallelism • Compute subset of both input/output feature maps • Accumulation done with a FIFO and/or from DDR
- 30. Memory control and data buffering !30 • Memory mapped to streaming and viceversa • Exploit the maximum transaction size and bursts
- 31. Weights Double Buffering !31 • Masks weights loading latency • Allows to not flush the MAC pipeline on each iteration ping pong
- 32. Input Caching !32 • Reduces memory accesses • Stores entire input for a layer • Used for small layers (avoid lots of small transactions) Datamover/ControlInput Cache in Datamover/ControlInput Cache in
- 33. Architecture Evaluation !33 •5 MB BRAM •2880 DSPs (27x15 bits mult) •1 DDR port (512 bits wide) •115.2 GFLOPs max (100Mhz) Alphadata Virtex-7 Setup Results •30.6 GOPs, 56MB parameters •4 input, 4 output ports •27.2 GFLOPs estimated •14.4 GFLOPs reached VGG16 Network
- 34. Lessons Learned !34 • Floating point is dead, long live the fixed! • Off-chip memory vs On-chip memory • Old hardware vs New Hardware
- 35. Next Steps !35 • Possibility to use URAMs as on chip storage (33.75 MB) • Higher number of DSPs (~2.3X) • Efficient multiplication (8 bits fixed point -> 2 mul/dsp) • Higher memory BW (4 DDR ports)
- 36. Next Steps MAC/ Window FSM Acc/ReLU Pooling 1024 out512 in 2-64 out Weights I/O Buffer
- 37. “A Framework with Cloud Integration for CNN Acceleration on FPGA Devices” Marco Bacis marco.bacis@mail.polimi.it Niccolo’ Raspa niccolo.raspa@mail.polimi.it Giuseppe Natale giuseppe.natale@polimi.it Marco D. Santambrogio marco.santambrogio@polimi.it twitter.com/CondorAtNECST facebook.com/CondorAtNECST