Systolic-CNN: An OpenCL-defined Scalable Run-time-flexible FPGA Accelerator Architecture for Accelerating Convolutional Neural Network Inference in Cloud/Edge Computing

Akshay Dua, Yixing Li, Fengbo Ren

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

This paper presents Systolic-CNN, an OpenCLdefined scalable, run-time-flexible FPGA accelerator architecture, optimized for performing the low-latency, energy-efficient inference of various convolutional neural networks (CNNs) in the context of multi-tenancy cloud/edge computing. Systolic-CNN adopts a highly pipelined and parallelized 1-D systolic array architecture, which efficiently explores both spatial and temporal parallelism for accelerating CNN inference on FPGAs. SystolicCNN is highly scalable and parameterized, which can be easily adapted by users to achieve 100% utilization of the coarsegrained computation resources (i.e., DSP blocks) for a given FPGA. In addition, Systolic-CNN is run-time-flexible, which can be time-shared, in the context of multi-tenancy cloud or edge computing, to accelerate a variety of CNN models at run time without the need of recompiling the FPGA kernel hardware nor reprogramming the FPGA. The experiment results based on an Intel Arria 10 GX FPGA Development board show that Systolic-CNN, when mapped with a single-precision data format, can achieve 100% utilization of the DSP block resource and an average inference latency of 10ms, 84ms, 1615ms, and 990ms per image for accelerating AlexNet, ResNet-50, RetinaNet, and Light-weight RetinaNet, respectively. The peak computational throughput is measured at 80-170 GFLOPS/s across the acceleration of different CNN models. Codes are available at https://github.com/PSCLab-ASU/SystolicCNN.

Original languageEnglish (US)
Title of host publicationProceedings - 28th IEEE International Symposium on Field-Programmable Custom Computing Machines, FCCM 2020
PublisherInstitute of Electrical and Electronics Engineers Inc.
Number of pages1
ISBN (Electronic)9781728158037
DOIs
StatePublished - May 2020
Event28th Annual IEEE International Symposium on Field-Programmable Custom Computing Machines, FCCM 2020 - Fayetteville, United States
Duration: May 3 2020May 6 2020

Publication series

NameProceedings - 28th IEEE International Symposium on Field-Programmable Custom Computing Machines, FCCM 2020

Conference

Conference28th Annual IEEE International Symposium on Field-Programmable Custom Computing Machines, FCCM 2020
CountryUnited States
CityFayetteville
Period5/3/205/6/20

ASJC Scopus subject areas

  • Computational Mathematics
  • Computer Networks and Communications
  • Computer Science Applications
  • Hardware and Architecture
  • Signal Processing

Fingerprint Dive into the research topics of 'Systolic-CNN: An OpenCL-defined Scalable Run-time-flexible FPGA Accelerator Architecture for Accelerating Convolutional Neural Network Inference in Cloud/Edge Computing'. Together they form a unique fingerprint.

  • Cite this

    Dua, A., Li, Y., & Ren, F. (2020). Systolic-CNN: An OpenCL-defined Scalable Run-time-flexible FPGA Accelerator Architecture for Accelerating Convolutional Neural Network Inference in Cloud/Edge Computing. In Proceedings - 28th IEEE International Symposium on Field-Programmable Custom Computing Machines, FCCM 2020 [9114649] (Proceedings - 28th IEEE International Symposium on Field-Programmable Custom Computing Machines, FCCM 2020). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/FCCM48280.2020.00064