Systolic-cnn: an opencl-defined scalable runtime-flexible programmable accelerator architecture for accelerating convolutional neural network inference in cloud/edge computing

Abstract

An OpenCL-defined scalable runtime-flexible programmable accelerator architecture for accelerating convolutional neural network (CNN) inference in cloud/edge computing is provided, referred to herein as Systolic-CNN. Existing OpenCL-defined programmable accelerators (e.g., field-programmable gate array (FPGA)-based accelerators) for CNN inference are insufficient due to limited flexibility for supporting multiple CNN models at runtime and poor scalability resulting in underutilized accelerator resources and limited computational parallelism. Systolic-CNN adopts a highly pipelined and paralleled one-dimensional (1-D) systolic array architecture, which efficiently explores both spatial and temporal parallelism for accelerating CNN inference on programmable accelerators (e.g., FPGAs). Systolic-CNN is highly scalable and parameterized, and can be easily adapted by users to efficiently utilize the coarse-grained computation resources for a given programmable accelerator. In addition, Systolic-CNN is runtime-flexible and can be time-shared to accelerate a variety of CNN models at runtime without the need to recompile the programmable accelerator kernel hardware or reprogram the programmable accelerator.