Systems and methods for vectorized fft for multi-dimensional convolution operations

Abstract

A new approach is proposed to support efficient convolution for deep learning by vectorizing multi-dimensional input data for multi-dimensional fast Fourier transform (FFT) and direct memory access (DMA) for data transfer. Specifically, a deep learning processor (DLP) includes a plurality of tensor engines each configured to perform convolution operations by applying one or more kernels on multi-dimensional input data for pattern recognition and classification based on a neural network, wherein each tensor engine includes, among other components, one or more vector processing engines each configured to vectorize the multi-dimensional input data at each layer of the neural network to generate a plurality of vectors and to perform multi-dimensional FFT on the generated vectors and/or the kernels to create output for the convolution operations. Each tensor engine further includes a data engine configured to prefetch the multi-dimensional data and/or the kernels to both on-chip and external memories via DMA.