Edgecortix Pte. Ltd.

Maxell, Ltd.

Leapmind Inc.

Nikolay Nez

Sakyasingha Dasgupta

Oleg Khavin

Antonio Tomas Nevado Vilchez

Jens Huthmann

Hamid Reza Zohouri

Tanvir Ahmed

Kunihiko Ietomi

Koumei Tomida

Mikhail Volkov

A method for controlling a neural network circuit that is provided with a first memory, a convolution operation circuit that performs a convolution operation, a second memory, a quantization operation circuit, a second write semaphore, a second read semaphore, a third write semaphore, and a third read semaphore, wherein the method for controlling the neural network circuit involves making the convolution operation circuit implement a convolution operation based on the third read semaphore and the second write semaphore.

Method for controlling neural network circuit

Low-Density Parity-Check (LDPC) data decoding using iteration-variable accuracy is performed by segmenting a LDPC encoded data block of probability values by dividing the LDPC encoded data block into a plurality of data probability value segments and a plurality of parity probability value segments, each probability value of the LDPC encoded data block representing a likelihood between binary values, decoding the LDPC encoded data block by adjusting, according to an iteration-variable accuracy parameter, the probability values of the LDPC encoded data block based on a parity-check matrix, the parity-check matrix defining correspondence among data probability value segments and parity probability value segments, and concatenating likely binary values that satisfy the parity-check matrix associated with the probability values of each data probability value segment to form a decoded data block. The iteration-variable accuracy parameter represents a tradeoff between accuracy and computational efficiency.

Low-density parity-check (LDPC) data decoding using iteration-variable accuracy

Integrated circuit data stream processing utilizing paged buffering is performed by an integrated circuit that includes an upstream random access memory, a local read counter, a downstream random access memory, a local write counter, and a processor. The local read counter is configured to store a value indicating whether any accessible blocks of data are stored in the upstream memory. The local write counter is configured to store a value indicating whether any downstream pages of the downstream memory are available for recording. The processor is configured to adjust the local read counter to indicate a page release, adjust the local write counter to indicate a page occupy, read a first block of data recorded to the upstream memory, process the first block of data to produce a second block of data, and record the second block of data to the downstream memory.

Integrated circuit data stream processing using paged buffering

A neural network circuit that can be embedded in an embedded device such as an IoT device, and that provides high performance. The neural network circuit includes a first memory unit that stores input data; a convolution operation circuit that performs a convolution operation on a weight and the input data stored in the first memory unit; a second memory unit that stores convolution operation output data from the convolution operation circuit; and a quantization operation circuit that performs a quantization operation on the convolution operation output data stored in the second memory unit; wherein the first memory unit stores a quantization operation output data from the quantization operation circuit; and the convolution operation circuit performs the convolution operation on the quantization operation output data stored in the first memory unit as the input data.

Neural network circuit, edge device and neural network operation process

Neural network hardware acceleration data parallelism is performed by an integrated circuit including a plurality of memory banks, each memory bank among the plurality of memory banks configured to store values and to transmit stored values, a plurality of computation units, each computation unit among the plurality of computation units including one of a channel pipeline and a multiply-and-accumulate (MAC) element configured to perform a mathematical operation on an input data value and a weight value to produce a resultant data value, and a computation controller configured to cause a value transmission to be received by more than one computation unit or memory bank.

Neural network hardware accelerator data parallelism

Sequence partition based schedule optimization is performed by generating a sequence and a schedule based on the sequence, dividing the sequence into a plurality of sequence partitions based on the schedule and the data dependency graph, each sequence partition including a portion of the plurality of instructions and a portion of the plurality of buffers, performing, for each sequence partition, a plurality of partition optimizing iterations, and merging the plurality of sequence partitions to produce a merged schedule.

Sequence partition based schedule optimization

Neural network inference may be performed by configuration of a device including an accumulation memory, a plurality of convolution modules configured to perform mathematical operations on input values, a plurality of adder modules configured to sum values output from the plurality of convolution modules, and a plurality of convolution output interconnects connecting the plurality of convolution modules, the plurality of adder modules, and the accumulation memory. The accumulation memory is an accumulation memory allocation of a writable memory block having a reconfigurable bank width, and each bank of the accumulation memory allocation is a virtual combination of consecutive banks of the writable memory block.

Neural network accelerator writable memory

Neural network hardware acceleration data parallelism is performed by an integrated circuit including a plurality of memory banks, each memory bank among the plurality of memory banks configured to store values and to transmit stored values, a plurality of computation units, each computation unit among the plurality of computation units including a processor including circuitry configured to perform a mathematical operation on an input data value and a weight value to produce a resultant data value, and a computation controller configured to cause a value transmission to be received by more than one computation unit or memory bank.

Preparation and execution of quantized scaling may be performed by operations including obtaining an original array and a scaling factor representing a ratio of a size of the original array to a size of a scaled array, determining, for each column of the scaled array, a horizontal coordinate of each of two nearest elements in the horizontal dimension of the original array, and, for each row of the scaled array, a vertical coordinate of each of two nearest elements in the vertical dimension of the original array, calculating, for each row of the scaled array and each column of the scaled array, a linear interpolation coefficient, converting each value of the original array from a floating point number into a quantized number, converting each linear interpolation coefficient from a floating point number into a fixed point number, storing, in a memory, the horizontal coordinates and vertical coordinates as integers, the values as quantized numbers, and the linear interpolation coefficients as fixed point numbers.

Preparation and execution of quantized scaling on integrated circuitry

Neural network accelerator hardware-specific division of inference may be performed by operations including obtaining a computational graph and a hardware chip configuration. The operations also include dividing inference of the plurality of layers into a plurality of groups. Each group includes a number of sequential layers based on an estimate of duration and energy consumption by the hardware chip to perform inference of the neural network by performing the mathematical operations on activation data, sequentially by layer, of corresponding portions of layers of each group. The operations further include generating instructions for the hardware chip to perform inference of the neural network, sequentially by group, of the plurality of groups.

Neural network accelerator hardware-specific division of inference into groups of layers

Neural network inference may be performed by configuration of a device including a plurality of convolution modules, a plurality of adder modules, an accumulation memory, and a convolution output interconnect control module configured to open and close convolution output interconnects among a plurality of convolution output interconnects connecting the plurality of convolution modules, the plurality of adder modules, and the accumulation memory. Inference may be performed while the device is configured according to at least one convolution output connection scheme whereby each convolution module has no more than one open direct connection through the plurality of convolution output interconnects to the accumulation memory or one of the plurality of adder modules. The device includes a convolution output interconnect control module to configure the plurality of convolution output interconnects according to the at least one convolution output connection scheme.

Growing community of inventors

Tokyo, Japan

Nikolay Nez