Cadence Design Systems, Inc.

Michael Patrick Zimmer

Yong Liu

Ngai Ngai William Hung

Dhiraj Goswami

An approach is described for a method, product, and apparatus for a machine learning process using weight sharing within a systolic array having reduced memory bandwidth. According to some embodiments, this approach includes providing a systolic array that includes processing elements which each have some number of storage elements for storing weights. For example, the weights can be reused for different data sets by identifying/capturing a current state of the storage elements, generating a plan to transition to a target state of those storage elements, and application of the transition plan such that weights that are already stored in those storage elements can be reused and/or relocate. This lowers the bandwidth requirements for weight memory by allowing weights that have previously been read into the systolic array to be reused.

Method, product, and apparatus for a machine learning process using weight sharing within a systolic array having reduced memory bandwidth

An approach includes receiving a machine learning processing job, executing the machine learning processing job using parallel processing of multiple output pixels each cycle by walking data across processing elements with broadcast weights within regions and executing parallel multiplication operations, and generating an output indicating whether the machine learning processing job was successful or failed. In some embodiments, a schedule of actions is generated for respective machine learning processing jobs. The schedule of actions may include any of a plurality of shift operations in a many to many arrangement or a one to many arrangement, shifting data across region boundaries, fetching data and weights from a memory and distribution thereof to a plurality of regions (e.g., weights are distributed to respective weight memories which subsequently broadcasts those weights in a specified order based on a schedule of actions, and where data is distributed to respective processing elements).

Method, product, and apparatus for a multidimensional processing array for hardware acceleration of convolutional neural network inference

An approach includes a method, product, and apparatus for dynamically removing sparse data on a pixel by pixel basis. In some embodiments, a machine learning processing job is received. The machine learning processing job is then executed on a pixel by pixel basis by selecting non-zero data values for input into a systolic array, wherein sparse data is not selected for input into the systolic array. Subsequently, a message is generated that provides an indication of whether the execution completed successfully. In some embodiments, the machine learning processing job comprises at least a plurality of multiply and accumulate operations. In some embodiments, at least one data value equal to zero for the machine learning processing job is not input into a systolic array. In some embodiments, a plurality of weights are input into a plurality of columns for each cycle.

Method, product, and apparatus for a machine learning process leveraging input sparsity on a pixel by pixel basis

An approach is described for a method, product, and apparatus for a machine learning process using dynamic rearrangement of sparse data and corresponding weights. This approach includes a method, product, and apparatus for dynamically rearranging input data to move sparse data to a location such that computations on the sparse data might be avoided when executing a machine learning processing job. For example, sparse data within each row of the input matrix can be moved to the end of each corresponding row. When the input data is folded to fit the array, that sparse data might be at least partially contained within a fold that comprises only sparse data and possibly filler data. In such an event, computations on the fold are unnecessary and are avoided. In some embodiments, the approach includes dynamically rearranging a weight matrix to maintain a correspondence between the input data and the weights.

Method, product, and apparatus for a machine learning process using dynamic rearrangement of sparse data and corresponding weights

An approach includes identification of a machine learning model for processing and generating an ordered set of weights with varying precisions and metadata that specifies where those values can be found in order to allow the identification of weights needed during processing. In a first embodiment, the variable precision weights are separated into different memory segments where each segment has weights of only a single precision. In a second embodiment, the variable precision weights are provided in a memory where weights of different precisions are intermingled, and those weights are identified using a sequence of pairs of data representing a number of weights with the same precision and the precision of those weights. In some embodiments, both the first and second embodiments are combined, where some segments contain weights with only a single precision and at least one segment stores weights with different precisions within a respective segment.

Growing community of inventors

Chicago, IL, United States of America

Michael Patrick Zimmer