Via Alliance Semiconductor Co., Ltd.

Kyle T O'Brien

G Glenn Henry

Terry J Parks

A neural network unit has a first memory that holds elements of a data matrix and a second memory that holds elements of a convolution kernel. An array of neural processing units (NPU) each have a multiplexed register that receives a corresponding element of a row from the first memory and that also receives the multiplexed register output of an adjacent NPU. A register receives a corresponding element of a row from the second memory. An arithmetic unit receives the outputs of the register, the multiplexed register and an accumulator and performs a multiply-accumulate operation on them. For each sub-matrix of a plurality of sub-matrices of the data matrix, each arithmetic unit selectively receives either the element from the first memory or the adjacent NPU multiplexed register output and performs a series of the multiply-accumulate operations to accumulate into the accumulator a convolution of the sub-matrix with the convolution kernel.

Neural network unit that performs convolutions using collective shift register among array of neural processing units

A neural network unit has at least one RAM, an output buffer and an array of neural processing units that: read first time step context layer node values from the output buffer; read second time step input layer node values from the RAM; generate second time step hidden layer node values based on the read input and context layer node values; output the hidden layer node values to the output buffer rather than to the RAM; read the hidden layer node values from the output buffer; generate second time step context layer node values based on the read hidden layer node values; output the context layer node values to the output buffer rather than to the RAM; generate output layer node values using the hidden layer node values; write the output layer node values to the RAM; and repeat for a sequence of time steps.

Neural network unit with output buffer feedback for performing recurrent neural network computations

An output buffer holds N words arranged as N/J mutually exclusive output buffer word groups (OBWG) of J words each of the N words. N processing units (PU) are arranged as N/J mutually exclusive PU groups. Each PU group has an associated OBWG. Each PU includes an accumulator and an arithmetic unit that performs operations on inputs, which include the accumulator output, to generate a first result for accumulation into the accumulator. Activation function units selectively perform an activation function on the accumulator outputs to generate results for provision to the N output buffer words. For each PU group, four of the J PUs and at least one of the activation function units compute an input gate, a forget gate, an output gate and a candidate state of a Long Short Term Memory (LSTM) cell, respectively, for writing to respective first, second, third and fourth words of the associated OBWG.

Neural network unit that performs concurrent LSTM cell calculations

An output buffer holds N words arranged as N/J mutually exclusive output buffer word groups (OBWG) of J words each. N processing units (PU) are arranged as N/J mutually exclusive PU groups each having an associated OBWG. Each PU has an accumulator, arithmetic unit, and first and second multiplexed registers each having at least J+1 inputs. A first input receives a memory operand and the other J inputs receive the J words of the associated OBWG. Each accumulator provides its output to a respective OBWG. Each arithmetic unit performs an operation on the first and second multiplexed register outputs and accumulator output to generate a result for accumulation into the accumulator. A mask input to the output buffer controls which words, if any, of the N words retain their current value or are updated with their respective accumulator output. Each PU group operates as a recurrent neural network LSTM cell.

Neural network unit with output buffer feedback and masking capability with processing unit groups that operate as recurrent neural network LSTM cells

An output buffer holds N words arranged as N/J mutually exclusive output buffer word groups (OBWG) of J words each. N processing units (PU) are arranged as N/J mutually exclusive PU groups each having an associated OBWG. Each PU has an accumulator, an arithmetic unit, and first and second multiplexed registers each having at least J+1 inputs and an output. A first input receives a memory operand and the other J inputs receive the J words of the associated OBWG. Each accumulator provides its output to a respective output buffer word. Each arithmetic unit performs an operation on the first and second multiplexed register outputs and the accumulator output to generate a result for accumulation into the accumulator. A mask input to the output buffer controls which words, if any, of the N words retain their current value or are updated with their respective accumulator output.

Neural network unit with output buffer feedback and masking capability

A hardware data compressor. A first hardware engine scans an input block of characters and uses a plurality of lists of nodes to produce back pointers to matching strings in the input block to compress the input block. Each node points to a character in the input block previously scanned and has an associated probability that a back pointer to a matching string that begins with the pointed-to character will be produced by the first hardware engine. A second hardware engine, for each list of nodes of the plurality of lists, sorts the list according to the probabilities of the nodes in the list so that higher probability nodes appear earlier in the list for use by the first hardware engine to search for matching strings during the scan of the input block of characters.

Growing community of inventors

Austin, TX, United States of America

Kyle T O'Brien