Amazon Technologies, Inc.

Via Alliance Semiconductor Co., Ltd.

Other

International Business Machines Corporation

Arm Limited

Thomas Elmer

Thomas A Volpe

Sundeep Amirineni

Ron Diamant

Randy Renfu Huang

Paul Gilbert Meyer

Joshua Wayne Bowman

Kiran Kalkunte Seshadri

Dana Michelle Vantrease

Nishith Desai

Michael Putrino

Mohammad El-Shabani

Kenneth Wayne Patton

Akshay Balasubramanian

Nikhil A Patil

Fnu Arun Kumar

Neil Burgess

Vasanta Kumar Palisetti

Mairin Imro Kroes

Anisha Saini

Systems and methods are provided to round the numbers produced by a systolic array. A rounder can obtain a number from the systolic array and identify a data type conversion associated with the number. The data type conversion may indicate a first bit-length and a second bit-length of the number. The rounder can select a random number generator for rounding the number based on the data type conversion. The bit-length of the random number generator may be equal to a difference in bit-length between the first bit-length and the second bit-length. The rounder can perform a rounding operation using a random number generated by the random number generator.

Systolic array with output rounding for multiple source/destination data type pairs

Apparatuses, methods, computer readable media, and systems are disclosed in which a floating point processing instruction is decoded to generate control signals to trigger a floating point processing operation. In response to the control signals, the floating point processing operation is performed, comprising: performing processing that yields more than two floating point values; and performing, for each of the more than two floating point values: a determination of a shift value for a significand of that floating point value by subtracting an exponent value for that floating point value from a predetermined constant anchor value determined based on a maximum calculable product exponent for a product of the more than two floating point operands, and a shift of the significand by the shift value determined for that floating point value.

Significand shifting in floating point processing operations

Systems and methods are provided to perform multiply-accumulate operations of reduced precision numbers in a systolic array. Each row of the systolic array can receive reduced inputs from a respective reducer. The reducer can receive a particular input and generate multiple reduced inputs from the input. The reduced inputs can include reduced input data elements and/or a reduced weights. The systolic array may lack support for inputs with a first bit-length and the reducers may reduce the bit-length of a given input from the first bit-length to a second shorter bit-length and provide multiple reduced inputs with second shorter bit-length to the array. The systolic array may perform multiply-accumulate operations on each unique combination of the multiple reduced input data elements and the reduced weights to generate multiple partial outputs. The systolic array may sum the partial outputs to generate the output.

Systolic array with input reduction to multiple reduced inputs

To improve performance of a computational array, the architecture of the array can be modified to allow the processing engines of a column to operate in parallel and the clock frequency of the array to be increased. The processing engines of each column of the array can be grouped into a series of row groups. The processing engines of each row group can be loaded with input values, and computations on the input values can be carried out in parallel to generate the column output. One or more flip-flop stages can be inserted into the computational logic of each of the processing engines. The computational logic can then be distributed across the flip-flop stages to reduce the propagation delay between flip-flop stages of the processing engine, hence allowing the clock frequency of the array to be increased.

Increasing performance of computational array accelerators

Systems and methods are provided to enable parallelized multiply-accumulate operations in a systolic array. Each column of the systolic array can include multiple busses enabling independent transmission of input partial sums along the respective bus. Each processing element of a given columnar bus can receive an input partial sum from a prior element of the given columnar bus, and perform arithmetic operations on the input partial sum. Each processing element can generate an output partial sum based on the arithmetic operations, provide the output partial sum to a next processing element of the given columnar bus, without the output partial sum being processed by a processing element of the column located between the two processing elements that uses a different columnar bus. Use of columnar busses can enable parallelization to increase speed or enable increased latency at individual processing elements.

Multiple accumulate busses in a systolic array

Systems and methods are provided to perform multiply-accumulate operations of at least one normalized number in a systolic array. The systolic array can obtain a first input and detect that the first input is denormal. Based on determining the first input is denormal, the systolic array can generate a first normalized number by normalizing the first input. Processing elements of the systolic array can include a multiplier and an adder. The multiplier can multiply the first normalized number by a second normal or normalized number to generate a multiplier product and the adder can add an input partial sum to the multiplier product to generate an addition result.

Systolic array including fused multiply accumulate with efficient prenormalization and extended dynamic range

Systems and methods are provided to perform multiply-accumulate operations of reduced precision numbers in a systolic array. Each row of the systolic array can receive reduced inputs from a respective reducer. The reduced input can include a reduced input data element and/or a reduced weight. The systolic array may lack support for inputs with a first bit-length and the reducers may reduce the bit-length of a given input from the first bit-length to a second shorter bit-length and provide the reduced input to the array. In order to reduce the bit-length, the reducer may reduce the number of trailing bits of the input. Further, the systolic array can receive a reduced and rounded input. The systolic array can propagate the reduced input through the processing elements in the systolic array. Each processing element may include a multiplier and/or an adder to perform arithmetical operations based on the reduced input.

Systolic array with efficient input reduction and extended array performance

Systems and methods are provided to perform multiplication-delayed-addition operations in a systolic array to increase clock speeds, reduce circuit area, and/or reduce dynamic power consumption. Each processing element in the systolic array can have a pipeline configured to perform a multiplication during a first systolic interval and to perform an accumulation during a second systolic interval. The multiplication result from the first systolic interval can be stored in a delay register for use by the accumulator during the second systolic interval. A skip detection circuit can be used to skip one or more of the multiplication, storing in the delay register, and the addition during skip conditions for improved energy efficiency.

Systolic multiply delayed accumulate processor architecture

Systems and methods are provided to perform multiply-accumulate operations of multiple data types in a systolic array. One or more processing elements in the systolic array can include a shared multiplier and one or more adders. The shared multiplier can include a separate and/or a shared circuitry where the shared circuitry can perform at least a part of integer multiplication and at least a part of non-integer multiplication. The one or more adders can include one or more shared adders or one or more separate adders. The shared adder can include a separate and/or a shared circuitry where the shared circuitry can perform at least a part of integer addition and at least a part of non-integer addition.

Systolic array component combining multiple integer and floating-point data types

Systems and methods are provided to perform multiply-accumulate operations of normalized numbers in a systolic array to enable greater computational density, reduce the size of systolic arrays required to perform multiply-accumulate operations of normalized numbers, and/or enable higher throughput operation. The systolic array can be provided normalized numbers by a column of normalizers and can lack support for denormal numbers. Each normalizer can normalize the inputs to each processing element in the systolic array. The systolic array can include a multiplier and an adder. The multiplier can have multiple data paths that correspond to the data type of the input. The multiplier and adder can employ expanded exponent range to operate on normalized floating-point numbers and can lack support for denormal numbers.

Methods and systems for performing hardware approximation of function are provided. In one example, a system comprises a controller, configurable arithmetic circuits, and a mapping table. The mapping table stores a first set of function parameters in a first mode of operation and stores a second set of function parameters in a second mode of operation. Depending on the mode of operation, the controller may configure the arithmetic circuits to compute a first approximation result of a function at an input value based on the first set of function parameters, or to compute a second approximation result of the function at the input value based on the second set of function parameters and to perform post-processing, such as quantization, of the second approximation result.

Configurable function approximation based on switching mapping table content

Systems and methods are provided to enable parallelized multiply-accumulate operations in a systolic array. Each row of the systolic array can include multiple busses enabling independent transmission of inputs along the respective bus. Each processing element can include a plurality of interconnects to receive a plurality of inputs corresponding to the multiple busses. Each processing element of a given row-oriented bus can receive an input from a prior element of the given row-oriented bus at an active bus position and perform arithmetic operations on the input. Each processing element can further receive a plurality of inputs at passive bus positions and provide the plurality of inputs to subsequent processing elements without the plurality of inputs being processed by the processing element. Use of row-oriented busses can enable parallelization to increase speed or enable increased latency at individual processing elements.

Multiple busses within a systolic array processing element

Systems and methods are provided to enable parallelized multiply-accumulate operations in a systolic array. Each row of the systolic array can include multiple busses enabling independent transmission of inputs along the respective bus. Each processing element of a given row-oriented bus can receive an input from a prior element of the given row-oriented bus, and perform arithmetic operations on the input. Each processing element can generate an output partial sum based on the arithmetic operations, provide the input to a next processing element of the given row-oriented bus, without the input being processed by a processing element of the row located between the two processing elements that uses a different row-oriented bus. Use of row-oriented busses can enable parallelization to increase speed or enable increased latency at individual processing elements.

Growing community of inventors

Austin, TX, United States of America

Thomas Elmer