International Business Machines Corporation

Globalfoundries Inc.

Ankur Agrawal

Kailash Gopalakrishnan

Silvia Melitta Mueller

John Francis Bulzacchelli

Bruce Martin Fleischer

Jungwook Choi

Naigang Wang

Sergey Vladimirovich Rylov

Kyu-hyoun Kim

Chia-Yu Chen

Mingu Kang

Daniel Joseph Friedman

Vijayalakshmi Srinivasan

Sunil K Shukla

Daniel Brand

Zeynep Toprak Deniz

Dongsoo Lee

Monodeep Kar

Balaram Sinharoy

Xiao Lin Sun

Bodhisatwa Sadhu

Thomas Helmut Toifl

Kerstin Claudia Schelm

Jonathan E Proesel

Swagath Venkataramani

Timothy O Dickson

Andrea Fasoli

Martin Cochet

Jinwook Oh

Sanchari Sen

Xiao Sun

Jiamin Ni

Hyunkyu Ouh

An apparatus includes a floating-point gradient register; an integer register; a memory bank; and an array of processing units. Each of the units includes a plurality of binary shifters having an integer input configured to obtain corresponding bits of a 4-bit integer multiplicand, and a shift-specifying input configured to obtain corresponding bits in an exponent field of a 4-bit floating point multiplier. The multiplier is specified in a mantissaless four-bit floating point format including a sign bit, three exponent bits, and no mantissa bits. An adder tree has a plurality of inputs coupled to outputs of the plurality of shifters, and a rounder has an input coupled to an output of the adder tree. The integer inputs are connected to the integer register; the shift-specifying inputs are connected to the floating-point gradient register; and outputs of the rounders are coupled to the memory bank.

Four-bit training for machine learning

Techniques for performing analog-to-digital conversion are described. For example, a method performs an analog-to-digital conversion of an analog input to a digital output including a set of bits, the set of bits including a most significant bit and one or more additional bits, the analog-to-digital conversion starting at a given one of the one or more additional bits following the most significant bit.

Analog-to-digital conversion with bit skipping functionality

An apparatus includes multiple analog to digital converters. Individual analog to digital converters are configured to produce a digital output from an analog input and configured to compute a least significant bit of the digital output by comparing an internal residual voltage for determination of the least significant bit and a residual voltage from another analog to digital converter.

Stochastic rounding switched capacitor computation cores useful for efficient deep learning inference

An apparatus comprises at least one processor and at least one memory including instruction code configured to, with the at least one processor, cause the apparatus at least to perform a successive approximation analog-to-digital conversion of an analog input, representing a result of multiplication of first and second vectors, to a digital output by determining an upper bound on the result of multiplication of the first and second vectors, identifying, based at least in part on the determined upper bound, at least a portion of the successive approximation analog-to-digital conversion to be skipped, and skipping the identified portion of the successive approximation analog-to-digital conversion.

Energy-efficient analog-to-digital conversion in mixed signal circuitry

A multiply-accumulate device comprises a digital multiplication circuit and a mixed signal adder. The digital multiplication circuit is configured to input L m-bit multipliers and L m-bit multiplicands and configured to generate N one-bit multiplication outputs, each one-bit multiplication output corresponding to a result of a multiplication of one bit of one of the L m-bit multipliers and one bit of one of the L m-bit multiplicands. The mixed signal adder comprises one or more stages, at least one stage configured to input the N one-bit multiplication outputs, each stage comprising one or more inner product summation circuits; and a digital reduction stage coupled to an output of a last stage of the one or more stages and configured to generate an output of the multiply-accumulate device based on the L m-bit multipliers and the L m-bit multiplicands.

Integer matrix multiplication based on mixed signal circuits

Techniques for operating on and calculating binary floating-point numbers using an enhanced floating-point number format are presented. The enhanced format can comprise a single sign bit, six bits for the exponent, and nine bits for the fraction. Using six bits for the exponent can provide an enhanced exponent range that facilitates desirably fast convergence of computing-intensive algorithms and low error rates for computing-intensive applications. The enhanced format can employ a specified definition for the lowest binade that enables the lowest binade to be used for zero and normal numbers; and a specified definition for the highest binade that enables it to be structured to have one data point used for a merged Not-a-Number (NaN)/infinity symbol and remaining data points used for finite numbers. The signs of zero and merged NaN/infinity can be 'don't care' terms. The enhanced format employs only one rounding mode, which is for rounding toward nearest up.

Enhanced low precision binary floating-point formatting

A systolic array can be configured to skip distributed operands that have zero-values, resulting in improved resource efficiency. A skip module is introduced to receive operands from memory, identify whether they have a zero value or not, and, if they are nonzero, generate an operand vector including an index before sending the operand vector to a processing element.

Sparse systolic array design

In an embodiment, a method includes configuring a specialized circuit for floating point computations using numbers represented by a hybrid format, wherein the hybrid format includes a first format and a second format. In the embodiment, the method includes operating the further configured specialized circuit to store an approximation of a numeric value in the first format during a forward pass for training a deep learning network. In the embodiment, the method includes operating the further configured specialized circuit to store an approximation of a second numeric value in the second format during a backward pass for training the deep learning network.

Hybrid floating point representation for deep learning acceleration

Embodiments for implementing a fused multiply-multiply-accumulate ('FMMA') unit by one or more processors in a computing system. Mantissas for two products, an exponent difference of the two products serving as an alignment shift amount for a product of the two products having a smallest exponent, and an alignment shift amount for an addend relative to an alternative product of the two product having a larger exponent may be determined in parallel. The addend may be aligned relative to the alternative product having the larger exponent. The product having the smallest exponent may be aligned relative to the alternative product having the larger exponent according to the alignment shift amount.

Ultra-low precision floating-point fused multiply-accumulate unit

A reduced precision based programmable and single instruction multiple data (SIMD) dataflow architecture includes reduced precision execution units with a majority of the execution units operating at reduced precision and a minority of the execution units are capable of operating at higher precision. The execution units operate in parallel within a programmable execution element to share instruction fetch, decode, and issue pipelines and operate on the same instruction in lock-step to minimize instruction-related overhead.

Reduced precision based programmable and SIMD dataflow architecture

Embodiments of the present invention provide a computer-implemented method for adaptive residual gradient compression for training of a deep learning neural network (DNN). The method includes obtaining, by a first learner, a current gradient vector for a neural network layer of the DNN, in which the current gradient vector includes gradient weights of parameters of the neural network layer that are calculated from a mini-batch of training data. A current residue vector is generated that includes residual gradient weights for the mini-batch. A compressed current residue vector is generated based on dividing the residual gradient weights of the current residue vector into a plurality of bins of a uniform size and quantizing a subset of the residual gradient weights of one or more bins of the plurality of bins. The compressed current residue vector is then transmitted to a second learner of the plurality of learners or to a parameter server.

Robust gradient weight compression schemes for deep learning applications

Techniques facilitating binary floating-point multiply and scale operation for compute-intensive numerical applications and apparatuses are provided. An embodiment relates to a system that can comprise a memory that stores computer executable components and a processor that executes the computer executable components stored in the memory. The computer executable components can comprise a receiver component that receives an instruction to perform a multiply and scale operation of the first floating point operand value, the second floating point operand value, and the integer operand value, wherein the multiplication component obtains the floating-point product in response to the instruction to perform the multiply and scale operation. The multiplication can be performed as a single instruction.

Binary floating-point multiply and scale operation for compute-intensive numerical applications and apparatuses

A computer-implemented method for performing an exponential calculation using only two fully-pipelined instructions in a floating point unit that includes. The method includes computing an intermediate value y' by multiplying an input operand with a predetermined constant value. The input operand is received in floating point representation. The method further includes computing an exponential result for the input operand by executing a fused instruction. The fused instructions includes converting the intermediate value y&#x2032; to an integer representation z represented by v most significant bits (MSB), and w least significant bits (LSB). The fused instruction further includes determining exponent bits of the exponential result based on the v MSB from the integer representation z. The method further includes determining mantissa bits of the exponential result according to a piece-wise linear mapping function using a predetermined number of segments based on the w LSB from the integer representation z.

Growing community of inventors

White Plains, NY, United States of America

Ankur Agrawal