International Business Machines Corporation

Mingu Kang

Seyoung Kim

Seonghoon Woo

Kyu-hyoun Kim

Eun Kyung Lee

Ankur Agrawal

Chia-Yu Chen

Monodeep Kar

Balaram Sinharoy

Silvia Melitta Mueller

Bruce Martin Fleischer

Kailash Gopalakrishnan

Pin-Yu Chen

Jintao Zhang

Kyu-Hyoun Kim

Sukjay Chey

Techniques for performing analog-to-digital conversion are described. For example, a method performs an analog-to-digital conversion of an analog input to a digital output including a set of bits, the set of bits including a most significant bit and one or more additional bits, the analog-to-digital conversion starting at a given one of the one or more additional bits following the most significant bit.

Analog-to-digital conversion with bit skipping functionality

An electronic circuit includes word lines; bit lines intersecting the word lines at a plurality of grid points; and resistive processing units located at the grid points. Baseline stochastic pulse input units are coupled to the word lines; differential stochastic pulse input units are coupled to the word lines; and bitline stochastic pulse input units are coupled to the bit lines. Control circuitry coupled to the pulse input units is configured to cause each of the baseline stochastic pulse input units to generate a baseline pulse train using base input data, each of the differential stochastic pulse input units to generate a differential pulse train using differential input data defining differences from the base input data, and each of the bitline stochastic pulse input units to generate a bitline pulse train using bit line input data. Neural network weights can thus be stored in the resistive processing units.

Enabling hierarchical data loading in a resistive processing unit (RPU) array for reduced communication cost

Mixed signal multipliers and methods for operating the same include a sampling capacitor and an accumulate capacitor. A sampling switch is configured to store an analog value on the sampling capacitor when a digital bit value of a digital signal is one and to store a zero when the digital bit value of the digital signal is a zero. An accumulate switch is configured to store an average of the stored value of the sampling capacitor and a previous stored value of the accumulate capacitor. A processor is configured to alternately trigger the sampling capacitor and the sampling capacitor for each bit value in the digital signal.

Dual capacitor mixed signal mutiplier

Techniques for performing improved machine learning using decision trees are disclosed. In one example, a system includes a plurality of decision tree structures, and configuration logic operatively coupled to the plurality of decision tree structures. The configuration logic selectively configures the plurality of decision tree structures to form at least one of: one or more combined decision tree structures, wherein a combined decision tree structure comprises multiple interconnected ones of the plurality of decision tree structures; and one or more individual decision tree structures, wherein an individual decision tree structure comprises a single one of the plurality of decision tree structures.

Machine learning accelerator with decision tree interconnects

A system comprises a floating-point computation unit configured to perform a dot-product operation in accordance with a first floating-point value and a second floating-point value, and detection logic operatively coupled to the floating-point computation unit. The detection logic is configured to compute a difference between fixed-point summations of exponent parts of the first floating-point value and the second floating-point value and, based on the computed difference, detect the presence of a condition prior to completion of the dot-product operation by the floating-point computation unit. In response to detection of the presence of the condition, the detection logic is further configured to cause the floating-point computation unit to avoid performing a subset of computations otherwise performed as part of the dot-product operation. Such techniques serve as a predictor that the avoided subset of computations would have resulted in a dot product of the two floating-point values being below a threshold value.

Floating-point computation with threshold prediction for artificial intelligence system

An apparatus comprises at least one processor and at least one memory including instruction code configured to, with the at least one processor, cause the apparatus at least to perform a successive approximation analog-to-digital conversion of an analog input, representing a result of multiplication of first and second vectors, to a digital output by determining an upper bound on the result of multiplication of the first and second vectors, identifying, based at least in part on the determined upper bound, at least a portion of the successive approximation analog-to-digital conversion to be skipped, and skipping the identified portion of the successive approximation analog-to-digital conversion.

Energy-efficient analog-to-digital conversion in mixed signal circuitry

A resistive memory device includes a magnetic tunnel junction structure. The magnetic tunnel junction structure includes a free magnetic layer. The free magnetic layer includes a magnetic material configurable to host topological spin textures to tune a conductance state of the resistive memory device.

Resistive memory device with magnetic layer having topological spin textures for tuning

A mixed-signal logic processor is provided. The mixed-signal logic processor includes a plurality of mixed-signal multiplier branches. Each of the plurality of mixed-signal multiplier branches has a set of branch-dedicated switches and a single branch-dedicated capacitor. The mixed-signal logic further includes a common switch. The common switch is external and common to each of the plurality of mixed-signal multiplier branches. The mixed-signal logic also includes a first shared branch-external capacitor and a second shared branch-external capacitor. The first and the second shared branch-external capacitors are external to and shared by each of the plurality of mixed-signal multiplier branches. Various settings of the set of switches and the common switch enable various modes of the mixed-signal dot product processor.

Mixed-signal dot product processor with single capacitor per multiplier

Methods, systems, and circuits for training a neural network include applying noise to a set of training data across wordlines using a respective noise switch on each wordline. A neural network is trained using the noise-applied training data to generate a classifier that is robust against adversarial training.

Integrated noise generation for adversarial training

A computer-implemented method for improving the efficiency of computing an activation function in a neural network system includes initializing, by a controller, weights in a weight vector associated with the neural network system. Further, the method includes receiving, by the controller, an input vector of input values for computing a dot product with the weight vector for the activation function, which determines an output value of a node in the neural network system. The method further includes predicting, by a rectifier linear unit (ReLU), which computes the activation function, that the output value of the node will be negative based on computing an intermediate value for computing the dot product, and based on a magnitude of the intermediate value exceeding a precomputed threshold value. Further, the method includes, in response to the prediction, terminating, by the ReLU, the computation of the dot product, and outputting a 0 as the output value.

Activation function computation for neural networks

Embodiments for implementing a fused multiply-multiply-accumulate ('FMMA') unit by one or more processors in a computing system. Mantissas for two products, an exponent difference of the two products serving as an alignment shift amount for a product of the two products having a smallest exponent, and an alignment shift amount for an addend relative to an alternative product of the two product having a larger exponent may be determined in parallel. The addend may be aligned relative to the alternative product having the larger exponent. The product having the smallest exponent may be aligned relative to the alternative product having the larger exponent according to the alignment shift amount.

Ultra-low precision floating-point fused multiply-accumulate unit

A system is provided for error resiliency in a bit serial computation. A delay monitor enforces an overall processing duration threshold for bit-serial processing all iterations for the bit serial computation, while determining a threshold for processing each iteration. At least some iterations correspond to a respective bit in an input bit sequence. A clock generator generates a clock signal for controlling a performance of the iterations. Each of iteration units perform a particular iteration, starting with a Most Significant Bit (MSB) of the input bit sequence and continuing in descending bit significant order, and by selectively increasing the threshold for at least one iteration while skipping from processing at least one subsequent iteration whose iteration-level processing duration exceeds a remaining amount of an overall processing duration for all iterations, responsive to the at least one iteration requiring more time to complete than a current value of the threshold.

Bit-serial computation with dynamic frequency modulation for error resiliency in neural network

A differential mixed-signal logic processor is provided. The differential mixed-signal logic processor includes a plurality of mixed-signal multiplier branches for multiplication of an analog value A and a N-bit digital value B. Each of the plurality of mixed-signal multiplier branches include a first capacitor connected across a second capacitor and a third capacitor to provide a differential output across the second and third capacitors. A capacitance of the first capacitor is equal to half a capacitance of the second and third capacitors.

Growing community of inventors

Old Tappan, NJ, United States of America

Mingu Kang