International Business Machines Corporation

Leland Stanford Junior University

Suyog Gupta

Bahman Hekmatshoartabari

Chia-Yu Chen

Shu-Jen Han

Leland Chang

Damon B Farmer

Chandrasekharan Kothandaraman

Ravi Nair

Kailash Gopalakrishnan

Jonathan Zanhong Sun

James S Harris, Jr

Robert J Chen

Jungwook Choi

Pritish Narayanan

A system, having a memory that stores computer executable components, and a processor that executes the computer executable components, reduces data size in connection with training a neural network by exploiting spatial locality to weight matrices and effecting frequency transformation and compression. A receiving component receives neural network data in the form of a compressed frequency-domain weight matrix. A segmentation component segments the initial weight matrix into original sub-components, wherein respective original sub-components have spatial weights. A sampling component applies a generalized weight distribution to the respective original sub-components to generate respective normalized sub-components. A transform component applies a transform to the respective normalized sub-components. A cropping component crops high-frequency weights of the respective transformed normalized sub-components to yield a set of low-frequency normalized sub-components to generate a compressed representation of the original sub-components.

Compression of fully connected / recurrent layers of deep network(s) through enforcing spatial locality to weight matrices and effecting frequency compression

Technical solutions are described to accelerate training of a multi-layer convolutional neural network. According to one aspect, a computer implemented method is described. A convolutional layer includes input maps, convolutional kernels, and output maps. The method includes a forward pass, a backward pass, and an update pass that each include convolution calculations. The described method performs the convolutional operations involved in the forward, the backward, and the update passes based on a first, a second, and a third perforation map respectively. The perforation maps are stochastically generated, and distinct from each other. The method further includes interpolating results of the selective convolution operations to obtain remaining results. The method includes iteratively repeating the forward pass, the backward pass, and the update pass until the convolutional neural network is trained. Other aspects such as a system, apparatus, and computer program product are also described.

Acceleration of convolutional neural network training using stochastic perforation

A method (and system) for generating random numbers includes setting a drain voltage Vd on an MOSFET (metal oxide semiconductor field effect transistor) device and a gate voltage Vg of the MOSFET device so that the MOSFET device comprises a noise source configured in a manner such as to tune as desired a random number statistical distribution of an output of the MOSFET device. An output voltage of the MOSFET is provided as an input signal into a low noise amplifier and an output voltage of the low noise amplifier provides values for a random number generator.

Random telegraph noise native device for true random number generator and noise injection

Techniques facilitating synchronization of processing engines for parallel deep learning are provided. In one example, a first processing component associated with a processor and processing components can: generate first output data based on input data associated with a machine learning process, wherein the processing components are communicatively coupled with an assignment component via a network; transmit the first output data to a second processing component of the processing components, wherein the first processing component and the second processing component comprise a first group of the processing components and the first group of the processing components is determined by the assignment component based on a first defined criterion; receive communication data generated by the second processing component; and generate second output data based on the communication data, wherein the second output data is an updated version of the first output data stored in the memory of the first processing component.

Approximate synchronization for parallel deep learning

A method (and system) for generating random numbers includes setting a drain voltage Vd on an MOSFET device to maximize a transconductance of the MOSFET device and setting a gate voltage Vg of the MOSFET device to tune as desired a random number statistical distribution of an output of the MOSFET device. The MOSFET device includes a gate structure with an oxide layer including at least one artificial trapping layer in which carrier traps are designed to occupy a predetermined distance from conduction and valance bands of material of the artificial trapping layer.

An apparatus is presented for generating a true random number generator (TRNG). The apparatus includes a magnetic tunnel junction (MTJ) device including a first layer, a second layer, and third layer, as well as a bias circuit to bias the MTJ device along with a pulse height discriminator and a time-to-amplitude convertor to generate random bit-streams. The second layer is a barrier layer with an energy barrier height in the order of 20 kT, where k is the Boltzmann constant and T is the absolute temperature. Random flipping of an orientation of magnetization of the third layer is induced by thermal fluctuations in the MTJ device.

Magnetic tunnel junction (MTJ) based true random number generators (TRNG)

A method includes forming a tunneling dielectric layer on a semiconductor substrate, a first portion of the tunneling dielectric layer is directly above a channel region in the semiconductor substrate and a second portion of the tunneling dielectric layer is directly above source-drain regions located on opposing sides of the channel region, the second portion of the tunneling dielectric layer is thicker than the first portion of the tunneling dielectric layer, forming a floating gate directly above the first portion of the tunneling dielectric layer and the second portion of the tunneling dielectric layer, and forming a control dielectric layer directly above the floating gate.

Spatially decoupled floating gate semiconductor device

Suspended structures are provided using selective etch technology. Such structures can be protected on all sides when the selective undercut etch is performed, thereby providing excellent control of feature geometry combined with superior material quality.

Growing community of inventors

White Plains, NY, United States of America

Suyog Gupta