Oracle America, Inc.

Sun Microsystems, Inc.

Neuralmagic Inc.

Oracle International Corporation

Massachusetts Institute of Technology

Harvard College

Nir N Shavit

David Dice

Mark S Moir

Victor M Luchangco

Alexander Matveev

Ori Shalev

Guy Lewis Steele, Jr

Yosef Lev

Virendra J Marathe

Maurice P Herlihy

Christine H Flood

Ole Agesen

Steven K Heller

Daniel Nussbaum

David L Detlefs

Paul A Martin

Xiaolan Zhang

Govind Ramnarayan

Alexander T Garthwaite

Guy L Steele

Jeff Lichtman

Justin Kopinsky

Mark Kurtz

Dan Alistarh

Rati Gelashvili

Aravinathan Samuel

Lu Mi

Antonios Printezis

Irina Calciu

Edya Ladan-Mozes

Sage Moore

Mark A Moir

Tyler Michael Smith

Embodiments of the invention may execute a NN by executing sub-tensor columns, each sub-tensor column including computations from portions of a layers of the NN, and each sub-tensor column performing computations entirely within a first layer of cache (e.g. Lin one embodiment) and saving its output entirely within a second layer of cache (e.g. Lin one embodiment). Embodiments may include partitioning the execution of a NN by partitioning the execution of the NN into sub-tensor columns, each sub-tensor column including computations from portions of layers of the NN, each sub-tensor column performing computations entirely within a first layer of cache and saving its output entirely within a second layer of cache.

System and method of determining and executing deep tensor columns in neural networks

A system and method may partition and/or execute a NN, by, for a graph including nodes and hyper edges, each node representing a data item in the NN and each hyper edge representing an operation in the NN, identifying a deep tensor column comprising a subset of the nodes and a subset of the hyper edges, such that the operations in the deep tensor column, when executed, use only data which fits within a preselected cache.

A method and system for computing one or more outputs of a neural network having a plurality of layers is provided. The method and system can include determining a plurality of sub-computations from total computations of the neural network to execute in parallel wherein the computations to execute in parallel involve computations from multiple layers. The method and system also can also include avoiding repeating overlapped computations and/or multiple memory reads and writes during execution.

Systems and methods for improved neural network execution

A system and method of accelerating execution of a NN model, by at least one processor may include: receiving a first matrix A, representing elements of a kernel K of the NN model and a second matrix B, representing elements of an input I to kernel K; producing from matrix A, a group-sparse matrix A', comprising G tensors of elements. The number of elements in each tensor is defined by, or equal to a number of entries in each index of an input tensor register used for a specific Single Instruction Multiple Data (SIMD) tensor operation, and all elements of A&#x2032; outside said G tensors are null. The system and method may further include executing kernel K on input I, by performing at least one computation of the SIMD tensor operation, having as operands elements of a tensor of the G tensors and corresponding elements of the B matrix.

System and method of accelerating execution of a neural network

Systems and methods may make exchanging data in a neural network (NN) during training more efficient. Exchanging weights among a number of processors training a NN across iterations may include sorting generated weights, compressing the sorted weights, and transmitting the compressed sorted weights. On each Kth iteration a sort order of the sorted weights may be created and transmitted. Exchanging weights among processors training a NN may include executing a forward pass to produce a set of loss values for processors, transmitting loss values to other processors, and at each of the processors, performing backpropagation on at least one layer of the NN using loss values received from other processors.

Systems and methods for exchange of data in distributed training of machine learning algorithms

A system and method is provided for rapidly collecting high quality images of a specimen through controlling a re-focusable beam of an electron microscope. An intelligent acquisition system instructs the electron microscope to perform an initial low-resolution scan of a sample. A low-resolution image of the sample is received by the intelligent acquisition system as scanned image information from the electron microscope. The intelligent acquisition system then determines regions of interest within the low-resolution image and instructs the electron microscope to perform a high-resolution scan of the sample, only in areas of the sample corresponding to the determined regions of interest or portions of the determined regions of interest, so that other regions within the sample are not scanned at high-resolution, where the high-resolution scanning in the regions of interest is guided by a probability map using a deep neural network for segmentation.

System and method for learning-guided electron microscopy

Embodiments of the invention may execute a NN by executing sub-tensor columns, each sub-tensor column including computations from portions of a layers of the NN, and each sub-tensor column performing computations entirely within a first layer of cache (e.g. L2 in one embodiment) and saving its output entirely within a second layer of cache (e.g. L3 in one embodiment). Embodiments may include partitioning the execution of a NN by partitioning the execution of the NN into sub-tensor columns, each sub-tensor column including computations from portions of layers of the NN, each sub-tensor column performing computations entirely within a first layer of cache and saving its output entirely within a second layer of cache.

System and method of executing deep tensor columns in neural networks

A system and method is provided for rapidly collecting high quality images of a specimen through controlling a re-focusable beam of an electron microscope. An intelligent acquisition system instructs the electron microscope to perform an initial low-resolution scan of a sample. A low-resolution image of the sample is received by the intelligent acquisition system as scanned image information from the electron microscope. The intelligent acquisition system then determines regions of interest within the low-resolution image and instructs the electron microscope to perform a high-resolution scan of the sample, only in areas of the sample corresponding to the determined regions of interest or portions of the determined regions of interest, so that other regions within the sample are not scanned at high-resolution. The intelligent acquisition system then reconstructs an image using the collected high-resolution scan pixels and pixels in the received low-resolution image.

A system and method of inferring a neural network (NN) on one or more target computing devices. The NN may include a plurality of layers, where at least one layer includes one or more kernels. Embodiments may include: receiving a data structure representing the NN; analyzing the data structure to produce one or more tasks, where each task may include computations pertaining to a kernel of the NN; selecting a sparse version of at least one kernel and replacing the at least one kernel with the sparse version; and compiling the one or more tasks to produce one or more respective tensor columns, The one or more tensor columns are adapted to fit in respective one or more cache memories of the one or more target computing devices, and include task instruction code that represents at least one computation of the kernel of the NN.

System and method of executing neural networks

A system and method for convolutional layer in convolutional neural networks is provided. The convolution is performed via a transformation that includes relocating input, relocating convolution filters and performing an aggregate matrix multiply.

Methods and systems for improved transforms in convolutional neural networks

Efficiency with respect to traditional techniques is a key issue facing designers of software transactional synchronization mechanisms. Meta-transactional synchronization allows integration of transactional support into an object-oriented programming language, such as the Java language through the existing synchronization structure of the JVM. Meta-transactional synchronization provides source-level transactional operations that co-exist with synchronized operations. An implementation of a shared object in an object-oriented programming language tracks concurrently executing transactions attempting to access the shared object with at least one header word of the shared object.

Meta-transactional synchronization

Hybrid transaction memory systems and accompanying methods. A transaction to be executed is received, and an initial attempt is made to execute the transaction in a hardware path. Upon a failure to successfully execute the transaction in the hardware path, an attempt is made to execute the transaction in a hardware-software path. The hardware-software path includes a software path and at least one hardware transaction.

Methods and systems for enhancing hardware transactions using hardware transactions in software slow-path

Growing community of inventors

Cambridge, MA, United States of America

Nir N Shavit