Intel Corporation

Nvidia Corporation

Digital Equipment Corporation

Hewlett-packard Development Company, L.p.

Compaq Computer Corporation, Inc.

Compaq Information Technologies Group, L.p.

Joel S Emer

Simon C Steely, Jr

Shubhendu Sekhar Mukherjee

Stephen William Keckler

William C Hasenplaugh

William James Dally

Steven Karl Reinhardt

Aamer Jaleel

Michael C Adler

Francis X McKeen

Angshuman Parashar

Larry Robert Dennison

Christopher T Weaver

David J Sager

Tryggve Fossum

Edward J McLellan

Robert P Nix

Yakun Shao

Jesus Corbal

Brucek Kurdo Khailany

Bruce Edwards

Federico Ardanaz

Roger Gramunt

Roger Espasa

George Z Chrysos

Rangharajan Venkatesan

Geoff Lowney

Santiago Galan

Toni Juan

P Geoffrey Lowney

Isaac Hernandez

James E Hicks, Jr

Arijit Biswas

Robert H Halstead, Jr

Henry M Levy

Susan Jane Eggers

Matthew H Reilly

Craig B Zilles

Christopher F Joerg

Samantika Subramaniam

Neal C Crago

Rebecca L Stamm

Trggve Fossum

Rebecca Stamm

Dean Tullsen

Simon Steely

Bushra Ahsan

Michael J Smith

Miaorong Wang

Michael I Pellauer

Daniel Smith

John H Edmondson

Matthew Mattina

Derrick R Meyer

George Zacharias Chrysos

John H Mylius

Malini Krishnan Bhandaru

Samantika S Sury

Paolo L Narvaez

Ricky C Hetherington

Daniel Lawrence Leibholz

Nan Jiang

Brian Matthew Zimmer

William R Grundmann

Eric R Borch

David A Webb, Jr

Matthew Rudolph Fojtik

Steven E Raasch

Paul B Racunas

David A Webb

Bruce E Edwards

Joseph Dominic Macri

Harish G Patil

Stephen Felix

Ching-En Lee

Jason Lavar Clemons

Kenzo Van Craeynest

Chi-Keung Luk

Chuan-Hua Chang

Joseph D Marci

P Geoffrey Lowny

Carole-Jean Wu

Scott McFarling

Nathaniel Pinckney

Antonio Juan-Hormigo

Ramon Matas-Navarro

Sudhanva Gurumurthi

Peter Soderquist

A distributed deep neural net (DNN) utilizing a distributed, tile-based architecture includes multiple chips, each with a central processing element, a global memory buffer, and a plurality of additional processing elements. Each additional processing element includes a weight buffer, an activation buffer, and vector multiply-accumulate units to combine, in parallel, the weight values and the activation values using stationary data flows.

Efficient neural network accelerator dataflows

A sparse convolutional neural network accelerator system that dynamically and efficiently identifies fine-grained parallelism in sparse convolution operations. The system determines matching pairs of non-zero input activations and weights from the compacted input activation and weight arrays utilizing a scalable, dynamic parallelism discovery unit (PDU) that performs a parallel search on the input activation array and the weight array to identify reducible input activation and weight pairs.

Deep neural network accelerator with fine-grained parallelism discovery

A method, computer program product, and system perform computations using a processor. A first instruction including a first index vector operand and a second index vector operand is received and the first index vector operand is decoded to produce first coordinate sets for a first array, each first coordinate set including at least a first coordinate and a second coordinate of a position of a non-zero element in the first array. The second index vector operand is decoded to produce second coordinate sets for a second array, each second coordinate set including at least a third coordinate and a fourth coordinate of a position of a non-zero element in the second array. The first coordinate sets are summed with the second coordinate sets to produce output coordinate sets and the output coordinate sets are converted into a set of linear indices.

Sparse convolutional neural network accelerator

A distributed deep neural net (DNN) utilizing a distributed, tile-based architecture implemented on a semiconductor package. The package includes multiple chips, each with a central processing element, a global memory buffer, and processing elements. Each processing element includes a weight buffer, an activation buffer, and multiply-accumulate units to combine, in parallel, the weight values and the activation values.

Scalable multi-die deep learning system

A method, computer program product, and system perform computations using a sparse convolutional neural network accelerator. Compressed-sparse data is received for input to a processing element, wherein the compressed-sparse data encodes non-zero elements and corresponding multi-dimensional positions. The non-zero elements are processed in parallel by the processing element to produce a plurality of result values. The corresponding multi-dimensional positions are processed in parallel by the processing element to produce destination addresses for each result value in the plurality of result values. Each result value is transmitted to a destination accumulator associated with the destination address for the result value.

A method, computer program product, and system perform computations using a sparse convolutional neural network accelerator. A first vector comprising only non-zero weight values and first associated positions of the non-zero weight values within a 3D space is received. A second vector comprising only non-zero input activation values and second associated positions of the non-zero input activation values within a 2D space is received. The non-zero weight values are multiplied with the non-zero input activation values, within a multiplier array, to produce a third vector of products. The first associated positions are combined with the second associated positions to produce a fourth vector of positions, where each position in the fourth vector is associated with a respective product in the third vector. The products in the third vector are transmitted to adders in an accumulator array, based on the position associated with each one of the products.

A technology for implementing a method for distributed memory operations. A method of the disclosure includes obtaining distributed channel information for an algorithm to be executed by a plurality of spatially distributed processing elements. For each distributed channel in the distributed channel information, the method further associates one or more of the plurality of spatially distributed processing elements with the distributed channel based on the algorithm.

Executing distributed memory operations using processing elements connected by distributed channels

A processing device for executing distributed memory operations using spatial processing units (SPU) connected by distributed channels is disclosed. A distributed channel may or may not be associated with memory operations, such as load operations or store operations. Distributed channel information is obtained for an algorithm to be executed by a group of spatially distributed processing elements. The group of spatially distributed processing elements can be connected to a shared memory controller. For each distributed channel in the distributed channel information, one or more of the group of spatially distributed processing elements may be associated with the distributed channel based on the algorithm. By associating the spatially distributed processing elements to a distributed channel, the functionality of the processing element can vary depending on the algorithm mapped onto the SPU.

A micro-architecture may provide a hardware and software of a high bandwidth write command. The micro-architecture may invoke a method to perform the high bandwidth write command. The method may comprise sending a write request from a requester to a record keeping structure. The write request may have a memory address of a memory that stores requested data. The method may further determine copies of the requested data being present in a distributed cache system outside the memory, sending invalidation requests to elements holding copies of the requested data in the distributed cache system, sending a notification to the requester to inform presence of copies of the requested data and sending a write response message after a latest value of the requested data and all invalidation acknowledgements have been received.

High bandwidth full-block write commands

Methods and apparatuses to control cache line coherence are described. A hardware processor may include a first processor core with a cache to store a cache line, a second set of processor cores that each include a cache to store a copy of the cache line, and cache coherence logic to aggregate in a tag directory an acknowledgment message from each of the second set of processor cores in response to a request from the first processor core to modify the copy of the cache line in each of the second set of processor cores and send a consolidated acknowledgment message to the first processor core.

Hardware apparatuses and methods to control cache line coherence

Method and apparatus to efficiently maintain cache coherency by reading/writing a domain state field associated with a tag entry within a cache tag directory. A value may be assigned to a domain state field of a tag entry in a cache tag directory. The cache tag directory may belong to a hierarchy of cache tag directories. Each tag entry may be associated with a cache line from a cache belonging to a first domain. The first domain may contain multiple caches. The value of the domain state field may indicate whether its associated cache line can be read or changed.

Domain state

A method and apparatus to reduce unnecessary write backs of cached data to a main memory and to optimize the usage of a cache memory tag directory. In one embodiment of the invention, the power consumption of a processor can be saved by eliminating write backs of cache memory lines that has information that has reached its end-of-life. In one embodiment of the invention, when a processing unit is required to clear one or more cache memory lines, it uses a write-zero command to clear the one or more cache memory lines. The processing unit does not perform a write operation to move or pass data values of zero to the one or more cache memory lines. By doing so, it reduces the power consumption of the processing unit.

Method and apparatus for optimizing the usage of cache memories

Growing community of inventors

Acton, MA, United States of America

Joel S Emer