Hewlett Packard Enterprise Development LP

Google Inc.

Hewlett-packard Development Company, L.p.

Doe Hyun Yoon

Norman Paul Jouppi

Jichuan Chang

Naveen Muralimanohar

Parthasarathy Ranganathan

Robert S Schreiber

Sheng Li

Paolo Faraboschi

Nishant P Patil

Lifeng Nai

Dejan S Milojicic

Moray McLaren

Dwight L Barron

Terence Patrick Kelly

Jishen Zhao

Ke Chen

Peter Ma

Parthasarathy Ranganthan

Nishant Patil

The technology relates to a neural network dual in-line memory module (NN-DIMM), a microelectronic system comprising a CPU and a plurality of the NN-DIMMs, and a method of transferring information between the CPU and the plurality of the NN-DIMMS. The NN-DIMM may include a module card having a plurality of parallel edge contacts adjacent to an edge of a slot connector thereof and configured to have the same command and signal interface as a standard dual in-line memory module (DIMM). The NN-DIMM may also include a deep neural network (DNN) accelerator affixed to the module card, and a bridge configured to transfer information between the DNN accelerator and the plurality of parallel edge contacts via a DIMM external interface.

Neural network accelerator in DIMM form factor

The subject matter described herein provides systems and techniques for the design and use of multiply-and-accumulate (MAC) units to perform matrix multiplication by systolic arrays, such as those used in accelerators for deep neural networks (DNNs). These MAC units may take advantage of the particular way in which matrix multiplication is performed within a systolic array. For example, when a matrix A is multiplied with a matrix B, the scalar value, a, of the matrix A is reused many times, the scalar value, b, of the matrix B may be streamed into the systolic array and forwarded to a series of MAC units in the systolic array, and only the final values and not the intermediate values of the dot products, computed for the matrix multiplication, may be correct. MAC unit hardware that is particularized to take advantage of these observations is described herein.

Multiplier and adder in systolic array

Aspects of the disclosure are directed to a computation unit implementing a systolic array and configured for detecting errors while processing data on the systolic array. Checksum circuit in communication with a systolic array is configured to compute checksums and perform error detection while the systolic array processes input data. Instead of pre-generating checksums in input matrices, input matrices can be directly fed into the systolic array through the checksum circuit. On the output side, the checksum circuit can generate and compare checksums with checksums in an output matrix generated by the systolic array. Error checking the operations to generate the output matrix can be performed without delaying the operations of the systolic array, and without preprocessing the input matrices.

Error checking for systolic array computation

Methods, systems, and apparatus for updating machine learning models to improve locality are described. In one aspect, a method includes receiving data of a machine learning model. The data represents operations of the machine learning model and data dependencies between the operations. Data specifying characteristics of a memory hierarchy for a machine learning processor on which the machine learning model is going to be deployed is received. The memory hierarchy includes multiple memories at multiple memory levels for storing machine learning data used by the machine learning processor when performing machine learning computations using the machine learning model. An updated machine learning model is generated by modifying the operations and control dependencies of the machine learning model to account for the characteristics of the memory hierarchy. Machine learning computations are performed using the updated machine learning model.

Modifying machine learning models to improve locality

A first memory controller receives an access command from a second memory controller, where the access command is timing non-deterministic with respect to a timing specification of a memory. The first memory controller sends at least one access command signal corresponding to the access command to the memory, wherein the at least one access command signal complies with the timing specification. The first memory controller determines a latency of access of the memory. The first memory controller sends feedback information relating to the latency to the second memory controller.

Separate memory controllers to access data in memory

A memory controller is to interface with a memory, associated with a plurality of pins, based on a codeword. The codeword is to include a plurality of n-bit symbols. An n-bit symbol of the codeword is to be formed from a plurality of n bursts over time associated with one of the pins of the memory.

Memory controllers to form symbols based on bursts

An example method involves receiving, at a first memory node, data to be written at a memory location in the first memory node. The data is received from a device. At the first memory node, old data is read from the memory location, without sending the old data to the device. The data is written to the memory location. The data and the old data are sent from the first memory node to a second memory node to store parity information in the second memory node without the device determining the parity information. The parity information is based on the data stored in the first memory node.

Smart memory buffers

Example methods, systems, and apparatus to provide selective memory error protection and memory access granularity are disclosed herein. An example system includes a memory controller to determine a selected memory mode based on a request. The memory mode indicates that a memory page is to store a corresponding type of error protection information and is to store data for retrieval using a corresponding access granularity. The memory controller is to store the data and the error protection information in the memory page for retrieval using the error protection information and the access granularity.

Selective error correcting code and memory access granularity switching

Example methods and systems to provide persistent memory are disclosed herein. An example system includes a nonvolatile cache to store data received from a volatile cache. The data is associated with a transaction and the data is to be identified as durable when the transaction is committed. The example system includes a nonvolatile memory to store the data received from the nonvolatile cache when the data is identified as durable.

Multiversioned nonvolatile memory hierarchy for persistent memory

Local checkpointing using a multi-level call is described herein. An example method includes storing a first datum in a first level of a multi-level cell. A second datum is stored in a second level of the multi-level cell, the second datum representing a checkpoint of the first datum. The first datum is copied from the first level to the second level of the multi-level cell to create the checkpoint.

Local checkpointing using a multi-level cell

A method that includes evaluating, with a controller, local error detection (LED) information in response to a first memory access operation is disclosed. The LED information is evaluated per cache line segment of data associated with a rank of a memory. The method further includes determining an error in at least one of the cache line segments based on an error detection code and determining whether global error correction (GEC) data for a first cache line associated with the at least one cache line segment is stored in a GEC cache in the controller. The method also includes correcting the first cache line associated with the at least one cache line segment based on the GEC data retrieved from the GEC cache in the controller without accessing GEC data from a memory.

Growing community of inventors

Palo Alto, CA, United States of America

Doe Hyun Yoon