Samsung Electronics Co., Ltd.

Krishna Teja Malladi

Hongzhong Zheng

Dimin Niu

Peng Gu

Mu-Tien Chang

Andrew Zhenwen Chang

Jongmin Gim

Yang Seok Ki

Robert Brennan

Wenqin Huangfu

Bob Brennan

Rekha Pitchumani

Shuangchen Li

Dongyan Jiang

Ehsan M Najafabadi

Changhui Lin

Shaungchen Li

Mingyu Gao

Manu Awasthi

Marie Mai Nguyen

Shiyu Li

Qiumin Xu

Zongwang Li

Chaohong Hu

Jinhyun Kim

Uksong Kang

Suhas

Byung Choi

Hyungseuk Kim

Rakesh Ramesh

A coherent memory system. In some embodiments, the coherent memory system includes a first memory device. The first memory device may include a cache coherent controller; a volatile memory controller; a volatile memory; a nonvolatile memory controller; and a nonvolatile memory. The first memory device may be configured to receive a quality of service requirement and to selectively enable a first feature in response to the quality of service requirement.

Coherent memory system

A high-bandwidth memory (HBM) includes a memory and a controller. The controller receives a data write request from a processor external to the HBM and the controller stores an entry in the memory indicating at least one address of data of the data write request and generates an indication that a data bus is available for an operation during a cycle time of the data write request based on the data write request comprising sparse data or data-value similarity. Sparse data includes a predetermined percentage of data values equal to zero, and data-value similarity includes a predetermined amount of spatial value locality of the data values. The predetermined percentage of data values equal to zero of sparse data and the predetermined amount of spatial value locality of the special-value pattern are both based on a predetermined data granularity.

High bandwidth memory system

A method for in-memory computing. In some embodiments, the method includes: executing, by a first function-in-memory circuit, a first instruction, to produce, as a result, a first value, wherein a first computing task includes a second computing task and a third computing task, the second computing task including the first instruction; storing, by the first function-in-memory circuit, the first value in a first buffer; reading, by a second function-in-memory circuit, the first value from the first buffer; and executing, by a second function-in-memory circuit, a second instruction, the second instruction using the first value as an argument, the third computing task including the second instruction, wherein: the storing, by the first function-in-memory circuit, of the first value in the first buffer includes directly storing the first value in the first buffer.

Adjustable function-in-memory computation system

A system for computing. In some embodiments, the system includes: a memory, the memory including one or more function-in-memory circuits; and a cache coherent protocol interface circuit having a first interface and a second interface. A function-in-memory circuit of the one or more function-in-memory circuits may be configured to perform an operation on operands including a first operand retrieved from the memory, to form a result. The first interface of the cache coherent protocol interface circuit may be connected to the memory, and the second interface of the cache coherent protocol interface circuit may be configured as a cache coherent protocol interface on a bus interface.

In-memory computing with cache coherent protocol

A processor includes a plurality of memory units, each of the memory units including a plurality of memory cells, wherein each of the memory units is configurable to operate as memory, as a computation unit, or as a hybrid memory-computation unit.

Scaling out architecture for dram-based processing unit (DPU)

A system and method for training a neural network. In some embodiments, the system includes a computational storage device including a backing store. The computational storage device may be configured to: store, in the backing store, an embedding table for a neural network embedding operation; receive a first index vector including a first index and a second index; retrieve, from the backing store: a first row of the embedding table, corresponding to the first index, and a second row of the embedding table, corresponding to the second index; and calculate a first embedded vector based on the first row and the second row.

Neural network training with acceleration

A high bandwidth memory system. In some embodiments, the system includes: a memory stack having a plurality of memory dies and eight 128-bit channels; and a logic die, the memory dies being stacked on, and connected to, the logic die; wherein the logic die may be configured to operate a first channel of the 128-bit channels in: a first mode, in which a first 64 bits operate in pseudo-channel mode, and a second 64 bits operate as two 32-bit fine-grain channels, or a second mode, in which the first 64 bits operate as two 32-bit fine-grain channels, and the second 64 bits operate as two 32-bit fine-grain channels.

Bandwidth boosted stacked memory

According to one embodiment, a memory module includes: a memory die including a dynamic random access memory (DRAM) banks, each including: an array of DRAM cells arranged in pages; a row buffer to store values of one of the pages; an input/output (IO) module; and an in-memory compute (IMC) module including: an arithmetic logic unit (ALU) to receive operands from the row buffer or the IO module and to compute an output based on the operands and one of a plurality of ALU operations; and a result register to store the output of the ALU; and a controller to: receive, from a host processor, operands and an instruction; determine, based on the instruction, a data layout; supply the operands to the DRAM banks in accordance with the data layout; and control an IMC module to perform one of the ALU operations on the operands in accordance with the instruction.

Systems and methods for data placement for in-memory-compute

An accelerator is disclosed. A tier storage may store data. A circuit may process the data to produce a processed data. The accelerator may load the data from a device using a cache-coherent interconnect protocol.

Low-latency input data staging to execute kernels

According to one general aspect, an apparatus may include a plurality of stacked integrated circuit dies that include a memory cell die and a logic die. The memory cell die may be configured to store data at a memory address. The logic die may include an interface to the stacked integrated circuit dies and configured to communicate memory accesses between the memory cell die and at least one external device. The logic die may include a reliability circuit configured to ameliorate data errors within the memory cell die. The reliability circuit may include a spare memory configured to store data, and an address table configured to map a memory address associated with an error to the spare memory. The reliability circuit may be configured to determine if the memory access is associated with an error, and if so completing the memory access with the spare memory.

HBM RAS cache architecture

A general matrix-matrix multiplication (GEMM) dataflow accelerator circuit is disclosed that includes a smart 3D stacking DRAM architecture. The accelerator circuit includes a memory bank, a peripheral lookup table stored in the memory bank, and a first vector buffer to store a first vector that is used as a row address into the lookup table. The circuit includes a second vector buffer to store a second vector that is used as a column address into the lookup table, and lookup table buffers to receive and store lookup table entries from the lookup table. The circuit further includes adders to sum the first product and a second product, and an output buffer to store the sum. The lookup table buffers determine a product of the first vector and the second vector without performing a multiply operation. The embodiments include a hierarchical lookup architecture to reduce latency. Accumulation results are propagated in a systolic manner.

Dataflow accelerator architecture for general matrix-matrix multiplication and tensor computation in deep learning

A system and method for training a neural network. In some embodiments, the system includes: a graphics processing unit cluster; and a computational storage cluster connected to the graphics processing unit cluster by a cache-coherent system interconnect. The graphics processing unit cluster may include one or more graphics processing units. The computational storage cluster may include one or more computational storage devices. A first computational storage device of the one or more computational storage devices may be configured to (i) store an embedding table, (ii) receive an index vector including a first index and a second index; and (iii) calculate an embedded vector based on: a first row of the embedding table, corresponding to the first index, and a second row of the embedding table, corresponding to the second index.

A storage device and method of controlling a storage device are disclosed. The storage device includes a host, a logic die, and a high bandwidth memory stack including a memory die. A computation lookup table is stored on a memory array of the memory die. The host sends a command to perform an operation utilizing a kernel and a plurality of input feature maps, includes finding the product of a weight of the kernel and values of multiple input feature maps. The computation lookup table includes a row corresponding to a weight of the kernel, and a column corresponding to a value of the input feature maps. A result value stored at a position corresponding to a row and a column is the product of the weight corresponding to the row and the value corresponding to the column.

HBM based memory lookup engine for deep learning accelerator

A high bandwidth memory (HBM) system includes a first HBM+ card. The first HBM+ card includes a plurality of HBM+ cubes. Each HBM+ cube has a logic die and a memory die. The first HBM+ card also includes a HBM+ card controller coupled to each of the plurality of HBM+ cubes and configured to interface with a host, a pin connection configured to connect to the host, and a fabric connection configured to connect to at least one HBM+ card.

Growing community of inventors

San Jose, CA, United States of America

Krishna Teja Malladi