Inphi Corporation

D-matrix Corporation

Siddharth Sheth

Radhakrishnan L Nagarajan

Sudeep Bhoja

An AI accelerator apparatus using in-memory compute chiplet devices. The apparatus includes one or more chiplets, each of which includes a plurality of tiles. Each tile includes a plurality of slices, a central processing unit (CPU), and a hardware dispatch device. Each slice can include a digital in-memory compute (DIMC) device configured to perform high throughput computations. In particular, the DIMC device can be configured to accelerate the computations of attention functions for transformer-based models (a.k.a. transformers) applied to machine learning applications. A single input multiple data (SIMD) device configured to further process the DIMC output and compute softmax functions for the attention functions. The chiplet can also include die-to-die (D2D) interconnects, a peripheral component interconnect express (PCIe) bus, a dynamic random access memory (DRAM) interface, and a global CPU interface to facilitate communication between the chiplets, memory and a server or host system.

AI accelerator apparatus using in-memory compute chiplet devices for transformer workloads

An AI accelerator apparatus using in-memory compute chiplet devices. The apparatus includes one or more chiplets, each of which includes a plurality of tiles. Each tile includes a plurality of slices, a central processing unit (CPU), and a hardware dispatch device. Each slice can include a digital in-memory compute (DIMC) device configured to perform high throughput computations. In particular, the DIMC device can be configured to accelerate the computations of attention functions for transformer-based models (a.k.a. transformers) applied to machine learning applications, including generative AI. A single input multiple data (SIMD) device configured to further process the DIMC output and compute softmax functions for the attention functions. The chiplet can also include die-to-die (D2D) interconnects, a peripheral component interconnect express (PCIe) bus, a dynamic random access memory (DRAM) interface, and a global CPU interface to facilitate communication between the chiplets, memory and a server or host system.

Generative AI accelerator apparatus using in-memory compute chiplet devices for transformer workloads

Ai accelerator apparatus using in-memory compute chiplet devices for transformer workloads

The present invention provides an integrated system-on-chip device. The device is configured on a single silicon substrate member. The device has a data input/output interface provided on the substrate member. The device has an input/output block provided on the substrate member and coupled to the data input/output interface. The device has a signal processing block provided on the substrate member and coupled to the input/output block. The device has a driver module provided on the substrate member and coupled to the signal processing block. The device further includes a driver interface and coupled to the driver module and configured to be coupled to a silicon photonics device. In an example, a control block is configured to receive and send instruction(s) in a digital format to the communication block and is configured to receive and send signals in an analog format to communicate with the silicon photonics device.

Growing community of inventors

Santa Clara, CA, United States of America

Siddharth Sheth