Nvidia Corporation

Yakun Shao

Joel S Emer

Stephen William Keckler

William James Dally

Brucek Kurdo Khailany

Rangharajan Venkatesan

Daniel Smith

Miaorong Wang

Nan Jiang

Brian Matthew Zimmer

Matthew Rudolph Fojtik

Angshuman Parashar

Jason Lavar Clemons

Ching-En Lee

Nathaniel Pinckney

A distributed deep neural net (DNN) utilizing a distributed, tile-based architecture includes multiple chips, each with a central processing element, a global memory buffer, and a plurality of additional processing elements. Each additional processing element includes a weight buffer, an activation buffer, and vector multiply-accumulate units to combine, in parallel, the weight values and the activation values using stationary data flows.

Efficient neural network accelerator dataflows

A sparse convolutional neural network accelerator system that dynamically and efficiently identifies fine-grained parallelism in sparse convolution operations. The system determines matching pairs of non-zero input activations and weights from the compacted input activation and weight arrays utilizing a scalable, dynamic parallelism discovery unit (PDU) that performs a parallel search on the input activation array and the weight array to identify reducible input activation and weight pairs.

Deep neural network accelerator with fine-grained parallelism discovery

A distributed deep neural net (DNN) utilizing a distributed, tile-based architecture implemented on a semiconductor package. The package includes multiple chips, each with a central processing element, a global memory buffer, and processing elements. Each processing element includes a weight buffer, an activation buffer, and multiply-accumulate units to combine, in parallel, the weight values and the activation values.

Scalable multi-die deep learning system

Growing community of inventors

Santa Clara, CA, United States of America

Yakun Shao