Nvidia Corporation

Calos Fund Limited Liability Company

Calos Fund, LLC

Stream Processors, Inc.

Brucek Kurdo Khailany

William James Dally

Rangharajan Venkatesan

John Wood Poulton

Haoxing Ren

Stephen William Keckler

Thomas Hastings Greer

Joel S Emer

Carl Thomas Gray

Ujval J Kapasi

Brian Matthew Zimmer

Jim Jian Lin

Yakun Shao

Steve Haihang Dai

Steven Lee Scott

Michael A Parker

Ronny Krashinsky

Matthew Rudolph Fojtik

James David Balfour

Benjamin Andrew Keller

Miaorong Wang

Daniel Smith

John H Edmondson

Sean Jeffrey Treichler

Raghunath Krishna Rao

Stephen G Tell

Mark P Rygh

Nan Jiang

Karthikeyan Natarajan

Udo Uebel

Mark Alan Gebhart

Mahmut Ersin Sinangil

DeForest Tovey

Harbinder Sikka

Lijuan Luo

Zhiyao Xie

Jason Lavar Clemons

Yanqing Zhang

Yuzhe Ma

Mark Ren

Nathaniel Pinckney

Charbel Sakr

Vidya Chhabria

Sheng Ye

Today neural networks are used to enable autonomous vehicles and improve the quality of speech recognition, real-time language translation, and online search optimizations. However, operation of the neural networks for these applications consumes energy. Quantization of parameters used by the neural networks reduces the amount of memory needed to store the parameters while also reducing the power consumed during operation of the neural network. Matrix operations performed by the neural networks require many multiplication calculations, so reducing the number of bits that are multiplied reduces the energy that is consumed. Quantizing smaller sets of the parameters using a shared scale factor improves accuracy compared with quantizing larger sets of the parameters. Accuracy of the calculations may be maintained by quantizing and scaling the parameters using fine-grained per-vector scale factors. A vector includes one or more elements within a single dimension of a multi-dimensional matrix.

Fine-grained per-vector scaling for neural network quantization

Quantizing tensors and vectors processed within a neural network reduces power consumption and may accelerate processing. Quantization reduces the number of bits used to represent a value, where decreasing the number of bits used can decrease the accuracy of computations that use the value. Ideally, quantization is performed without reducing accuracy. Quantization-aware training (QAT) is performed by dynamically quantizing tensors (weights and activations) using optimal clipping scalars. 'Optimal' in that the mean squared error (MSE) of the quantized operation is minimized and the clipping scalars define the degree or amount of quantization for various tensors of the operation. Conventional techniques that quantize tensors during training suffer from high amounts of noise (error). Other techniques compute the clipping scalars offline through a brute force search to provide high accuracy. In contrast, the optimal clipping scalars can be computed online and provide the same accuracy as the clipping scalars computed offline.

Optimally clipped tensors and vectors

A distributed deep neural net (DNN) utilizing a distributed, tile-based architecture includes multiple chips, each with a central processing element, a global memory buffer, and a plurality of additional processing elements. Each additional processing element includes a weight buffer, an activation buffer, and vector multiply-accumulate units to combine, in parallel, the weight values and the activation values using stationary data flows.

Efficient neural network accelerator dataflows

To facilitate crosstalk analysis for an IC design, a plurality of input vectors are input into a gate-level simulation. In response, the gate-level simulation determines timing windows for all nets within the IC design, may perform aggressor pruning, and may then determine and output aggressor/victim pairs and associated features for the IC design. This gate-level simulation may be accelerated utilizing one or more graphics processor units (GPUs). Additionally, the aggressor/victim pairs and associated features for the IC design are then input into a trained machine learning environment, which outputs predicted delta delays for each of the aggressor/victim pairs. In this way, crosstalk analysis may be performed more accurately and efficiently.

Reducing crosstalk pessimism using GPU-accelerated gate simulation and machine learning

Neural networks, in many cases, include convolution layers that are configured to perform many convolution operations that require multiplication and addition operations. Compared with performing multiplication on integer, fixed-point, or floating-point format values, performing multiplication on logarithmic format values is straightforward and energy efficient as the exponents are simply added. However, performing addition on logarithmic format values is more complex. Conventionally, addition is performed by converting the logarithmic format values to integers, computing the sum, and then converting the sum back into the logarithmic format. Instead, logarithmic format values may be added by decomposing the exponents into separate quotient and remainder components, sorting the quotient components based on the remainder components, summing the sorted quotient components using an asynchronous accumulator to produce partial sums, and multiplying the partial sums by the remainder components to produce a sum. The sum may then be converted back into the logarithmic format.

Inference accelerator using logarithmic-based arithmetic

Neural networks, in many cases, include convolution layers that are configured to perform many convolution operations that require multiplication and addition operations. Compared with performing multiplication on integer, fixed-point, or floating-point format values, performing multiplication on logarithmic format values is straightforward and energy efficient as the exponents are simply added. However, performing addition on logarithmic format values is more complex. Conventionally, addition is performed by converting the logarithmic format values to integers, computing the sum, and then converting the sum back into the logarithmic format. Instead, logarithmic format values may be added by decomposing the exponents into separate quotient and remainder components, sorting the quotient components based on the remainder components, summing the sorted quotient components to produce partial sums, and multiplying the partial sums by the remainder components to produce a sum. The sum may then be converted back into the logarithmic format.

Neural network accelerator using logarithmic-based arithmetic

Asynchronous accumulator using logarithmic-based arithmetic

A distributed deep neural net (DNN) utilizing a distributed, tile-based architecture implemented on a semiconductor package. The package includes multiple chips, each with a central processing element, a global memory buffer, and processing elements. Each processing element includes a weight buffer, an activation buffer, and multiply-accumulate units to combine, in parallel, the weight values and the activation values.

Scalable multi-die deep learning system

IR drop predictions are obtained using a maximum convolutional neural network. A circuit structure is partitioned into a grid. For cells of the circuit structure in sub-intervals of a clock period, power consumption of the cell is amortized into a set of grid tiles that include portions of the cell, thus forming a set of power maps. The power maps are applied to a neural network to generate IR drop predictions for the circuit structure.

IR drop prediction with maximum convolutional neural network

Techniques to improve the accuracy and speed for detection and remediation of difficult to test nodes in a circuit design netlist. The techniques utilize improved netlist representations, test point insertion, and trained neural networks.

Deep learning testability analysis with graph convolutional networks

A neural network including an embedding layer to receive a gate function vector and an embedding width and alter a shape of the gate function vector by the embedding width, a concatenator to receive a gate feature input vector and concatenate the gate feature input vector with the gate function vector altered by the embedding width, a convolution layer to receive a window size, stride, and output feature size and generate an output convolution vector with a shape based on a length of the gate function vector, the window size of the convolution layer, and the output feature size of the convolution layer, and a fully connected layer to reduce the gate output convolution vector to a final path delay output.

Machine learning based post route path delay estimator from synthesis netlist

An integrated circuit device comprises pin resources, a memory controller circuit, a network interface controller circuit, and transmitter circuitry. The pin resources comprise pads coupled to off-chip pins of the integrated circuit device. The memory controller circuit comprises a first interface and the network interface controller circuit comprises a second interface. The transmitter circuitry is configurable to selectively couple either a first signal of the first interface or a second signal of the second interface to a first pad of the pin resources based on a pin distribution between the first interface and the second interface.

System and method for configuring a channel

Growing community of inventors

San Francisco, CA, United States of America

Brucek Kurdo Khailany