Nvidia Corporation

Jeffrey Michael Pool

Ming Yiu Siu

Stuart F Oberman

Andrew Robert Kerr

John Tran

Jorge Albericio Latorre

William James Dally

Liang Chen

Jack H Choquette

Jeffrey A Bolz

Ganesh Venkatesh

Timothy Tsai

Michael Alex Frumkin

Ronny Krashinsky

Xiaojun Wang

Manan Patel

Michael Brendan Sullivan

Franciscus W Sijstermans

Siva Kumar Sastry Hari

Paulius Micikevicius

Chong Yu

Steven William Keckler

Yangxiang Huang

Zhou Yan

Yuanzhi Hua

Apparatuses, systems, and techniques to transform data sets, such as matrices representing layers of neural networks, to increase sparsity and/or other characteristics of said data sets to improve performance in computations, such as neural network computations. In at least one embodiment, one or more subsets of data in one or more sets of data are rearranged as part of a process to increase sparsity in said one or more sets of data to satisfy one or more one or more structural sparsity constraints.

Increasing sparcity in data sets

A method, computer readable medium, and processor are described herein for inline data inspection by using a decoder to decode a load instruction, including a signal to cause a circuit in a processor to indicate whether data loaded by a load instruction exceeds a threshold value. Moreover, an indication of whether data loaded by a load instruction exceeds a threshold value may be stored.

Inline data inspection for workload simplification

A packed error correction code (ECC) technique opportunistically embeds ECC check-bits with compressed data. When compressed, the data is encoded in fewer bits and is therefore fragmented when stored or transmitted compared with the uncompressed data. The ECC check-bits may be packed with compressed data at 'source' points. The check-bits are transmitted along with the compressed data and, at any 'intermediate' point between the source and a &#x201c;destination&#x201d; the check-bits may be used to detect and correct errors in the compressed data. In contrast with conventional systems, packed ECC enables end-to-end coverage for sufficiently-compressed data within the processor and also externally. While storage circuitry typically is protected by structure-specific ECC, protection is also beneficial for data as it is transmitted between processing and/or storage units.

Packed error correction code (ECC) for compressed data protection

Compressed data is oftentimes beneficial for reducing the computing resources required, for example, to transmit and store data. The compression of data is particularly useful when dealing with sparse data (data that includes numerous zeros or near-zero values) and only non-zero values above a certain threshold have significance. When dealing with compressed data, oftentimes the data needs to be decompressed for processing (e.g., by deep learning networks or other applications configured to operate on sparse, or other uncompressed data). Instructions are disclosed for supporting the decompression of compressed data by a processing unit such as a CPU and GPU.

Decompression techniques for processing compressed data suitable for artificial neural networks

Many computing systems process data organized in a matrix format. For example, artificial neural networks perform numerous computations on data organized into matrices using conventional matrix arithmetic operations. One such operation is the transpose operation. Techniques are introduced for storing a matrix in a compressed format that allows, for example, a transpose operation to be performed during decompression. Thus, by utilizing the introduced techniques, transformations of compressed matrices such transposition can be achieved in a more effective way. Parallel processing may also be used to more efficiently compress and/or decompress.

Efficient matrix format suitable for neural networks

A method, computer readable medium, and system are disclosed for inline data inspection. The method includes the steps of receiving, by a load/store unit, a load instruction and obtaining, by an inspection circuit that is coupled to the load/store unit, data specified by the load instruction. Additional steps include determining that the data equals zero and transmitting the data and a predicate signal to the load/store unit, wherein the predicate signal indicates that the data equals zero. Alternative additional steps include computing a predicate value based on a comparison between the data and a threshold value and transmitting the data and the predicate value to the load/store unit, wherein the predicate value is asserted when the data is less than the threshold value and is negated when the data is not less than the threshold value.

A subsystem configured to encode an RGBA8 data stream assembles sequences of four-byte groups from the data stream. The subsystem decorrelates the red and blue channels, and computes a difference between each four-byte group and an anchor value. The anchor is encoded at full value. The subsystem then assigns each group a five-bit header based on the number and location of non-zero bytes and on the data content of the non-zero bytes within the group. The subsystem favors zero valued bytes. Thus, when a group includes only zero valued bytes, the header is sufficient to encode the group; no data bits are necessary. Further, two successive groups of zero-valued bytes may be encoded as a single header with no data bits, achieving further data reduction. Finally, the subsystem concatenates all the headers with associated data to yield the source data stream compressed to some ratio, e.g. four-to-one.

Technique for performing variable width data compression using a palette of encodings

A method, computer program product, and system for sparse convolutional neural networks that improves efficiency is described. Multi-bit data for input to a processing element is received at a compaction engine. The multi-bit data is determined to equal zero and a single bit signal is transmitted from the memory interface to the processing element in lieu of the multi-bit data, where the single bit signal indicates that the multi-bit data equals zero. A compacted data sequence for input to a processing element is received by a memory interface. The compacted data sequence is transmitted from the memory interface to an expansion engine. Non-zero values are extracted from the compacted data sequence and zeros are inserted between the non-zero values by the expansion engine to generate an expanded data sequence that is output to the processing element.

Data compaction and memory bandwidth reduction for sparse neural networks

Growing community of inventors

Chapel Hill, NC, United States of America

Jeffrey Michael Pool