Samsung Electronics Co., Ltd.

Via Technologies, Inc.

Arm Limited

Advanced Micro Devices Corporation

Ati Technologies Ulc

John Wakefield Brothers

Abhinav Golas

Timour T Paltashev

Joohoon Lee

Boris Prokopenko

Karthik Ramani

Hsilin Huang

Konstantine I Iourcha

Soojung Ryu

Santosh Abraham

Jeongjoon Yoo

Zhengping Ji

Zahid Hussain

Seokyoon Jung

Erik Persson

Rune Holm

Yang (Jeff) Jiao

Graeme Leslie Ingram

Liang Li

Ilia Ovsiannikov

Greg Sadowski

Dominic Hugo Symes

Daren Croxford

David I J Glen

Sundeep Krishnadasan

Jens Olson

Jim Xu

Yuanfeng Wang

Christopher Thomas Cheng

Jared Smolens

Wen-Chung Chen

Yijung Su

Huy Duc Bui

Jiangming Xu

Liangjun Zhang

Ki Fung Chow

Eunsoo Shim

Warren Fritz Kruger

Ian Rudolf Bratt

Wonjong Lee

Michael L Schmit

Balaji Venu

Sharjeel Saeed

Boris Beylin

Alexander Eugene Chalfin

Jaedon Lee

Kwontaek Kwon

Radhakrishna Giduthuri

Stephen David Presant

Metin Gokhan Ünal

Danny Loh

Mike Hong

Maxim Lukyanov

Lingjie Xu

Fred Liao

Seonggun Kim

Eric Kunze

David Ij Glen

Lingchuan Meng

Yi-Jung Su

Qunfeng (Fred) Liao

Krishnadasan Sundeep

Peter Mattias Hansson

Sibyl Shao

Alex Grosul

Chi-wen Cheng

Qunfeng Liao

Elliott Maurice Simon Rosemarine

Kartikeya Bhardwaj

Santosh George Abraham

Christopher Cheng

Woody Chow

A processor to execute a plurality of tasks comprising a first task and a second task. At least a part of the first task is to be executed simultaneously with at least a part of the second task. The processor comprises a handling unit to: determine an available portion of a storage available during execution of the part of the first task; determine a mapping between at least one logical address associated with data associated with the part of the second task and a corresponding at least one physical address of the storage corresponding to the available portion; and identify, based on the mapping, the at least one physical address corresponding to the at least one logical address associated with the data, for storing the data in the available portion of the storage.

Data storage

A method and processing unit for mapping coordinates of a tensor to a shared storage of the processing unit. The processing unit comprises processing slices, each for performing a suboperation of the operation, and having prioritized access to a preferred portion of the shared storage. The method comprises obtaining a layout indicating configurable data regions in the shared storage. The configurable data regions are banks comprising regions of shared storage. A stride based on for at least one dimension of the tensor and the layout is obtained. Coordinates of the tensor are mapped to locations in the shared storage, wherein the mapping comprises calculating a bank number and offset which are calculated based on coordinates of the tensor, the layout of the configurable data regions, and the stride of the at least one dimension. Each processing slice performs the suboperation on the data of the tensor based on the mapping.

Efficient data processing

A processor, method, and non-transitory computer-readable storage medium for performing a block matching between first data and second data are provided. The block matching is performed using an iterative process, wherein, for each iteration, a portion of the first data and a corresponding portion of the second data is selected using a sliding window approach. When differences data, used for block matching, is calculated for a specific subset of first and second data, many of the calculations overlap with those needed for a nearby subset. Summed area table, SAT, data used for determining the difference data is continuously stored and updated in a buffer, such that overlapping computations can be avoided.

Block matching between first data and second data

A processor to generate accumulated data comprising, for an operation cycle: performing an operation on a first bit range of a set of first input data to generate a set of operation data, which is accumulated with stored data within a first storage device. A lowest n bits of the accumulated data are accumulated with first further stored data within a first bit range of a second storage device, and are bit-shifted from the first storage device. Further accumulated data is generated, comprising, for an operation cycle: performing the operation on a second bit range of the set of first input data to generate a further set of operation data, which is accumulated with the stored data within the first storage device. A lowest m bits of the further accumulated data is accumulated with second further stored data within a second bit range of the second storage device.

Data accumulation

A processing unit, method, and medium for decompressing or generating textures within a graphics processing unit (GPU). The textures are compressed with a variable-rate compression scheme such as JPEG. The compressed textures are retrieved from system memory and transferred to local cache memory on the GPU without first being decompressed. A table is utilized by the cache to locate individual blocks within the compressed texture. A decompressing shader processor receives compressed blocks and then performs on-the-fly decompression of the blocks. The decompressed blocks are then processed as usual by a texture consuming shader processor of the GPU.

Real time on-chip texture decompression using shader processors

A method of processing input data for a given layer of a neural network using a data processing system comprising compute resources for performing convolutional computations is described. The input data comprises a given set of input feature maps, IFMs, and a given set of filters. The method comprises generating a set of part-IFMs including pluralities of part-IFMs which correspond to respective IFMs, of the given set of IFMs. The method further includes grouping part-IFMs in the set of part-IFMs into a set of selections of part-IFMs. The method further includes convolving, by respective compute resources of the data processing system, the set of selections with the given set of filters to compute a set of part-output feature maps. A data processing system for processing input data for a given layer of a neural network is also described.

Processing data for a layer of a neural network

Systems and methods for processing data for a neural network are described. The system comprises non-transitory memory configured to receive data bits defining a kernel of weights, the data bits being suitable for processing input data; and a data processing unit, configured to: receive bits defining a kernel of weights for the neural network, the kernel of weights comprising one or more non-zero value weights and one or more zero-valued weights; generate a set of mask bits, a position of each bit in the set of mask bits corresponds to a position within the kernel of weights and the value of each bit indicates whether a weight in the corresponding position is a zero-valued weight or a non-zero value weight; and transmit the non-zero value weights and the set of mask bits for storage, the non-zero value weights and the set of mask bits represent the kernel of weights.

Weight processing for a neural network

The present disclosure relates generally to multi-processor arrangements and, more particularly, to a point of serialization for broadcast communications within multi-processor arrangements.

Multi-processor arrangement with point of serialization for broadcast communications

An embodiment includes a method, comprising: pruning a layer of a neural network having multiple layers using a threshold; and repeating the pruning of the layer of the neural network using a different threshold until a pruning error of the pruned layer reaches a pruning error allowance.

Automatic thresholds for neural network pruning and retraining

The present disclosure relates generally to multi-processor arrangements and, more particularly, to broadcast regions for multi-processor arrangements.

Broadcast regions for multi-processor arrangement

A processor arranged to compress neural network activation data comprising an input module for obtaining neural network activation data. The processor also comprises a block creation module arranged to split the neural network activation data into a plurality of blocks; and a metadata generation module for generating metadata associated with at least one of the plurality of blocks. Based on the metadata generated a selection module selects a compression scheme for each of the plurality of blocks, and a compression module for applying the selected compression scheme to the corresponding block to produce compressed neural network activation data. An output module is also provided for outputting the compressed neural network activation data.

Compression of neural network activation data

The present disclosure relates generally to multi-processor arrangements and, more particularly, to broadcast hubs for multi-processor arrangements. A processing tile may comprise a broadcast hub to obtain a plurality of parameters applicable in a particular operation from at least one of a plurality of processing tiles and initiate distribution of the plurality of parameters to the plurality of processing tiles, wherein the plurality of processing tiles may execute the particular operation based at least in part on the plurality of distributed parameters.

Broadcast hub for multi-processor arrangement

To perform neural network processing to modify an input data array to generate a corresponding output data array using a filter comprising an array of weight data, at least one of the input data array and the filter are subdivided into a plurality of portions, a plurality of neural network processing passes using the portions are performed, and the output generated by each processing pass is combined to provide the output data array.

Neural network processing

A system and method to reduce weight storage bits for a deep-learning network includes a quantizing module and a cluster-number reduction module. The quantizing module quantizes neural weights of each quantization layer of the deep-learning network. The cluster-number reduction module reduces the predetermined number of clusters for a layer having a clustering error that is a minimum of the clustering errors of the plurality of quantization layers. The quantizing module requantizes the layer based on the reduced predetermined number of clusters for the layer and the cluster-number reduction module further determines another layer having a clustering error that is a minimum of the clustering errors of the plurality of quantized layers and reduces the predetermined number of clusters for the another layer until a recognition performance of the deep-learning network has been reduced by a predetermined threshold.

Method and algorithm of recursive deep learning quantization for weight bit reduction

When performing a matrix-vector multiply operation for neural network processing, a set of one or more input vectors to be multiplied by a matrix of data values is scanned to identify data positions of the input vector(s) for which the data value is non-zero in at least one of the input vectors. For each of the data positions identified as having a non-zero value in at least one of the input vectors, the set of data values from the matrix of data values for that data position is fetched from memory and the matrix-vector multiply operation is performed using the data values for the input vectors for the data positions identified as being non-zero and the fetched set(s) of data values from the matrix of data values for those data position(s).

Performing matrix-vector multiply operations for neural networks on electronic devices

Growing community of inventors

Calistoga, CA, United States of America

John Wakefield Brothers