Advanced Micro Devices Corporation

Daniel I Lowell

Manish Satyapal Gupta

Chao Liu

Wen Heng Chung

Jing Zhang

David Andrew Roberts

Vilas K Sridharan

Steven E Raasch

Mitesh Ramesh Meswani

Mayank Daga

Shi Dong

Sergey Voronov

Jehandad Khan

A technique for manipulating a generic tensor is provided. The technique includes receiving a first request to perform a first operation on a generic tensor descriptor associated with the generic tensor, responsive to the first request, performing the first operation on the generic tensor descriptor, receiving a second request to perform a second operation on generic tensor raw data associated with the generic tensor, and responsive to the second request, performing the second operation on the generic tensor raw data, the performing the second operation including mapping a tensor coordinate specified by the second request to a memory address, the mapping including evaluating a delta function to determine an address delta value to add to a previously determined address for a previously processed tensor coordinate.

Composable neural network kernels

A technique for manipulating a generic tensor is provided. The technique includes receiving a first request to perform a first operation on a generic tensor descriptor associated with the generic tensor, responsive to the first request, performing the first operation on the generic tensor descriptor, receiving a second request to perform a second operation on generic tensor raw data associated with the generic tensor, and responsive to the second request, performing the second operation on the generic tensor raw data.

A processing device, which improves processing performance, is provided which comprises memory configured to store data and a processor, in communication with the memory. The processor is configured to receive tuning parameters, each having a numeric value, for executing a portion of a program on an identified hardware device and convert the numeric values of the tuning parameters to words. The processor is also configured to predict, using one or more machine language learning algorithms, which combination of the words to execute the portion of the program on the identified hardware device based on performance efficiency and convert the predicted combination of the words to corresponding numeric values for executing the portion of the program on the identified hardware device.

Method and apparatus for predicting kernel tuning parameters

Methods, devices, systems, and instructions for adaptive quantization in an artificial neural network (ANN) calculate a distribution of ANN information; select a quantization function from a set of quantization functions based on the distribution; apply the quantization function to the ANN information to generate quantized ANN information; load the quantized ANN information into the ANN; and generate an output based on the quantized ANN information. Some examples recalculate the distribution of ANN information and reselect the quantization function from the set of quantization functions based on the resampled distribution if the output does not sufficiently correlate with a known correct output. In some examples, the ANN information includes a set of training data. In some examples, the ANN information includes a plurality of link weights.

Adaptive quantization for neural networks

An electronic device that includes a processor configured to execute training iterations during a training process for a neural network, each training iteration including processing a separate instance of training data through the neural network, and a sparsity monitor is described. During operation, the sparsity monitor acquires, during a monitoring interval in each of one or more monitoring periods, intermediate data output by at least some intermediate nodes of the neural network during training iterations that occur during each monitoring interval. The sparsity monitor then generates, based at least in part on the intermediate data, one or more values representing sparsity characteristics for the intermediate data. The sparsity monitor next sends, to the processor, the one or more values representing the sparsity characteristics and the processor controls one or more aspects of executing subsequent training iterations based at least in part on the values representing the sparsity characteristics.

Data sparsity monitoring during neural network training

Techniques for selecting one of a plurality of heterogeneous memory units for placement of blocks of data (e.g., memory pages), based on both reliability and performance, are disclosed. A 'cost' for each data block/memory unit combination is determined, based on the frequency of access of the data block, the latency of the memory unit, and, optionally, an architectural vulnerability factor (which represents the level of exposure of a particular memory data value to memory faults such as bit flips). A memory unit is selected for the data block for which the determined cost is the lowest, out of all memory units considered, and the data block is placed into that memory unit.

Performance-aware and reliability-aware data placement for n-level heterogeneous memory systems

Systems, apparatuses, and methods for implementing bufferless communication for redundant multithreading applications using register permutation are disclosed. In one embodiment, a system includes a parallel processing unit, a register file, and a scheduler. The scheduler is configured to cause execution of a plurality of threads to be performed in lockstep on the parallel processing unit. The plurality of threads include a first thread and a second thread executing on adjacent first and second lanes, respectively, of the parallel processing unit. The second thread is configured to perform a register permute operation from a first register location to a second register location in a first instruction cycle, with the second register location associated with the second processing lane. The second thread is configured to read from the second register location in a second instruction cycle, wherein the first and second instruction cycles are successive instruction cycles.

Bufferless communication for redundant multithreading using register permutation

Techniques for performing redundant multi-threading ('RMT') include the use of an RMT compare instruction by two program instances ('work-items'). The RMT compare instruction specifies a value from each work-item to be compared. Upon executing the RMT compare instructions, the work-items transmit the values to a hardware comparator unit. The hardware comparator unit compares the received values and performs an error action if the values do not match. The error action may include sending an error code in a return value back to the work-items that requested the comparison or emitting a trap signal. Optionally, the work-items also send addresses for comparison to the comparator unit. If the addresses and values match, then the comparator stores the value at the specified address. If either or both of the values or the addresses do not match, then the comparator performs an error action.

Paired value comparison for redundant multi-threading operations

A first processing element is configured to execute a first thread and one or more second processing elements are configured to execute one or more second threads that are redundant to the first thread. The first thread and the one or more second threads are to selectively bypass one or more comparisons of results of operations performed by the first thread and the one or more second threads depending on whether an event trigger for the comparison has occurred a configurable number of times since a previous comparison of previously encoded values of the results. In some cases the comparison can be performed based on hashed (or encoded) values of the results of a current operation and one or more previous operations.

Growing community of inventors

Austin, TX, United States of America

Daniel I Lowell