Qualcomm Incorporated

Koustav Bhattacharya

Colin Beaton Verrilli

Natarajan Vaidhyanathan

Mattheus Cornelis Antonius Adrianus Heddes

Robert Dreyer

Carl Alan Waldspurger

Providing flexible matrix processors for performing neural network convolution in matrix-processor-based devices is disclosed. In this regard, a matrix-processor-based device provides a central processing unit (CPU) and a matrix processor. The matrix processor reorganizes a plurality of weight matrices and a plurality of input matrices into swizzled weight matrices and swizzled input matrices, respectively, that have regular dimensions natively supported by the matrix processor. The matrix-processor-based device then performs a convolution operation using the matrix processor to perform matrix multiplication/accumulation operations for the regular dimensions of the weight matrices and the input matrices, and further uses the CPU to execute instructions for handling the irregular dimensions of the weight matrices and the input matrices (e.g., by executing a series of nested loops, as a non-limiting example). The matrix-processor-based device thus provides efficient hardware acceleration by taking advantage of dimensional regularity, while maintaining the flexibility to handle different variations of convolution.

Providing flexible matrix processors for performing neural network convolution in matrix-processor-based devices

Providing efficient floating-point operations using matrix processors in processor-based systems is disclosed. In this regard, a matrix-processor-based device provides a matrix processor comprising a positive partial sum accumulator and a negative partial sum accumulator. As the matrix processor processes pairs of floating-point operands, the matrix processor calculates an intermediate product based on a first floating-point operand and a second floating-point operand and determines a sign of the intermediate product. Based on the sign, the matrix processor normalizes the intermediate product with a partial sum fraction of the positive partial sum accumulator or the negative partial sum accumulator, then adds the intermediate product to the positive sum accumulator or the negative sum accumulator. After processing all pairs of floating-point operands, the matrix processor subtracts the negative partial sum accumulator from the positive partial sum accumulator to generate a final sum, then renormalizes the final sum a single time.

Providing efficient floating-point operations using matrix processors in processor-based systems

Providing efficient multiplication of sparse matrices in matrix-processor-based devices is disclosed herein. In one aspect, a matrix processor of a matrix-processor-based device includes a plurality of sequencers coupled to a plurality of multiply/accumulate (MAC) units for performing multiplication and accumulation operations. Each sequencer determines whether a product of an element of a first input matrix to be multiplied with an element of a second input matrix has a value of zero (e.g., by determining whether the element of the first input matrix has a value of zero, or by determining whether either the element of the first input matrix or that of the second input matrix has a value of zero). If the product of the elements of the first input matrix and the second input matrix does not have a value of zero, the sequencer provides the elements to a MAC unit to perform a multiplication and accumulation operation.

Providing efficient multiplication of sparse matrices in matrix-processor-based devices

Providing flexible management of heterogeneous memory systems using spatial Quality of Service (QoS) tagging in processor-based systems is disclosed. In one aspect, a heterogeneous memory system of a processor-based system includes a first memory and a second memory. The heterogeneous memory system is divided into a plurality of memory regions, each associated with a QoS identifier (QoSID), which may be set and updated by software. A memory controller of the heterogeneous memory system provides a QoS policy table, which operates to associate each QoSID with a QoS policy state, and which also may be software-configurable. Upon receiving a memory access request including a memory address of a memory region, the memory controller identifies a software-configurable QoSID associated with the memory address, and associates the QoSID with a QoS policy state using the QoS policy table. The memory controller then applies the QoS policy state to perform the memory access operation.

Growing community of inventors

Austin, TX, United States of America

Koustav Bhattacharya