Qualcomm Incorporated

Robert Dreyer

Colin Beaton Verrilli

Natarajan Vaidhyanathan

Mattheus Cornelis Antonius Adrianus Heddes

Koustav Bhattacharya

Michael Douglas Sharp

Sumit Sur

James Lyall Esliger

James Michael Artmeier

Joel Kevin Jones

Ivan Dimitrov Baev

Providing flexible matrix processors for performing neural network convolution in matrix-processor-based devices is disclosed. In this regard, a matrix-processor-based device provides a central processing unit (CPU) and a matrix processor. The matrix processor reorganizes a plurality of weight matrices and a plurality of input matrices into swizzled weight matrices and swizzled input matrices, respectively, that have regular dimensions natively supported by the matrix processor. The matrix-processor-based device then performs a convolution operation using the matrix processor to perform matrix multiplication/accumulation operations for the regular dimensions of the weight matrices and the input matrices, and further uses the CPU to execute instructions for handling the irregular dimensions of the weight matrices and the input matrices (e.g., by executing a series of nested loops, as a non-limiting example). The matrix-processor-based device thus provides efficient hardware acceleration by taking advantage of dimensional regularity, while maintaining the flexibility to handle different variations of convolution.

Providing flexible matrix processors for performing neural network convolution in matrix-processor-based devices

Providing efficient floating-point operations using matrix processors in processor-based systems is disclosed. In this regard, a matrix-processor-based device provides a matrix processor comprising a positive partial sum accumulator and a negative partial sum accumulator. As the matrix processor processes pairs of floating-point operands, the matrix processor calculates an intermediate product based on a first floating-point operand and a second floating-point operand and determines a sign of the intermediate product. Based on the sign, the matrix processor normalizes the intermediate product with a partial sum fraction of the positive partial sum accumulator or the negative partial sum accumulator, then adds the intermediate product to the positive sum accumulator or the negative sum accumulator. After processing all pairs of floating-point operands, the matrix processor subtracts the negative partial sum accumulator from the positive partial sum accumulator to generate a final sum, then renormalizes the final sum a single time.

Providing efficient floating-point operations using matrix processors in processor-based systems

Providing efficient multiplication of sparse matrices in matrix-processor-based devices is disclosed herein. In one aspect, a matrix processor of a matrix-processor-based device includes a plurality of sequencers coupled to a plurality of multiply/accumulate (MAC) units for performing multiplication and accumulation operations. Each sequencer determines whether a product of an element of a first input matrix to be multiplied with an element of a second input matrix has a value of zero (e.g., by determining whether the element of the first input matrix has a value of zero, or by determining whether either the element of the first input matrix or that of the second input matrix has a value of zero). If the product of the elements of the first input matrix and the second input matrix does not have a value of zero, the sequencer provides the elements to a MAC unit to perform a multiplication and accumulation operation.

Providing efficient multiplication of sparse matrices in matrix-processor-based devices

Systems and methods for dynamic granularity control of parallelized work in a heterogeneous multi-processor portable computing device (PCD) are provided. During operation a first parallelized portion of an application executing on the PCD is identified. The first parallelized portion comprising a plurality of threads for parallel execution on the PCD. Performance information is obtained about a plurality of processors of the PCD, each of the plurality of processors corresponding to one of the plurality of threads. A number M of workload partition granularities for the plurality of threads is determined, and a total execution cost for each of the M workload partition granularities is determined. An optimal granularity comprising a one of the M workload partition granularities with a lowest total execution cost is determined, and the first parallelized portion is partitioned into a plurality of workloads having the optimal granularity.

System and method for dynamic granularity control of parallelized work in a portable computing device (PCD)

Generating parallelized executable code from input code includes statically analyzing the input code to determine aspects of data flow and control flow of the input code; dynamically analyzing the input code to determine additional aspects of data flow and control flow of the input code; generating an intermediate representation of the input code based at least in part on the aspects of data flow and control flow of the input code identified by the static analysis and the additional aspects of data and control flow of the input code identified by the dynamic analysis; and processing the intermediate representation to determine portions of the intermediate representation that are eligible for parallel execution; and generating parallelized executable code from the processed intermediate representation.

Growing community of inventors

Menlo Park, CA, United States of America

Robert Dreyer