Memory organization and access for efficient matrix operations

Abstract

Certain aspects of the present disclosure provide techniques and apparatus for efficiently accessing memory in a computing system. An example method includes organizing a plurality of physical memory banks having a base size into a plurality of logical memory banks. A request to execute operations on the plurality of physical memory banks is received. The request to execute the operations comprises a request to interact with data having a sample width based on the base size. Responsive to receiving the request to execute the operations, the operations are executed on one or more logical memory banks of the plurality of logical memory banks via a memory crossbar shared across the plurality of logical memory banks. An amount of the data on which the operations are executed is a multiple of the sample width, and each logical memory bank has a size based on the base size and a multiplier value.