Data optimization for high bandwidth (hbw) nvm ai inference system

Abstract

A method for storing weight data used to compute node values during inferencing operations conducted by a neural network comprises receiving a neural network definition. The neural network definition defines a neural network having a plurality of layers, each having a plurality of nodes. A set of weights used to compute a neural network inferencing result for each neural network node of the plurality of network nodes in the layer is determined. The set of weights determined for the layer in a page of memory can be stored in a high bandwidth non-volatile memory (NVM), such that any weights used to compute the neural network inference result for each neural network node of the plurality of nodes in the layer are stored together in the page of memory for retrieval together. Weights can be stored in different arrays across multiple memory channels.