Sandisk Technologies Inc.

Western Digital Technologies, Inc.

Tung Thanh Hoang

Martin Lueker-Boden

Won Ho Choi

Wen Ma

Anand Kulkarni

Dejan Vucinic

Chao Sun

Daniel Bedau

Justin Kinney

A device includes arrays of Non-Volatile Memory (NVM) cells. Reference sequences representing portions of a genome are stored in respective groups of NVM cells. Exact matching phase substring sequences representing portions of at least one sample read are loaded into groups of NVM cells. One or more groups of NVM cells are identified where the stored reference sequence matches the loaded exact matching phase substring sequence using the arrays at Content Addressable Memories (CAMs). Approximate matching phase substring sequences are loaded into groups of NVM cells. One or more groups of NVM cells are identified where the stored reference sequence approximately matches the loaded approximate matching phase substring sequence using the arrays as Ternary CAMs (TCAMs). At least one of the reference sequence and the approximate matching phase substring sequence for each group of NVM cells includes at least one wildcard value when the arrays are used as TCAMs.

Devices and methods for genome sequencing

A non-volatile memory device is configured for in-memory computation of layers of a neural network by storing weight values as conductance values in memory cells formed of a series combination of a threshold switching selector, such as an ovonic threshold switch, and a programmable resistive element, such as a ReRAM element. By scaling the input voltages (representing inputs for the layer of the neural network) relative to the threshold values of the threshold switching selectors, dropout for inputs can be implemented to reduce overfitting by the neural network.

Dropout in neutral networks using threshold switching selectors in non-volatile memories

Certain aspects of the present disclosure provide techniques for performing compute in memory (CIM) computations. A device comprises a CIM module configured to apply analog weights to input data using multiply-accumulate operations to generate an output. The device further comprises a digital weight storage unit configured to store digital weight references, wherein a digital weight reference corresponds to an analog weight of the analog weights. The device also comprises a device controller configured to program the analog weights to the CIM module, cause the CIM module to process the input data, and reprogram one or more analog weights that are degraded. The digital weight references in the digital weight storage unit are populated with values from a host processing device. Degraded analog weights in the CIM module are reprogrammed based on the corresponding digital weight references from the digital weight storage unit without reference to the host processing device.

Systems and methods of compensating degradation in analog compute-in-memory (ACIM) modules

Anon-volatile memory structure capable of storing weights for layers of a deep neural network (DNN) and perform an inferencing operation within the structure is presented. An in-array multiplication can be performed between multi-bit valued inputs, or activations, for a layer of the DNN and multi-bit valued weights of the layer. Each bit of a weight value is stored in a binary valued memory cell of the memory array and each bit of the input is applied as a binary input to a word line of the array for the multiplication of the input with the weight. To perform a multiply and accumulate operation, the results of the multiplications are accumulated by adders connected to sense amplifiers along the bit lines of the array. The adders can be configured to multiple levels of precision, so that the same structure can accommodate weights and activations of 8-bit, 4-bit, and 2-bit precision.

Multi-precision digital compute-in-memory deep neural network engine for flexible and energy efficient inferencing

Certain aspects of the present disclosure provide techniques for performing compute in memory (CIM) computations. A device comprises a CIM module configured to apply a plurality of analog weights to data using multiply-accumulate operations to generate an output. The device further comprises a digital weight storage unit configured to store digital weight references, wherein a digital weight reference corresponds to an analog weight of the plurality of analog weights. The device also comprises a device controller configured to program the plurality of analog weights to the CIM module based on the digital weight references and determine degradation of one or more analog weights. The digital weight references in the digital weight storage unit are populated with values from a host device. Degraded analog weights in the CIM module are replaced with corresponding digital weight references from the digital weight storage unit without reference to the host device.

Systems and methods of determining degradation in analog compute-in-memory (ACIM) modules

Non-volatile memory structures for performing compute in memory inferencing for neural networks are presented. To improve performance, both in terms of speed and energy consumption, weight matrices are replaced with their singular value decomposition (SVD) and use of a low rank approximations (LRAs). The decomposition matrices can be stored in a single array, with the resultant LRA matrices requiring fewer weight values to be stored. The reduced sizes of the LRA matrices allow for inferencing to be performed more quickly and with less power. In a high performance and energy efficiency mode, a reduced rank for the SVD matrices stored on a memory die is determined and used to increase performance and reduce power needed for an inferencing operation.

Compute-in-memory deep neural network inference engine using low-rank approximation technique

Techniques are presented for performing in-memory matrix multiplication operations for binary input, binary weight valued convolution neural network (CNN) inferencing. The weights of a filter are stored in pairs of memory cells of a storage class memory device, such as a ReRAM or phase change memory based devices. To reduce current consumption, the binary valued filters are transformed into ternary valued filters by taking sums and differences of binary valued filter pairs. The zero valued weights of the transformed filters are stored as a pair of high resistance state memory cells, reducing current consumption during convolution. The results of the in-memory multiplications are pair-wise combined to compensate for the filter transformations. To compensate for zero valued weights, a zero weight register stores the number of zero weights along each bit line and is used to initialize counter values for accumulating the multiplication operations.

Kernel transformation techniques to reduce power consumption of binary input, binary weight in-memory convolutional neural network inference engine

Use of a NAND array architecture to realize a binary neural network (BNN) allows for matrix multiplication and accumulation to be performed within the memory array. A unit synapse for storing a weight of a BNN is stored in a pair of series connected memory cells. A binary input is applied on a pair of word lines connected to the unit synapse to perform the multiplication of the input with the weight. The results of such multiplications are determined by a sense amplifier, with the results accumulated by a counter. The arrangement extends to ternary inputs to realize a ternary-binary network (TBN) by adding a circuit to detect 0 input values and adjust the accumulated count accordingly. The arrangement further extends to a ternary-ternary network (TTN) by allowing 0 weight values in a unit synapse, maintaining the number of 0 weights in a register, and adjusting the count accordingly.

Realization of neural networks with ternary inputs and ternary weights in NAND memory arrays

Techniques are presented for accelerating in-memory matrix multiplication operations for a convolution neural network (CNN) inference in which the weights of a filter are stored in the memory of a storage class memory device, such as a ReRAM or phase change memory based device. To improve performance for inference operations when filters exhibit sparsity, a zero column index and a zero row index are introduced to account for columns and rows having all zero weight values. These indices can be saved in a register on the memory device and when performing a column/row oriented matrix multiplication, if the zero row/column index indicates that the column/row contains all zero weights, the access of the corresponding bit/word line is skipped as the result will be zero regardless of the input.

Accelerating sparse matrix multiplication in storage class memory-based convolutional neural network inference

A non-volatile memory device includes arrays of non-volatile memory cells that are configured to the store weights for a recurrent neural network (RNN) inference engine with a gated recurrent unit (GRU) cell. A set three non-volatile memory arrays, such as formed of storage class memory, store a corresponding three sets of weights and are used to perform compute-in-memory inferencing. The hidden state of a previous iteration and an external input are applied to the weights of the first and the of second of the arrays, with the output of the first array used to generate an input to the third array, which also receives the external input. The hidden state of the current generation is generated from the outputs of the second and third arrays.

Recurrent neural network inference engine with gated recurrent unit cell and non-volatile memory arrays

Non-volatile memory structures are presented for a content addressable memory (CAM) that can perform in-memory search operations for both ternary and binary valued key values. Each ternary or binary valued key bit is stored in a pair of memory cells along a bit line of a NAND memory array, with the stored keys searched by applying each ternary or binary valued bit of an input key as voltage levels on a pair of word lines. The system is highly scalable. The system can also be used to perform nearest neighbor searches between stored vectors and an input vector to find stored vectors withing a specified Hamming distance of the input vector.

Scalable search system design with single level cell NAND-based binary and ternary valued content addressable memory cells

A non-volatile memory structure capable of storing layers of a deep neural network (DNN) and perform an inferencing operation within the structure is presented. A stack of bonded die pairs is connected by through silicon vias. Each bonded die pair includes a memory die, having one or more memory arrays onto which layers of the neural network are mapped, and a peripheral circuitry die, including the control circuits for performing the convolution or multiplication for the bonded die pair. The multiplications can either be done in-array on the memory die or in-logic on the peripheral circuitry die. The arrays can be formed into columns along the vias, allowing an inferencing operation to be performed by propagating an input up and down the columns, with the output of one level being the input of the subsequent layer.

Vertical mapping and computing for deep neural networks in non-volatile memory

Use of a NAND array architecture to realize a binary neural network (BNN) allows for matrix multiplication and accumulation to be performed within the memory array. A unit synapse for storing a weight of a BNN is stored in a pair of series connected memory cells. A binary input is applied as a pattern of voltage values on a pair of word lines connected to the unit synapse to perform the multiplication of the input with the weight by determining whether or not the unit synapse conducts. The results of such multiplications are determined by a sense amplifier, with the results accumulated by a counter. The arrangement can be extended to ternary inputs to realize a ternary-binary network (TBN) by adding a circuit to detect 0 input values and adjust the accumulated count accordingly.

Realization of neural networks with ternary inputs and binary weights in NAND memory arrays

Systems and methods for dynamically assigning memory array die to CMOS die of a plurality of stacked die during memory operations are described. The plurality of stacked die may be vertically stacked and connected together via one or more vertical through-silicon via (TSV) connections. The memory array die may only comprise memory cell structures (e.g., vertical NAND strings) without column decoders, row decoders, charge pumps, sense amplifiers, control circuitry, page registers, or state machines. The CMOS die may contain support circuitry necessary for performing the memory operations, such as read and write memory operations. The one or more vertical TSV connections may allow each memory array die of the plurality of stacked die to communicate with or be electrically connected to one or more CMOS die of the plurality of stacked die.

Growing community of inventors

San Jose, CA, United States of America

Tung Thanh Hoang