Sandisk Technologies Inc.

Western Digital Technologies, Inc.

Applied Materials, Inc.

Wen Ma

Martin Lueker-Boden

Won Ho Choi

Pi-Feng Chiu

Minghai Qin

Tung Thanh Hoang

Gerrit Jan Hemink

Daniel Bedau

Lin Zhang

Zhuang Li

Justin Kinney

Martin Lueker-boden

Exemplary methods and apparatus are disclosed that implement super-sparse image/video compression by storing image dictionary elements within a cross-bar resistive random access memory (ReRAM) array (or other suitable cross-bar NVM array). In illustrative examples, each column of the cross-bar ReRAM array stores the values for one dictionary element (such as one 4×4 dictionary element). Methods and apparatus are described for training (configuring) the cross-bar ReRAM array to generate and store the dictionary elements by sequentially applying patches from training images to the array using an unstructured Hebbian training procedure. Additionally, methods and apparatus are described for compressing an input image by applying patches from the input image to the ReRAM array to read out cross-bar column indices identifying the columns storing the various dictionary elements that best fit the image. This may be done in parallel using a set of ReRAM arrays.

Non-volatile cross-bar memory with on-chip processing circuitry

A device includes arrays of Non-Volatile Memory (NVM) cells. Reference sequences representing portions of a genome are stored in respective groups of NVM cells. Exact matching phase substring sequences representing portions of at least one sample read are loaded into groups of NVM cells. One or more groups of NVM cells are identified where the stored reference sequence matches the loaded exact matching phase substring sequence using the arrays at Content Addressable Memories (CAMs). Approximate matching phase substring sequences are loaded into groups of NVM cells. One or more groups of NVM cells are identified where the stored reference sequence approximately matches the loaded approximate matching phase substring sequence using the arrays as Ternary CAMs (TCAMs). At least one of the reference sequence and the approximate matching phase substring sequence for each group of NVM cells includes at least one wildcard value when the arrays are used as TCAMs.

Devices and methods for genome sequencing

A non-volatile memory device is configured for in-memory computation of layers of a neural network by storing weight values as conductance values in memory cells formed of a series combination of a threshold switching selector, such as an ovonic threshold switch, and a programmable resistive element, such as a ReRAM element. By scaling the input voltages (representing inputs for the layer of the neural network) relative to the threshold values of the threshold switching selectors, dropout for inputs can be implemented to reduce overfitting by the neural network.

Dropout in neutral networks using threshold switching selectors in non-volatile memories

An innovative low-bit-width device may include a first digital-to-analog converter (DAC), a second DAC, a plurality of non-volatile memory (NVM) weight arrays, one or more analog-to-digital converters (ADCs), and a neural circuit. The first DAC is configured to convert a digital input signal into an analog input signal. The second DAC is configured to convert a digital previous hidden state (PHS) signal into an analog PHS signal. NVM weight arrays are configured to compute vector matrix multiplication (VMM) arrays based on the analog input signal and the analog PHS signal. The NVM weight arrays are coupled to the first DAC and the second DAC. The one or more ADCs are coupled to the plurality of NVM weight arrays and are configured to convert the VMM arrays into digital VMM values. The neural circuit is configured to process the digital VMM values into a new hidden state.

Hardware accelerated discretized neural network

A non-volatile memory device includes arrays of non-volatile memory cells that are configured to the store weights for a recurrent neural network (RNN) inference engine with a gated recurrent unit (GRU) cell. A set three non-volatile memory arrays, such as formed of storage class memory, store a corresponding three sets of weights and are used to perform compute-in-memory inferencing. The hidden state of a previous iteration and an external input are applied to the weights of the first and the of second of the arrays, with the output of the first array used to generate an input to the third array, which also receives the external input. The hidden state of the current generation is generated from the outputs of the second and third arrays.

Recurrent neural network inference engine with gated recurrent unit cell and non-volatile memory arrays

Systems and methods for reducing the impact of defects within a crossbar memory array when performing multiplication operations in which multiple control lines are concurrently selected are described. A group of memory cells within the crossbar memory array may be controlled by a local word line that is controlled by a local word line gating unit that may be configured to prevent the local word line from being biased to a selected word line voltage during an operation; the local word line may instead be set to a disabling voltage during the operation such that the memory cell currents through the group of memory cells are eliminated. If a defect has caused a short within one of the memory cells of the group of memory cells, then the local word line gating unit may be programmed to hold the local word line at the disabling voltage during multiplication operations.

Methods to tolerate programming and retention errors of crossbar memory arrays

Technology for reconfigurable input precision in-memory computing is disclosed herein. Reconfigurable input precision allows the bit resolution of input data to be changed to meet the requirements of in-memory computing operations. Voltage sources (that may include DACs) provide voltages that represent input data to memory cell nodes. The resolution of the voltage sources may be reconfigured to change the precision of the input data. In one parallel mode, the number of DACs in a DAC node is used to configure the resolution. In one serial mode, the number of cycles over which a DAC provides voltages is used to configure the resolution. The memory system may include relatively low resolution voltage sources, which avoids the need to have complex high resolution voltage sources (e.g., high resolution DACs). Lower resolution voltage sources can take up less area and/or use less power than higher resolution voltage sources.

Reconfigurable input precision in-memory computing

Enhanced techniques and circuitry are presented herein for artificial neural networks. These artificial neural networks are formed from artificial synapses, which in the implementations herein comprise a memory arrays having non-volatile memory elements. In one implementation, an apparatus comprises a plurality of non-volatile memory arrays configured to store weight values for an artificial neural network. Each of the plurality of non-volatile memory arrays can be configured to receive data from a unified buffer shared among the plurality of non-volatile memory arrays, operate on the data, and shift at least portions of the data to another of the plurality of non-volatile memory arrays.

Shifting architecture for data reuse in a neural network

Non-volatile memory structures are presented for a content addressable memory (CAM) that can perform in-memory search operations for both ternary and binary valued key values. Each ternary or binary valued key bit is stored in a pair of memory cells along a bit line of a NAND memory array, with the stored keys searched by applying each ternary or binary valued bit of an input key as voltage levels on a pair of word lines. The system is highly scalable. The system can also be used to perform nearest neighbor searches between stored vectors and an input vector to find stored vectors withing a specified Hamming distance of the input vector.

Scalable search system design with single level cell NAND-based binary and ternary valued content addressable memory cells

Use of a NAND array architecture to realize a binary neural network (BNN) allows for matrix multiplication and accumulation to be performed within the memory array. A unit synapse for storing a weight of a BNN is stored in a pair of series connected memory cells. A binary input is applied as a pattern of voltage values on a pair of word lines connected to the unit synapse to perform the multiplication of the input with the weight by determining whether or not the unit synapse conducts. The results of such multiplications are determined by a sense amplifier, with the results accumulated by a counter.

Realization of binary neural networks in NAND memory arrays

Exemplary methods and apparatus are disclosed that implement super-sparse image/video compression by storing image dictionary elements within a cross-bar resistive random access memory (ReRAM) array (or other suitable cross-bar NVM array). In illustrative examples, each column of the cross-bar ReRAM array stores the values for one dictionary element (such as one 4&#xd7;4 dictionary element). Methods and apparatus are described for training (configuring) the cross-bar ReRAM array to generate and store the dictionary elements by sequentially applying patches from training images to the array using an unstructured Hebbian training procedure. Additionally, methods and apparatus are described for compressing an input image by applying patches from the input image to the ReRAM array to read out cross-bar column indices identifying the columns storing the various dictionary elements that best fit the image. This may be done in parallel using a set of ReRAM arrays.

Super-sparse image compression using cross-bar non-volatile memory device

Use of a non-volatile memory array architecture to realize a neural network (BNN) allows for matrix multiplication and accumulation to be performed within the memory array. A unit synapse for storing a weight of a neural network is formed by a differential memory cell of two individual memory cells, such as a memory cells having a programmable resistance, each connected between a corresponding one of a word line pair and a shared bit line. An input is applied as a pattern of voltage values on word line pairs connected to the unit synapses to perform the multiplication of the input with the weight by determining a voltage level on the shared bit line. The results of such multiplications are determined by a sense amplifier, with the results accumulated by a summation circuit. The approach can be extended from binary weights to multi-bit weight values by use of multiple differential memory cells for a weight.

Configurable precision neural network with differential binary non-volatile memory cell structure

Use of a non-volatile memory array architecture to realize a neural network (BNN) allows for matrix multiplication and accumulation to be performed within the memory array. A unit synapse for storing a weight of a neural network is formed by a differential memory cell of two individual memory cells, such as a memory cells having a programmable resistance, each connected between a corresponding one of a word line pair and a shared bit line. An input is applied as a pattern of voltage values on word line pairs connected to the unit synapses to perform the multiplication of the input with the weight by determining a voltage level on the shared bit line. The results of such multiplications are determined by a sense amplifier, with the results accumulated by a summation circuit.

Differential non-volatile memory cell for artificial neural network

Gap-fill and damascene methods are disclosed for depositing an insulating thin film of nitrofluorinated silicate glass on a substrate in a process chamber. A high-density plasma, generated from a gaseous mixture of silicon-, fluorine-, oxygen-, and nitrogen-containing gases, deposits a layer of nitrofluorinated silicate glass onto the substrate. For gap-fill applications, the substrate is biased with a bias power density between 4.8 and 11.2 W/cm and the ratio of flow rate for the oxygen-containing gas to the combined flow rate for all silicon-containing gases in the process chamber is between 1.0 and 1.8, preferably between 1.2 and 1.4. For damascene applications, the bias power density is less than 3.2 W/cm , preferably 1.6 W/cm , and the flow rate ratio is between 1.2 and 3.0. Using optimized parameters, the thin film has a lower dielectric constant and better adhesion properties than fluorosilicate glass.

Growing community of inventors

Sunnyvale, CA, United States of America

Wen Ma