Western Digital Technologies, Inc.

Hitachi Global Storage Technologies Netherlands B.v.

Sandisk Technologies Inc.

Sandisk Enterprise IP LLC

Anand Kulkarni

Zvonimir Z Bandic

Vladislav Bolkhovitin

Brian Walter O'Krafka

Marco Sanvido

Sanjay Subbarao

Kiran Kumar Gunnam

Cyril Guyot

Martin Lueker-Boden

Richard M H New

Jorge Campello De Souza

Tung Thanh Hoang

Aaron Keith Olbrich

Chunqi Han

Dejan Vucinic

Won Ho Choi

Warren Fritz Kruger

Jack Frayer

Qingbo Wang

Minghai Qin

Chao Sun

Donald Joseph Molaro

Navneeth Kankani

Douglas Alan Prins

Charles See Yeung Kwong

Paul Roger Stonelake

Frank R Chu

Huapeng G Guan

Yipei Yu

Jaco Hofmann

Martin Chen

Vikram Varadarajan Kalkunte

Matheus Almeida Ogleari

Graeme Weston-Lewis

Yale Yueh Ma

Certain aspects of the present disclosure provide techniques for partitioning feature maps to improve machine learning model processing. In one aspect, a method, includes partitioning a feature map row-wise into a plurality of feature sub-maps such that: each respective feature sub-map of the plurality of feature sub-maps is defined with respect to a split row determined based on a dense data element count for each row of the feature map; and each feature sub-map of the plurality of feature sub-maps has a same column dimensionality as the feature map; and assigning each of the plurality of feature sub-maps to one of a plurality of tensor compute units and one of a plurality of tensor feature map memory units for processing in parallel.

Just-in-time re-partitioning of feature maps for efficient balancing of compute core workloads

A memory device includes a first memory and a second memory that caches data stored in the first memory. At least one controller of the memory device receives page fault information from a host. The page fault information results from a request for data by the host that is stored in the first memory but is not cached in the second memory when requested by the host. The memory device uses the received page fault information for one or more inputs into a prefetch model trained by Machine Learning (ML) to generate at least one inference. Based at least in part on the at least one inference, prefetch data is cached in the second memory. In one aspect, the page fault information is used to train the prefetch model. In another aspect, the page fault information includes at least one virtual address used by the host for the requested data.

Memory prefetch based on machine learning

An apparatus includes a tensor compute cluster having a plurality of tensor compute units to process a plurality of sub-feature maps in a machine learning application and a tensor memory cluster having a plurality of tensor feature map memory units to store the plurality of sub-feature maps. The apparatus also includes circuitry to partition an input feature map into the plurality of sub-feature maps such that sparsity in each of the plurality of sub-feature maps satisfies a predetermined threshold, and assign each of the plurality of sub-feature maps to one of the plurality of tensor compute units and one of the plurality of tensor feature map memory units for processing in parallel.

Flexible accelerator for sparse tensors in convolutional neural networks

An apparatus includes a first tensor compute cluster configured to receive first input feature tensors, a second tensor compute cluster configured to receive second input feature tensors more sparse than the first input feature tensors, and a vector accelerator. The apparatus also includes circuitry configured to partition an input feature map into a plurality of input feature tensors based on a compression criteria and assign each of the plurality of input feature tensors to one of the first tensor compute cluster, the second tensor compute cluster, or the vector accelerator based upon at least one of parameters including a sparsity and an optimization parameter.

Flexible accelerator for sparse tensors (FAST) in machine learning

A non-volatile memory device includes an array of non-volatile memory cells that are configured to store weights of a neural network. Associated with the array is a data latch structure that includes a page buffer, which can store weights for a layer of the neural network that is read out of the array, and a transfer buffer, that can store inputs for the neural network. The memory device can perform multiply and accumulate operations between inputs and weight of the neural network within the latch structure, avoiding the need to transfer data out of the array and associated latch structure for portions of an inference operation. By using binary weights and inputs, multiplication can be performed by bit-wise XNOR operations. The results can then be summed and activation applied, all within the latch structure.

Accelerating binary neural networks within latch structure of non-volatile memory devices

A system with a multiplication circuit having a plurality of multipliers is disclosed. Each of the plurality of multipliers is configured to receive a data value and a weight value to generate a product value in a convolution operation of a machine learning application. The system also includes an accumulator configured to receive the product value from each of the plurality of multipliers and a register bank configured to store an output of the convolution operation. The accumulator is further configured to receive a portion of values stored in the register bank and combine the received portion of values with the product values to generate combined values. The register bank is further configured to replace the portion of values with the combined values.

A non-volatile memory structure capable of storing layers of a deep neural network (DNN) and perform an inferencing operation within the structure is presented. A stack of bonded die pairs is connected by through silicon vias. Each bonded die pair includes a memory die, having one or more memory arrays onto which layers of the neural network are mapped, and a peripheral circuitry die, including the control circuits for performing the convolution or multiplication for the bonded die pair. The multiplications can either be done in-array on the memory die or in-logic on the peripheral circuitry die. The arrays can be formed into columns along the vias, allowing an inferencing operation to be performed by propagating an input up and down the columns, with the output of one level being the input of the subsequent layer.

Vertical mapping and computing for deep neural networks in non-volatile memory

Example multi-device storage systems, storage devices, and methods provide hosted services on peer storage devices. Storage devices include a storage medium, a logical mapping memory, and a processor for executing hosted services using the logical mapping memory. Each storage device is configured to communicate with peer storage devices over an interconnect fabric. The logical mapping memory includes storage device media logical mapping information configured in continuous logical blocks with a media block size equal to a page programming size of the storage medium. The logical mapping memory also includes host logical mapping information, configured in host logical blocks with a host block size smaller than the media block size, for the peer storage devices.

Multi-device storage system with hosted services on peer storage devices

Systems and methods for dynamically assigning memory array die to CMOS die of a plurality of stacked die during memory operations are described. The plurality of stacked die may be vertically stacked and connected together via one or more vertical through-silicon via (TSV) connections. The memory array die may only comprise memory cell structures (e.g., vertical NAND strings) without column decoders, row decoders, charge pumps, sense amplifiers, control circuitry, page registers, or state machines. The CMOS die may contain support circuitry necessary for performing the memory operations, such as read and write memory operations. The one or more vertical TSV connections may allow each memory array die of the plurality of stacked die to communicate with or be electrically connected to one or more CMOS die of the plurality of stacked die.

Dynamic resource management in circuit bound array architecture

Data management functions are offloaded from a main controller to individual storage devices in a multi-device storage environment. The main controller receives a data management request from a host system, and responds by determining one or more storage devices and one or more data management operations to be performed by the one or more storage devices. The main controller initiates performance of a data management function corresponding to the data management request, by sending one or more data management commands to the one or more storage devices, and initiating one or more data transfers, such as a direct memory access operation to transfer data between a memory buffer of a storage device and a host memory buffer of the host system, and an internal data transfer between two or more of the storage devices using an internal communication fabric of the data storage subsystem.

Offloaded disaggregated storage architecture

Methods, systems, and other aspects for reconstructing data and rebuilding a failed storage device in a storage system using one or more functioning compute resources and/or storage resources of the failed storage device. For example, a method may include, responsive to a detection of a failed storage device in a storage system, locating data and redundancy information in functioning storage device(s) in the storage system for reconstructing data of the failed storage device; issuing peer-to-peer commands to the functioning storage device(s) to obtain the data and the redundancy information from the functioning storage device(s); and reconstructing the data of the failed storage device based on the data and the redundancy information obtained from the functioning storage device(s), wherein a functioning compute resource of the failed computing device at least partially performs one or more of the locating, issuing, and reconstructing.

Rebuild assist using failed storage device

Methods, systems, and other aspects for using a failed storage device in a peer-to-peer (P2P) storage system to perform a storage-centric task. For example, a method may include, responsive to a detection of a failed storage device in a P2P storage system, determining, by the failed storage device, that a storage-centric task is assigned to the failed storage device; and performing, by the failed storage device, the storage-centric task responsive to P2P communications with a functioning storage device in the P2P storage system.

Using failed storage device in peer-to-peer storage system to perform storage-centric task

Example multi-device storage systems, storage devices, and methods provide hosted services on peer storage devices. Storage devices include local memory resources, such as operating memory, remotely addressable memory, or logical mapping memory, and compute resources, such as a processor or coding engine. Each storage device is configured to communicate with a plurality of peer storage devices over an interconnect fabric. The storage devices identify requested hosted services from service host requests received through the interconnect fabric. The storage devices store a plurality of hosted services are to enable access to local memory resources and local compute resources for data management operations for the plurality of peer storage devices.

A main controller in a data storage system having multiple storage devices determines an initial set of memory block candidates for data lifetime operations by receiving from each of a plurality of the storage devices information identifying one or more potential memory block candidates, with respective received memory blocks having been classified by respective storage devices as potential candidates. The main controller determines a set of related memory blocks, and, based on received usage information for the candidate memory blocks and the related memory blocks, selects a target group of memory blocks and initiates performance of the data lifetime operations on the memory blocks of the selected target group.

Growing community of inventors

San Jose, CA, United States of America

Anand Kulkarni