Sandisk Technologies Inc.

Oracle America, Inc.

Western Digital Technologies, Inc.

Sun Microsystems, Inc.

Sandisk Enterprise IP LLC

International Business Machines Corporation

Wind Jammer Technologies, LLC

Oracle International Corporation

Schooner Information Technology, Inc.

Brian Walter O'Krafka

Manavalan Krishnan

Vladislav Bolkhovitin

Johann George

Sanjay Subbarao

Michael John Koster

Darpan Dinker

Ashok V Krishnamoorthy

John E Cunningham

Pranay Koka

Anand Kulkarni

Ramesh Chander

Niranjan Patre Neelakanta

Tomy Ammuthan Cheru

Sriram Srinivasan Mandyam

Earl T Cohen

Xuezhe Zheng

Ronald Ho

Harihara Kadayam

Thomas M McWilliams

Ilya A Sharapov

Robert James Ramirez

John Richard Busch

Ramanathan Raghavan

Evgeniy Firsov

Frederic H Tudor

Allen Samuels

Warren Fritz Kruger

Robert John Kroeger

Andrew David Eckhardt

Roy Stuart Moore

Ilya Gluhovsky

Darryl Ouye

Shahram Salamian

Evgeny H Firsov

Patrick Chiu

Xuehze Zheng

Vishal Kanaujia

Miwako Tokugawa

Kai Rothauge

Nadeem Malik

Aaron Keith Olbrich

Jeffrey Thomas Kreulen

Avijit Saha

Pankaj Mehra

Seung-Hwan Song

Christopher Lyle Johnson

James G Mitchell

Arup De

Akshay Mathur

Vivek Shivhare

Nagi Reddy Chodem

Brett Adam St Onge

Shailendra Deva

Robert J Kroeger

Archie Don Barrett, Jr

Ajit Kumar Tapse

Zhengheng Wang

Thibash Rajamani

Executing a query in a disaggregated cluster. A query is received at the disaggregated cluster. A query graph is created based on the query that identifies a hierarchy of vertices, where each vertex is associated with a set of data responsive to at least a portion of the query. The compute nodes process the query graph by first identifying a minimum set of tables, files, and objects stored on the storage nodes required to retrieve data that satisfy the query. Next, the compute nodes selectively assign the identified tables, files, and objects to a leaf vertex of the query graph to optimize retrieving data from the storage nodes. Thereafter, the compute nodes process the retrieved data sets associated with each vertex using separate threads of execution for each vertex of the query graph such that leaf vertices are performed in parallel. The compute nodes then provide a result set.

Disaggregated query processing utilizing precise, parallel, asynchronous shared storage repository access

Systems, methods and/or devices are used to perform memory-efficient mapping of block/object addresses. In one aspect, a method of managing a storage system having one or more storage devices includes a tiered data structure in which each node has a logical ID and entries in the nodes reference other nodes in the tiered data structure using the logical IDs. As a result, when a child node is updated and stored to a new location, but retains its logical ID, its parent node does not need to be updated, because the logical ID in the entry referencing the child node remains unchanged. Further, the storage system uses a secondary mapping table to translate the logical IDs to the corresponding physical locations of the corresponding nodes. Additionally, the secondary mapping table is cached in volatile memory, and as a result, the physical location of a required node is determined without accessing non-volatile memory.

Memory-efficient block/object address mapping

Executing a query in a disaggregated cluster. A query is received at the disaggregated cluster. A query graph is created based on the query that identifies a hierarchy of vertices, where each vertex is associated with a set of data responsive to at least a portion of the query. The compute nodes process the query graph by first identifying all tables, files, and objects stored on the storage nodes whose access is required to retrieve data that satisfy the query. Next, the compute nodes selectively assign the identified tables, files, and objects to a leaf vertex of the query graph to optimize retrieving data from the storage nodes. Thereafter, the compute nodes process the retrieved data sets associated with each vertex using separate threads of execution for each vertex of the query graph such that leaf vertices are performed in parallel. The compute nodes then provide a result set.

Disaggregated query processing on data lakes based on pipelined, massively parallel, distributed native query execution on compute clusters utilizing precise, parallel, asynchronous shared storage repository access

Example multi-device storage systems, storage devices, and methods provide hosted services on peer storage devices. Storage devices include a storage medium, a logical mapping memory, and a processor for executing hosted services using the logical mapping memory. Each storage device is configured to communicate with peer storage devices over an interconnect fabric. The logical mapping memory includes storage device media logical mapping information configured in continuous logical blocks with a media block size equal to a page programming size of the storage medium. The logical mapping memory also includes host logical mapping information, configured in host logical blocks with a host block size smaller than the media block size, for the peer storage devices.

Multi-device storage system with hosted services on peer storage devices

A system and method improve the performance of non-volatile memory storage by rebuilding, on the fly, 'lost data' in response to a read request, which identifies data to be read or recovered, by identifying a parity data storage device in a set of data storage devices that contains parity corresponding to the identified data; sending a reconstruction request to a respective data storage device, which may be the parity data storage device or other data storage device in the system, to reconstruct the identified data, and receiving the identified data from the respective data storage device. The reconstruction request commands the respective data storage device to retrieve, via peer-to-peer read requests, from other data storage devices, data from one or more data blocks, and to reconstruct the identified data based on the retrieved data and parity data locally stored at the parity data storage device.

Data storage drive rebuild with parity generation offload using peer-to-peer data transfers

Apparatuses, systems, methods, and computer program products are disclosed for key-value stores with partial data access. An interface module is configured to receive a data object for storage in a key-value store. The data object may include a key and a value. A block object module is configured to generate a plurality of block objects smaller than the data object. A block object may include a new key and a new value. The new key may be based on the key for the data object and on metadata for the new value. The new value may be based on at least a portion of the value for the data object. A storage module is configured to store the block objects in the key-value store.

Key-value store with partial data access

In response to receiving a request to perform a transaction with two or more memory operations on one or more tiered data structures, the memory controller: writes a start transaction record to the log stream including a transaction identifier corresponding to the transaction; and performs the two or more memory operations. For a first memory operation associated with a key, the memory controller: writes a new data object in a datastore; assigns, in a key-map, a location of the new data object to the key; maintains an old data object in the datastore whose location was previously assigned to the key; and writes an operation commit record to a log stream upon completion of the first memory operation. In accordance with a determination that the two or more memory operations are complete, the memory controller writes a transaction commit record to the log stream including the transaction identifier.

Methods and systems for efficient non-isolated transactions

Data management functions are offloaded from a main controller to individual storage devices in a multi-device storage environment. The main controller receives a data management request from a host system, and responds by determining one or more storage devices and one or more data management operations to be performed by the one or more storage devices. The main controller initiates performance of a data management function corresponding to the data management request, by sending one or more data management commands to the one or more storage devices, and initiating one or more data transfers, such as a direct memory access operation to transfer data between a memory buffer of a storage device and a host memory buffer of the host system, and an internal data transfer between two or more of the storage devices using an internal communication fabric of the data storage subsystem.

Offloaded disaggregated storage architecture

Methods, systems, and other aspects for reconstructing data and rebuilding a failed storage device in a storage system using one or more functioning compute resources and/or storage resources of the failed storage device. For example, a method may include, responsive to a detection of a failed storage device in a storage system, locating data and redundancy information in functioning storage device(s) in the storage system for reconstructing data of the failed storage device; issuing peer-to-peer commands to the functioning storage device(s) to obtain the data and the redundancy information from the functioning storage device(s); and reconstructing the data of the failed storage device based on the data and the redundancy information obtained from the functioning storage device(s), wherein a functioning compute resource of the failed computing device at least partially performs one or more of the locating, issuing, and reconstructing.

Rebuild assist using failed storage device

Memory-efficient object address mapping in a tiered data structure

Example multi-device storage systems, storage devices, and methods provide hosted services on peer storage devices. Storage devices include local memory resources, such as operating memory, remotely addressable memory, or logical mapping memory, and compute resources, such as a processor or coding engine. Each storage device is configured to communicate with a plurality of peer storage devices over an interconnect fabric. The storage devices identify requested hosted services from service host requests received through the interconnect fabric. The storage devices store a plurality of hosted services are to enable access to local memory resources and local compute resources for data management operations for the plurality of peer storage devices.

A system and method improve the performance of non-volatile memory storage by offloading parity computations to facilitate high speed data transfers, including direct memory access (DMA) transfers, between a remote host and a non-volatile memory based storage system, such as a flash memory based data storage device (e.g., SSD). In conjunction with writing to non-volatile memory storage, a stripe map is used to target a selected data storage device for parity generation. All data of a stripe is transmitted to the selected data storage device to generate the parity and the generated parity is propagated from the selected data storage device to other data storage devices in the stripe. The data for the stripe may also be propagated from the selected data storage device to the other data storage devices in the stripe.

Parity generation offload using peer-to-peer data transfers in data storage system

Systems, methods, and/or devices are used to store metadata in a storage system. In one aspect, a first user space module sends a logical memory request to a memory management module of a kernel space module. The logical memory request includes data and metadata. A second user space module obtains the metadata of the logical memory request. A storage engine of the second user space module determines, in accordance with the obtained metadata, a location in non-volatile memory for the data. A second user space module generates a physical memory request including an indication of the non-volatile memory for the data. The second user space module transmits the physical memory request to the kernel space memory management module.

User space data storage management

An apparatus includes a non-volatile memory and a controller coupled to the non-volatile memory. The controller is configured to receive a command from a device to perform a write operation at the non-volatile memory. The command indicates a plurality of logical addresses, data associated with the plurality of logical addresses, and a number of write operations associated with the command.

Group write operations for a data storage device

Growing community of inventors

Austin, TX, United States of America

Brian Walter O'Krafka