International Business Machines Corporation

Aayush Gupta

Sangeetha Seshadri

James Lee Hafner

Mohit Saxena

Dean Hildebrand

Nagapramod S Mandagere

Steven Robert Hetzler

Mario Blaum

Paul Henri Muench

William W Owen

Shripad Jayant Nadgowda

Lawrence Yiumchee Chiu

David Darden Chambliss

Nimrod Megiddo

Maohua Lu

Umesh Deshpande

Chetan Sharma

Yuquan Shan

Abdurrahman Yasar

One embodiment provides a method for recovery from failures during garbage collection processing in a system including recording, by a processor, a specific offset within a garbage collection target slot of a log structure associated with a garbage collection transaction. Each transaction record of the garbage collection transaction includes a garbage collection target slot, a victim slot and a beginning offset in the garbage collection target slot.

Chronologically ordered log-structured key-value store from failures during garbage collection

One embodiment provides a method for recovery after failure using a checkpoint in a chronological log-structured key-value store in a system including writing tombstone entries in a log structure for dirty checkpoint records to point to data records in an aborted target slot.

Fast recovery from failures in a chronologically ordered log-structured key-value storage system

In one general embodiment, a computer-implemented method includes analyzing a plurality of data management factors within an object-based storage system, including tracking which of a plurality of proxy servers within the object-based storage system currently holds a lock; determining a data management policy for predetermined data within the object-based storage system, based on the analyzing, including identifying a current location of the lock within one of the plurality of proxy servers within the object-based storage system; and sharing the data management policy for the predetermined data, including the current location of the lock within the object-based storage system, with a load balancer associated with the object-based storage system.

Sharing a data management policy with a load balancer

Embodiments for optimizing database backups to achieve a Recovery Time Object (RTO). A user-defined RTO configured for one or more databases is received. A backup frequency for initiating backups of the one or more databases is determined based on a continuously predicted recovery time associated with a plurality of factors. The backups of the one or more databases are executed at the determined backup frequency to ensure the user-defined RTO is achieved for the backups of the one or more databases. In some embodiments, a recovery window of the one or more databases may be increased using an RTO-aware tiered or remote storage caching operation for portions of the database, and an RTO-aware re-sharding operation on sharded databases may be performed when the backup frequency exceeds a predetermined threshold such that each shard may be restored within the user-defined RTO.

Selective placement and adaptive backups for point-in-time database recovery

One embodiment provides a method including storing a key-value store in a memory. Out-of-place operations are performed to maintain chronological ordering of the operations by enforcing ordering of the operations on a storage layer in presence of a garbage collection transaction without an explicit secondary index.

Chronologically ordered out-of-place update key-value storage system

A method for memory page erasure-correcting property generation in a storage array includes dividing data into multiple stripes for storage in a storage array including multiple storage devices with a topology of a hypercube of a dimension t&#x2265;3. The storage devices in same hypercubes of dimension t&#x2212;1 including the hypercube of dimension t have even parity. An intersection of two non-parallel planes in the hypercube topology is a line, and each point along a line is a storage device in the storage array. Erasure-correcting properties are generated for the data using three nested codes, wherein a first nested code has even parity over planes of class 0, 1 and 2, a second nested code has a first global parity, and a third nested code has a second global parity and a third global parity.

Erased memory page reconstruction using distributed coding for multiple dimensional parities

One embodiment provides a method for recovery from failures during garbage collection processing in a system including performing, by a processor, chronological garbage collection transactionally. A specific offset within a target slot of a log structure associated with the garbage collection transaction is recorded. Records are skipped from an initial offset in the target slot of the log structure recorded by the garbage collection transaction until an end of the target slot.

One embodiment provides a method for recovery after failure using a checkpoint in a chronological log-structured key-value store in a system including recording, by a processor, a system state prior to an aborted garbage collection operation. The processor writes tombstone entries in a log structure for dirty checkpoint records to point to data records in an aborted target slot. New checkpoint records are inserted in the log structure for the dirty checkpoint records.

Embodiments are provided for organizing data elements in a log-structured storage system. A profile is created based on a population of data elements in cache memory related to a log structured storage system. A layout for storage of the data elements on persistent storage related to the log-structured storage system is determined, which includes identifying two or more related data elements based on the profile. A garbage collection process is performed on the persistent storage based on the determined layout, which includes placing the related data elements within substantially close proximity on the persistent storage.

Determining a data layout in a log structured storage system

In one embodiment, a computer-implemented method includes selecting a cache block descriptor (CBD) from amongst a plurality of CBDs stored to a cache storage device to defragment based on a determination of utilization of a particular fine block descriptor (FBD) having a first size that is allocated to the selected CBD. The cache storage device includes a free pool of FBDs having various sizes that is available for use in the plurality of CBDs. Also, the particular FBD having the first size has a lowest availability in the free pool of FBDs. Other methods, systems, and computer program products are described in accordance with additional embodiments.

Reducing defragmentation in a multi-grained writeback cache

In one general embodiment, a computer program product for sharing a data management policy with a load balancer comprises a computer readable storage medium having program instructions embodied therewith, wherein the computer readable storage medium is not a transitory signal per se. Additionally, the program instructions are executable by a processor to cause the processor to perform a method comprising analyzing, by the processor, a plurality of data management factors within an object-based storage system, determining, by the processor, a data management policy for predetermined data within the object-based storage system, based on the analyzing, and sharing, by the processor, the data management policy for the predetermined data with a load balancer associated with the object-based storage system.

One embodiment provides a method for retrospective snapshot creation including creating, by a processor, a first snapshot that captures logical state of a data store at a first key. Creation of the first snapshot is based on determining a log offset corresponding to the first key, determining existence of a second snapshot that captures logical state of the data store and recording a retrospective snapshot at a last valid log address offset prior to the first key upon a determination that the second snapshot exists based on determining at least one of: whether log address offsets from a first log entry of a log to a log entry of the log at the first key are contiguous and whether log address offsets from the second snapshot to the first key are contiguous.

Retrospective snapshots in log-structured storage systems

Provided are a computer program product, system, and method for determining storage tiers for placement of data sets during execution of tasks in a workflow. A representation of a workflow execution pattern of tasks for a job indicates a dependency of the tasks and data sets operated on by the tasks. A determination is made of an assignment of the data sets for the tasks to a plurality of the storage tiers based on the dependency of the tasks indicated in the workflow execution pattern. A moving is scheduled of a subject data set of the data sets operated on by a subject task of the tasks that is subject to an event to an assigned storage tier indicated in the assignment for the subject task subject. The moving of the data set is scheduled to be performed in response to the event with respect to the subject task.

Determining storage tiers for placement of data sets during execution of tasks in a workflow

A computer-implemented method for distributing data among memory devices may include performing successive spatial partitionings of a graph of data based upon vertices and associated edges to generate spatial partitions with each spatial partition having a respective number of edges below a threshold. The method may also include ordering the spatial partitions and distributing the spatial partitions among the memory devices based upon the ordering.

Distributing data by successive spatial partitionings

Embodiments relate to host-side cache migration. An aspect is a method that includes determining pre-fetch hints associated with a source cache that is local to a source host machine. The source cache includes pages of cache data for a virtual machine. The pre-fetch hints are sent to a pre-fetch planner to create a pre-fetch plan. The pre-fetch hints are sent based on migration of the virtual machine to a target host machine including a target cache that is local. At the source host machine, a cache migration request is received based on the pre-fetch plan. A first subset of the pages is sent from the source cache through a host-to-host communication channel to the target cache based on the cache migration request. A second subset of the pages is sent from the source cache through a host-storage communication channel to a shared storage to be relayed to the target cache.

Growing community of inventors

San Jose, CA, United States of America

Aayush Gupta