International Business Machines Corporation

Globalfoundries Inc.

Globalfoudries Inc.

Paul Henri Muench

Lawrence Yiumchee Chiu

Sangeetha Seshadri

Yang Liu

Umesh Deshpande

Cheng-Chung Song

Chao G Li

Hyojun Kim

Lokesh M Gupta

Michael T Benhase

Roman Alexander Pletka

Ioannis Koltsidas

Yu-Cheng Hsu

Stephen LaRoux Blinick

Aayush Gupta

Clement Lambert Dickey

Priyaranjan Behera

Matthew J Kalos

Nikolas Ioannou

Gail Andrea Spear

Joseph S Hyde, Ii

Yong Guo

Wilfred Edmund Plouffe, Jr

Mei Mei

Nicholas Linck

Timothy Lee Toohey

Evangelos Stavros Eleftheriou

Robert Haas

Robert Michael Rees

Maohua Lu

Mohit Saxena

Xiao-yu Hu

Xiaoyu Hu

Thanh Tan Pham

Shan Fan

Daniel W Fok

ZhiQiang Wang

Xioa-Yu Hu

Chetan Sharma

Yuquan Shan

An embodiment for a method of cost-aware tiering for data storage. The embodiment may detect data to be stored in one or more storage systems and gathering access and usage information. The embodiment may maintain a first data structure including periods of activity below a defined threshold for the detected data, and a common data structure to identify and track application-wide data patterns. The embodiment may gather static pricing information and context pricing information for the one or more storage systems and calculate a tiering threshold corresponding to a continuous period of activity below a second defined threshold required for which local storage costs exceed a tiering cost. The embodiment may calculate for inactive data segments, probabilities of the inactive data segments remaining inactive for a duration that exceeds the calculated tiering threshold. The embodiment may calculate a risk score and generate a tiering list for storing the data.

Cost-aware tiering for data storage

Technology for identifying relevant data (also called 'data of interest') for applying a machine logic based rule and for increasing the frequency of making backups (for example, snapshots) of the relevant data.

Intelligent backup engine for data freshness

A computer-implemented method according to one embodiment includes retrieving details of a data backup system; determining whether a first data backup operation will complete within the data backup system before a second data backup operation is initiated within the data backup system, based on the details of the data backup system; and conditionally adjusting one or more elements of the data backup system, based on the determining.

Adjusting data backups based on system details

A computer-implemented method according to one embodiment includes causing a failure event in each of a plurality of microservices of a system and collecting failure effect data associated with the caused failure events. A mapping is created detailing transition of the microservices between different states and the collected failure effect data is analyzed for creating the mapping. The method further includes outputting a predetermined notification in response to a determination that a first of the microservices is close to experiencing a predicted failure event, and outputting a suggested solution for repairing the system in response to a determination that the system has failed, using the mapping to identify a root cause of the system failure. Using the mapping to identify the root cause of the system failure includes identifying the microservices that caused the system failure.

Identification and/or prediction of failures in a microservice architecture for enabling automatically-repairing solutions

Provided are a computer program product, system, and method for managing data in a storage. Each index entry in and index identifies a log address in the log of one of the records. Each record in the log includes a key and value, and the log addresses map to physical addresses in the storage. A plurality of processed log addresses in the log are processed by determining whether the index indicates that the processed log address includes a current version of the record in the log; adding the record at the processed log address to a selected log address in the log in response to determining that the index indicates that the processed log address comprises the current version of the record; and deleting a record at the physical address in the storage to which the processed log address maps.

Managing data in storage according to a log structure

Embodiments for optimizing database backups to achieve a Recovery Time Object (RTO). A user-defined RTO configured for one or more databases is received. A backup frequency for initiating backups of the one or more databases is determined based on a continuously predicted recovery time associated with a plurality of factors. The backups of the one or more databases are executed at the determined backup frequency to ensure the user-defined RTO is achieved for the backups of the one or more databases. In some embodiments, a recovery window of the one or more databases may be increased using an RTO-aware tiered or remote storage caching operation for portions of the database, and an RTO-aware re-sharding operation on sharded databases may be performed when the backup frequency exceeds a predetermined threshold such that each shard may be restored within the user-defined RTO.

Selective placement and adaptive backups for point-in-time database recovery

In accordance with one aspect of the present description, in response to a detection by a storage controller, of an operation by a host relating to migration of input/output operations from one host to another, a cache server of a storage controller, transmits to a target cache client of the target host, a cache map of the source cache of the source host wherein the cache map identifies locations of a portion of the storage cached in the source cache. In response, the cache client of the target host, may populate the target cache of the target host with data from the locations of the portion of the storage, as identified by the cache map transmitted by the cache server, which may reduce cache warming time. Other features or advantages may be realized in addition to or instead of those described herein, depending upon the particular application.

Cache migration management in a virtualized distributed computing system

One embodiment provides a method for retrospective snapshot creation including creating, by a processor, a first snapshot that captures logical state of a data store at a first key. Creation of the first snapshot is based on determining a log offset corresponding to the first key, determining existence of a second snapshot that captures logical state of the data store and recording a retrospective snapshot at a last valid log address offset prior to the first key upon a determination that the second snapshot exists based on determining at least one of: whether log address offsets from a first log entry of a log to a log entry of the log at the first key are contiguous and whether log address offsets from the second snapshot to the first key are contiguous.

Retrospective snapshots in log-structured storage systems

In one embodiment, a method includes determining a data record partition size based on resources used for transferring data from a higher storage tier to one or more lower storage tiers. The method also includes determining which data records stored to the higher storage tier are suitable for export to the one or more lower storage tiers, determining a distribution mapping of the first memory, the distribution mapping indicating a relative distribution of storage locations for all of the data records that are stored to the higher storage tier, identifying all sets of contiguously stored data records on the higher storage tier that are suitable for export and greater in size than the data record partition size, logically sorting, in a descending order of size, the sets of contiguously stored data records, and sending a list of logically sorted sets of contiguously stored data records to an exporter.

Efficient export scheduling using index and schema to determine data distribution

Provided are a computer program product, system, and method for handling failure of a command to add a record to a log. The log has records at log addresses. A command is received to add a record to the log indicating a current log address at which the record is stored in the log. A determination is made as to whether the command has a resend parameter set, wherein the resend parameter indicates whether the command is being resent to add the record. A determination is made as to whether there is a log address in the log having the record following the current log address in response to determining that the resend parameter indicates that the command is being resent. The determined log address in the log is returned.

Handling failure of a command to add a record to a log

Provided are a computer program product, system, and method for providing access information to a storage controller to determine a storage tier for storing data. Access information is maintained for each data record in a data store, wherein the access information indicates a level of access to each data record in the data store. A write request directed to a data record in the data store is received. A command is generated identifying the data record and including the access information for the data record. The command is transmitted to the storage controller, wherein the storage controller uses the access information to determine one of the plurality of storage tiers on which to store the data record.

Providing access information to a storage controller to determine a storage tier for storing data

One embodiment provides a method for retrospective snapshot creation including creating, by a processor, a first snapshot that captures logical state of a data store at a first time in a time range. Creation of the first snapshot is based on determining existence of a second snapshot that captures logical state of the data store and recording a retrospective snapshot at a last valid log address offset prior to the first time upon a determination that the second snapshot exists based on determining at least one of: whether log address offsets from a first log entry of a log to a log entry of the log at the first time are contiguous and whether log address offsets from the second snapshot to the first time are contiguous.

Retrospective snapshots in log structured storage systems

A method to efficiently checkpoint and reconstruct an in-memory index associated with a log-structured object store includes enabling asynchronous write operations to occur to a log-structured object store. The log-structured object store utilizes an in-memory index to access objects therein. The method further enables checkpoint operations to occur to the log-structured object store without pausing the asynchronous write operations. When initiating checkpoint operations, the method establishes a 'begin checkpoint' marker on the log-structured object store. This 'begin checkpoint' marker is configured to point to an oldest known log location recorded in the in-memory index. In the event the in-memory index is lost, the method reconstructs the in-memory index by analyzing the log-structured object store starting from the oldest known log location. A corresponding system and computer program product are also disclosed and claimed herein.

Efficient index checkpointing in log-structured object stores

Growing community of inventors

San Jose, CA, United States of America

Paul Henri Muench