International Business Machines Corporation

Miles Mulholland

Lee Jason Sanders

Gordon Douglas Hutchison

Ben Sasson

Eric John Bartlett

Dominic Tomkins

Alex Dicks

Anuj Chandra

Roderick Guy Charles Moore

Keira Louise Hopkins

Adam Michael Farley

William John Passingham

Richard Alan Bordoli

Imran Imtiaz

Jason Hughes

William James Scales

Alexander Henry Ainscow

Kirsty G Rodwell

Jorden Luke Allcock

A method, system, and computer program product for managing a storage array in a storage system, the storage array comprising a first and second data blocks comprising: receiving a set of cryptographic values from the storage array, the set comprising a first value associated with the first data block, and a second value associated with the second data block; analysing the set of cryptographic values to determine a set of associations, the set of associations comprising a first association between the first data block and the second data block, storing the set of associations; and in response to determining the set of associations, performing a storage management operation on at least one of the first data block and the second data block.

Drive cryptographic value management

A remote copy relationship is maintained between a local computational device and a remote computational device for a first storage volume and a second storage volume. The local computational device transmits information to the remote computational device, wherein the information indicates that there are no deduplication matches between the first storage volume and the second storage volume in the local computational device, and wherein the remote computational device is configured to avoid scanning for deduplication matches between a remote copy of the first storage volume and a remote copy of the second storage volume based on the information that is transmitted.

Limiting deduplication search domains

A method, system, computer program product and computer program for managing a storage system, comprising a primary system and a backup system, wherein the backup system is in a copy relationship with the primary system, comprising: in response to a write operation, the write operation comprising first data: calculating a cryptographic value for the first data; scanning a first directory to identify an entry corresponding to the cryptographic value to determine a first set of addresses associated with the write operation; transmitting the first set of addresses to the backup system; and updating the first directory with a first entry for the write operation, the first entry comprising a pointer to the first set of addresses; updating a second directory with a second entry for the write operation, the second entry comprising a pointer to a second set of addresses corresponding to the first set of addresses.

Optimizing deduplication hit-rate in a remote copy environment

A method, computer program product, and computer system for full allocation volume to deduplication volume migration in a storage system. The method includes moving a physical allocation of stored data associated with a full allocation volume into a virtual address range in a deduplication domain and setting up the deduplication metadata to be a passthrough. The method then performs a background deduplication process on the virtual address range once populated with the physical allocations using a drive query hash interface to perform hash calculations on physical drives at which the data is stored.

Full allocation volume to deduplication volume migration in a storage system

A computer-implemented method enables workload scheduling in a storage system for optimized deduplication. The method includes determining dynamic correlations of deduplications between workload processes in a prior time window. Workload processes include one or more tasks with defined execution timing parameters. The method further includes determining deduplication ratios based on the correlations of the deduplications between the workload processes. The method further includes scheduling multiple workload processes based on a highest determined deduplication ratio of the determined deduplication ratios.

Storage system workload scheduling for deduplication

Method and system for partitioning of deduplication domains in storage systems. The method includes constructing a data structure having multiple nodes representing data chunks and edges between the nodes representing a weighting of deduplication references between the data chunks, and performing clustering of the nodes of the data structure to split the nodes into clusters of tightly related nodes based on the weightings of the edges. The data chunks represented by a cluster of nodes are migrated to a deduplication domain to restrict deduplication to between only the data chunks in the deduplication domain.

Partitioning of deduplication domains in storage systems

A method, computer system, and a computer program product for providing backend deduplication awareness at a virtualizing layer is disclosed. The present invention may include receiving a deduplication information from a backend storage controller associated with performing an input/output (IO) operation to a physical address of a disk. The present invention may include translating the physical address to a logical address to apply to a plurality of storage extents in a virtualizing layer. The present invention may include constructing a graph including corresponding nodes representing the plurality of storage extents and including corresponding deduplication edge weights representing a plurality of deduplications between the plurality of storage extents. The present invention may include identifying at least one subgraph within the constructed graph, wherein the identified at least one subgraph represents a storage extent cluster that is suitable for garbage collection as a cluster.

Backend deduplication awareness

Method, apparatus, and computer program product utilizing a hot-spare node in a storage network having a deduplication fingerprints database. A plurality of nodes is provided in the storage network including at least one active node and at least one hot-spare node. A portion of the deduplication fingerprints database is stored on each active node and on each hot-spare node. Data from the deduplication fingerprints database is provided from at least one of said at least one hot-spare node during normal operation. Responsive to a failure of one of said at least one active node, a portion of the deduplication fingerprints database associated with data stored on the failed one of said at least one active node is retained on one of the at least one hot-spare node. The failed one active node is replaced with one of the at least one hot-spare node.

Utilizing spare network nodes for deduplication fingerprints database

Method and system are provided for storage allocation enhancement of microservices. A method carried out at a microservice orchestrator, includes: categorizing a microservice container, wherein the categorization defines a predicted storage behavior of the microservice container input/output operations; and providing the categorization in association with the microservice container input/output operations to a storage system for use in storage allocation of the input/output operations. A method at a storage controller includes: receiving microservice container input/output operations with an associated categorization, wherein the categorization defines a predicted storage behavior of the microservice container input/output operations; and using the associated categorization for optimizing storage allocation for the input/output operations and/or optimizing garbage collection performance.

Storage allocation enhancement of microservices

Method and system are provided for storage allocation enhancement of microservices. A method carried out at a microservice orchestrator, includes: identifying distinct phases of a run of a microservice container; categorizing the phases of a run of a microservice container, wherein the categorization defines a predicted storage behavior of the microservice container input/output operations in the phase of the microservice container; and providing the categorization in association with the microservice container input/output operations in the phase to a storage system for use in storage allocation of the input/output operations.

Storage allocation enhancement of microservices based on phases of a microservice run

A data recovery method recovers corrupted data after a disaster event in a storage facility operating with deduplication in which copies of data are maintained between volumes through a copy services relationship. The deduplication operates with referencing domains, each having a source grain containing a reference pointing to a location where data is stored and referrers pointing to the source. The data recovery method identifies any source that is pointing to data which is corrupt and then establishes whether a copy services relationship exists between the referencing domain of the source with corrupted data and another referencing domain, in which case the other referencing domain will have a copy of the corrupted data. Provided that the copy is valid, it is written across to replace the corrupted data. The method allows corrupted source data to be recovered without having to perform a full resync or restore.

Data recovery mechanisms in deduplication-enabled storage facilities

Garbage collection is performed for a virtualized storage system whose virtual address space is addressed in extents. Valid data in source extents is copied via a cache into destination extents. Once all valid data in a source extent is copied into one or more destination extents, the source extent may be reused. A source extent is released for reuse only after the one or more destination extents that received the valid data copied from the source extent are determined to be full, and the valid data copied from the source extent to the destination extent via the cache is flushed out of the cache.

Protecting against data loss during garbage collection

Concepts for remote storage of data are presented. Once such concept is a system comprising: a primary storage controller; and a secondary storage controller of a remote data storage system. The primary storage controller is configured to determine a service characteristic of data storage to or data retrieval from the remote data storage system and to communicate service performance signals to the secondary storage controller based on the determined service characteristic. The secondary storage controller is configured to receive service performance signals from the primary storage controller, to compare the received service performance signals with a service requirement so as to determine a service comparison result, and to control data storage to or data retrieval from the remote data storage system based on the service comparison result.

Remote data storage

Method and system are provided for managing capacity in a storage system using copy services. The method is a computer-implemented method that predicts an imminent event due to a constrained resource and identifies one or more copy service relationship that will mitigate the imminent event. The method orchestrates a partial or full invalidation of a copy service relationship to reclaim some of the constrained resource to prevent the imminent event, including updating the copy service relationship metadata to indicate the invalidation.

Predicting and preventing events in a storage system using copy services

A data storage system comprises physical storage, cache memory and a processor connected to the physical storage and the cache memory. The processor is arranged to maintain a set of active regions in the cache memory, each active region having a size equal to an integer multiple of an update size of a flash chip within the physical storage, where the integer could be 1. The processor receives requests for one or more blocks of the cache memory from components within the storage system and allocates one or more blocks from an active region in response to a received request. If the processor determines that all blocks in an active region have been allocated and that all allocated blocks within this region have been written to, then the processor destages the content of this region to the physical storage.

Growing community of inventors

Chandlers Ford, United Kingdom

Miles Mulholland