International Business Machines Corporation

Ron Edelstein

Yariv Bachar

Oded Sonin

Shay Haim Akirav

Asaf Levy

Michael Hirsch

Ariel J Ish-Shalom

Liran Loya

Alon Horowitz

Itai Tzur

Ehud Meiri

Methods, computer systems, and computer program products for processing data a computing environment are provided. The computer environment for data deduplication storage receives a plurality of write operations for deduplication storage of the data. The data is buffered in a plurality of buffers with overflow temporarily stored to a memory hierarchy when the data received for deduplication storage is sequential or non sequential. The data is accumulated and updated in the plurality of buffers per a data structure, the data structure serving as a fragment map between the plurality of buffers and a plurality of user file locations. The data is restructured in the plurality of buffers to form a complete sequence of a required sequence size. The data is provided as at least one stream to a stream-based deduplication algorithm for processing and storage.

Method and system for processing data

A snapshot of selected objects in a source repository is created in response to the user-initiated replication. The snapshot is designated as a snapshot replication job. Unsynchronized objects in a target destination are detected by comparing a state of the selected objects in the snapshot with a current state of the target destination at the time of execution of the snapshot replication job. An object identification (ID) and a state ID are sent to the target destination and used to locate and compare the selected object with a current state of the selected object in the target destination.

User initiated replication in a synchronized object replication system

For efficient centralized stream initiation and retry control in a computing environment, using a centralized data streams management module for both managing when data streams should be opened and sent from a source location to a destination and for determining when to reattempt opening data streams sent from the source location to the destination after an nth number of consecutive failed attempts using an incrementing time calculation. The incrementing time calculation computes a dynamically calculated time period.

Efficient centralized stream initiation and retry control

For increasing data storage capacity using a processor device, increasing logical capacity of data storage having a multiplicity of storage units containing fixed partitioned segments by moving a selected number of the fixed partitioned segments in one or more storage units to a reserved storage unit.

Increasing data storage capacity

A snapshot of selected objects in a source repository is created in response to the user-initiated replication. The snapshot is designated as a snapshot replication job. Unsynchronized objects in a target destination are detected by comparing a state of the selected objects in the snapshot with a current state of the target destination at the time of execution of the snapshot replication job. One of the selected objects is retrieved in an ordered manner from the snapshot. An object identification (ID) and a state ID are sent to the target destination and used to locate and compare the selected object with a current state of the selected object in the target destination.

A task is marked as dependent upon a preceding task. The task that is attempted to be taken for execution from a head of a pending task queue that is marked is deferred. The deferred task is removed from the pending task queue and placed in a deferred task queue. The deferred task is reinserted back into the pending task queue for execution upon determining that the preceding tasks are completed.

Prioritizing deferred tasks in pending task queue based on creation timestamp

In one embodiment, by way of example only, a resource threshold limits the resource usage of each of the multiplicity of clients. The resource threshold of each of the multiplicity of clients increases over time and decreases when the resources deplete and a resource request of a client is rejected. A subset of clients is created from the multiplicity of clients having a resource usage greater than the resource usage of the rejected client. A dynamic average of a normalized number of resources is calculated from the subset of clients. The resource threshold of each client from the subset of clients is decreased based on the dynamic average of the subset of clients, and further acquisition of resources is restricted from each client from the subset of clients in order to bring the resource usage of each of the subset of clients under the resource threshold.

Dynamically balancing resource requirements for clients with unpredictable loads

A snapshot of selected objects in a source repository is created in response to the user-initiated replication. The snapshot is designated as a snapshot replication job. The snapshot replication job is added to the end of a replication queue to await execution for the synchronized object replication. Unsynchronized objects in a target destination are detected by comparing a state of the selected objects in the snapshot with a current state of the target destination at the time of execution of the snapshot replication job. The unsynchronized objects in the target destination are synchronized based upon the comparison of the state of the selected objects in the snapshot with the current state of the target destination at the time of execution of the snapshot replication job.

For resource allocation of resources for a dual activity system, each of the dual activities may be started at a static quota and is allocated its respective static quota of resources, and determining which of the dual activities is a demanding dual activity. The resource boundary may be increased for a resource request for at least one of the dual activities until a resource request for an alternative one of the at least one of the dual activities is rejected. A reduced actual resource boundary for the demanding dual activity based on a multiplicative decrease of the dual activity's actual resource boundary is calculated, and the resource boundary for the at least one of the dual activities may be reduced, and a wait after decrease mode may be commenced until a current resource usage is one of less than and equal to the reduced resource boundary.

Resource allocation for a plurality of resources for a dual activity system

Dependency management in task scheduling

A plurality of log processes are synchronized. Each is independently performed in parallel with one another, into a single set of log files. A line buffering mechanism of an operating system (OS) of the computing environment forecloses interleaving of the log processes. Log management operations are concurrently performed by a single process protected by a file-system lock of the OS. The log management operations include at least one of a log compression, log retention, and log rotation operation.

Growing community of inventors

Tel Aviv, Israel

Ron Edelstein