Emc IP Holding Company LLC

Dell Products Inc.

Emc Corporation

Maksim Vazhenin

Mikhail Danilov

Ilya Volzhev

Vadim Nikolaevich Agarkov

Sergey Storozhevykh

Michael E Bappe

Andrei Paduroiu

Mikhail Edkov

Sergey Storozhevjkh

Annulment of events written into an ordered event stream (OES) storage system is disclosed. Events can be written to one or more segments of an OES. An event can be invalid and overwriting the event can be desirable. A stream-cut event can be employed to facilitate annulment of the event via altering a portion of a segment of an OES. The stream-cut event can be durable. Annulment can avoid rewriting an entire stream and/or complicating a stream by adding mitigating events without overwriting invalid events. In an aspect, annulment can comprise moving a write cursor according to a stream-cut event. The annulment can be lossy or lossless. In an embodiment, annulment can be performed in single tier OES storage system. In another embodiment, multi-tier OES storage systems can facilitate annulment.

Ordered event stream event annulment in an ordered event stream storage system

Event prioritization for an ordered event stream (OES) is disclosed. Unlike conventional prioritization techniques, the disclosed subject matter can be performed by an OES data storage system to provide direct, rather than indirect, control of prioritization. In an embodiment, a prioritized hashed key (PHK) can be determined from an event characteristic and an indicated event priority value based on a selectable priority-sensitive hashing function. As such, events with a same key characteristic but different indicated priorities can have different PHKs, events with different key characteristics but the same indicated priority can have different PHKs, and events with the same key characteristic and the same priority can have a same PHK. An event priority can be inherently comprised in the PHK without needing to explicitly store the priority value with a written event in the OES. Moreover, the disclosed prioritization for the OES can be compatible with OES scaling techniques.

Event prioritization for an ordered event stream

Data verification technology for ordered event stream (OES) events written into an ordered event stream storage system is disclosed. The verification technology provides perfect reliability. The verification technology further requires low storage overhead in comparison to typical checksums, storing replicated data, etc. Test event data can be generated in a reproducible manner based upon determined OES metadata. OES metadata can be determined from input received via a user interface, via characteristics of an OES storage system, etc., and can be stored for later use in data verification. The test event data can be stored to a portion of an OES storage system under test. The stored test event data can subsequently be verified by using the stored OES metadata to regenerate test event data for comparison to the stored test event data. The test event ordering can be verified via sequence information included in the stored test event data.

Test data verification for an ordered event stream storage system

Data migration from a legacy data storage system into an ordered event stream (OES) data storage system is disclosed. In contrast to conventional techniques, the disclosed subject matter can provide for migrating legacy data into a first portion of a vintage OES (VOES) and new data, intended for the legacy data storage system during the migration process, into a second portion of the VOES. The first and second portions of the VOES can then be linked at completion of the migration process. Avoiding writing new data to the legacy data storage system prior to migration enables a corresponding reduction in data access disruption. Moreover, event sequencing by separating migrated legacy events from new events can preserving event order. Scaling of vintage streams can be supported. Vintage streams can support function calls to a single VOES in contrast to the more conventional use of multiple streams.

Migration of legacy data into an ordered event stream

The technology described herein is directed towards executing continuous query tasks for monitoring a data storage system to obtain real-time metrics and store query results, in which the continuous query tasks are resource-demanding yet need to execute on limited system resources. A continuous query tasks scheduler levels the load produced by scheduling continuous query tasks with starting time offsets based on the period of continuous query tasks and load descriptions for the tasks. Further, a continuous query task that produces too high of a workload is divided into continuous query subtasks, which are scheduled at subtask starting time offsets.

Continuous query scheduling and splitting in a cluster-based data storage system

Embodiments of the present disclosure relate to a system, a computer program product and a method for synchronizing data between a source disk and a target disk in a cluster by performing synchronization between a source disk and a target disk, the synchronization being performed while a plurality of application I/Os on a plurality of nodes in a cluster are configured to access the source disk; and wherein a coordinator and a plurality of workers in the cluster are configured to manage copying data from the source disk to the target disk.

Disk synchronization

Processing load and incoming inter-zone network traffic for remote zones during geographically distributed (GEO) erasure coding can be reduced. In one aspect, GEO erasure coding can be performed in multiple phases, wherein each phase can be distributed between zones. Moreover, during a first phase, partially-finished coding chunks can be generated by frontend zones. Further, during an intermediate phase, partially-finished coding chunks that are associated with a common coding chunk can be transferred between the frontend zones and combined to generate consolidated partial coding chunks. Furthermore, during a final phase, the remote zones can receive and combine the consolidated partial coding chunks to generate a complete coding chunk that can be utilized for data recovery.

Disaster recovery with consolidated erasure coding in geographically distributed setups

Data lifetime-aware migration is presented herein. A data sorting component can generate a list of data portions that have been sorted in ascending order of respective determined probabilities that the data portions will be deleted from a source storage device during a migration of the data portions from the source storage device to a destination storage device&#x2014;the migration comprising copying at least a part of the data portions from the source storage device to the destination storage device. Further, a data lifetime-aware component can iteratively migrate, based on the list in the ascending order of the respective determined probabilities, at least the part of the data portions from the source storage device to the destination storage device.

Data lifetime-aware migration

A method, computer program product, and computer system for performing, via a first computing device, a copy sweep operation to a first range of data on a source storage device. It may be determined that the copy sweep operation has failed. A message may be sent to a second computing device to suspend I/O operations to the first range of data. The copy sweep operation may be retried based upon, at least in part, determining that the copy sweep operation has failed, wherein the copy sweep operation may be retried without the first computing device receiving acknowledgement that the second computing device is suspending the I/O operations to the first range of data.

System and method for synchronization in a cluster environment

In one aspect, a method, includes designating a range of memory in a source disk to synchronize with a target disk, receiving blocks of data that have changed in the source disk and were cloned to a target disk, marking the blocks of data as synchronized and copying data in portions of the range that are not marked as synchronized in the source disk to the target disk.

Disk synchronizing

In one aspect, a method includes performing synchronization between a source disk and a target disk. The performing includes reading data, in a first read, with a first size from a source disk, writing the data from the first to a target disk, reading data, in a second read, with the first size from the source disk, determining if the data from the first read is equal to the data from the second read, reducing the first size if the data from the first read is not equal to the data in the second read and repeating the reading data in the first read, the writing, the reading data in the second read and the determining.

Growing community of inventors

Saint Petersburg, Russia

Maksim Vazhenin