Emc IP Holding Company LLC

Emc Corporation

Yu Teng

Lu Lei

Ao Sun

Gary Jialei Wu

Wesley Wei Sun

Chun Xi Kenny Chen

Jialei Wu

Mikhail Danilov

Chen Xiao Wang

Julius Jian Zhu

Rulian W Fiske

Changbin Gong

Wei Gao

Konstantin Buinov

Shashwat Srivastav

Dong Wang

Fernando Oliveira

Sriram Sankaran

Ivan Tchoub

Maxim S Trusov

Jun Luo

Frank Yifan Huang

Ronnie Yu Cai

Song Liu

Rui Tang

Chunxi Cheng

Data stream processing is performed. For instance, a method includes: in response to a query request on an input data stream, determining data labels for input data items in the input data stream, the data labels at least indicating a first matching relationship between the input data items and a first query condition of the query request; determining an output data stream corresponding to the input data stream, the output data stream including output data items corresponding to the input data items, and each output data item having a data label; and determining query information corresponding to the query request based on the output data stream. In this way, it is possible to respond to real-time query requests for data streams.

Method, electronic device, and computer program product for data stream processing

Storage blocks are managed. For instance, a set of write parameters and a set of deletion parameters are obtained related to a target storage block. In response to the set of write parameters matching the set of deletion parameters, a first data length is obtained for the target storage block, the first data length being determined in response to receiving a write request for the target storage block. Further, reclaim information is determined related to the target storage block based on the first data length and the set of deletion parameters. It is thus possible to reduce times of scanning the entire object table to determine whether there is an object referring to the storage block, thereby reducing the time consumed by the verification process and improving the reclaiming speed of storage blocks.

Determining reclaim information for a storage block based on data length and matching write and delete parameters

Embodiments of the present disclosure provide a method, a device and a computer program product for data processing. The method comprises in response to data associated with an object being stored in the storage location, generating an entry indicating a first association relationship between the data and a storage location. The method further comprises generating metadata of the object, the metadata indicating a second association relationship between the object and the entry. Further, the method comprises in response to the first association relationship between the data and the storage location being changed, updating the entry independent of the metadata. Embodiments of the present disclosure may avoid the huge cost caused by changing metadata multiple times and reduce the system load and the impacts to the front end load.

Method, device and computer program product for data processing

Embodiments of the present disclosure relate to a method, device and computer program product for data stream processing. The method comprises activating, in response to receiving a starting first predefined tag in a first input data stream from a first data source, a duplicate process of a process by duplicating a state of the process to the duplicate process. The method further comprises processing, by the duplicate process, only a data unit before an ending second predefined tag in a second input data stream from a second data source without interruption of processing of the first and second input data streams by the process. The second data source differs from the first data source. The method further comprises creating a snapshot of the duplicate process for recovery of the process due to failure in response to receiving the ending second predefined tag in the second input data stream.

Method, device and computer program product for data stream processing

Techniques provide for rebuilding data. Such techniques involve: obtaining health status information related to a first disk of a storage system, the first disk being divided into a plurality of disk units, and the health status information indicating a failure of a first disk unit of the plurality of disk units; determining a data block stored in the first disk unit based on a mapping between data blocks for the storage system and storage locations; and rebuilding the data block into a second disk of the storage system when maintaining accessibility of other data blocks in other disk units of the first disk than the first disk unit. Accordingly, it is possible to improve the data rebuilding efficiency when a disk fails partly and to continue utilizing the storage space portion in the disk that is not failed, without making the disk be offline temporarily.

Method, device, and computer program product for rebuilding data

The disclosed technology generally describes separating types of data chunks in a copy-on-write/MVCC B+ tree, chunk-based data storage system, and also allocating the sizes of leaf chunks to be smaller than that of other (e.g., internal and root node) chunks. By having leaf chunks separate from node chunks, the probability of having a fully reclaimable (without copying) chunk is increased. Similarly, by having smaller sized leaf chunks relative to node chunks, the probability of having a fully reclaimable (without copying) leaf chunks is increased. The technology thus facilitates more efficient garbage collection.

Efficient capacity management for a data storage system

A method, apparatus and computer program product for managing a storage system are provided. For example, in a method, a tree index of the storage system is obtained, where a leaf node among multiple leaf nodes in the tree index comprising a keyword and an attribute of an object among multiple objects in the storage system. Respective attributes of nodes in a set of the multiple leaf nodes in the tree index are obtained, and the set of the multiple leaf nodes having a same parent node. The tree index is updated based on respective attributes of the leaf nodes in the set of the multiple leaf nodes. With the above example implementations, performance of the index of the storage system can be increased, and further overall response speed of the storage system can be improved. Further, there is provided an apparatus and computer program product for managing a storage system.

Method, apparatus, and computer program product for managing storage system

Embodiments of the present disclosure provide methods, apparatuses, a system and computer program products for managing storage units. According to embodiments of the present disclosure, it is determined whether a first storage unit allocated at a first node is reclaimable, wherein data in the first storage unit is backed up to a second storage unit at a second node. In response to determining that the first storage unit is reclaimable, a condition to be satisfied for reclaiming the second storage unit is determined. A command indicating the condition is sent to the second node, such that the second node reclaims the second storage unit in response to the condition being satisfied. Moreover, in response to the command being sent, the first storage unit is reclaimed. The embodiments of the present disclosure enable timely reclaiming of storage units, thereby improving utilization of the storage space effectively.

Methods, apparatuses, system and computer program products for reclaiming storage units

A system, computer program product, and computer-executable method for use with a distributed storage system comprising a plurality of storage nodes each having attached storage devices, the system, computer program product, and computer-executable method including receiving a request, at a first storage node of the plurality of storage nodes, to store a large portion of data, using at least one of a first type of data chunk and a plurality of a second type of data chunks to store the large portion of data, processing each of the plurality of the second type of data chunks, processing each of the at least one of the first type of data chunk, and returning an acknowledgement to the request.

Efficient erasure coding of large data objects

A computer program product, system, and method for generating and storing a parity chunk using at least a first chunk and a second chunk; detecting at least one unused segment within the first chunk; generating a new chunk as a third chunk; associating one or more used segments from the first chunk to the third chunk; filling an unused segment within the third chunk with new data; calculating a delta parity between the first chunk and the third chunk; and updating the stored parity chunk using the delta parity.

Efficient garbage collection for distributed storage with forward error correction

Logging messages includes providing a logging subsystem that performs logging tasks, providing a log message database that contains log messages that are accessed according to a token value and providing calls to the logging subsystem, where a preprocessor modifies source code to replaces in-line text string parameters with tokens corresponding to the text string parameters. The in-line text string parameters may be log messages. The log message database may be coupled to the logging subsystem and the logging subsystem may access the log messages using the tokens. The preprocessor may determine if an in-line text message already exists in the log message database and, if not, may create a new entry for the in-line text message in the log message database.

Message logging system

In a virtualized environment, provisioned virtual machines may be migrated between different servers as needed and, accordingly, may be clustered according to particular configurations and requirements. The system described herein provides techniques for controlling the provisioning of users' virtual machines among clusters based on determined requirements obtained from users' service level agreements (SLAs) and in accordance with an optimized allocation of resources to service the processes corresponding to the virtual machines. The provisioning of virtual machines according to the system described herein may be dynamically controlled and allow for over-provisioning of virtual machines to a cluster beyond that which would normally be supported at the cluster absent the provisioning control techniques provided by the system described herein. The over-provisioning may be controllable according to specific requirements identified in each user's SLA.

Growing community of inventors

Shanghai, China

Yu Teng