Emc IP Holding Company LLC

Fenghao Zhang

Junping Zhao

Philippe Armangau

Yu Cao

Jun Tao

Ricky Yuxi Sun

Gang Xie

Yining Si

A method, device, and computer program product for fingerprint based status detection in a distributed processing system is provided. The method comprises: generating and sending, at a root node, an initial fingerprint based on the output message ID to be sent; at a middle node, generating and sending an updated fingerprint based on a received input message ID, an output message ID to be sent and a received fingerprint; sending, at the leaf node, a final updated fingerprint to a tracking task point; and obtaining, at the tracking task point, status information based on the final updated fingerprint. As the method does not generate traffic at each node but only generates tracking traffic at leaf nodes, the approach according to the present disclosure reduces tracking overhead significantly while guaranteeing the reliability of data processing.

Method and device for fingerprint based status detection in a distributed processing system

Embodiments of the present disclosure provide a method and apparatus for distributed data processing. The method comprises: receiving from an upstream node an input message attached with a shared count, the shared count being used for determining a processing status of a root message associated with the input message; processing the input message to generate one or more new messages; allocating to each of the one or more new messages a respective new shared count based on the received shared count; and transmitting the one or more new messages to one or more downstream nodes respectively. Compared with the prior art, the methods and apparatuses for distributed data processing according to the embodiments of the present disclosure can effectively reduce network traffic overheads and the consumed CPU and memory resources, and would be scalable for different topologies of various distributed data processing systems.

Method and apparatus for distributed data processing

A method and device for checking false sharing in deletion of a data block are disclosed. The method includes setting weight bits and a weight reset bit in a mapping pointer (MP) pointing to the data block, wherein the weight reset bit is configured to define whether a weight indicated by the weight bits has been increased; and determining, based on the weight bits and the weight reset bit, whether false sharing occurs. The method determines whether false sharing occurs accurately and with low cost by using a computable formula. All information for such computable formula comes from in-memory MP, with no need to load any extra metadata, for example, with no need to access Block Meta Data. The method also considers a scenario where a weight increase occurs during a weight split and merge with other pointers.

Method and device for checking false sharing in data block deletion using a mapping pointer and weight bits

A method is used in managing deletion of replicas of files. A request is received to delete a replica of a file of a file system. The replica of the file represents a state of the file at a particular prior point in time. The replica is associated with a group including a set of replicas of the file system identified for deletion. Each replica is deleted in the group concurrently by using a delete marker. The delete marker is used for tracking deletion of each replica in the group.

Managing deletion of replicas of files

A method is used in managing deletion of replicas of files. A request to delete a set of replicas of a file of a file system is received. A replica of the file represents a state of the file at a particular prior point in time. A destination replica is identified. A set of file system blocks shared between a replica of the set of replicas and the destination replica is de-allocated by updating metadata of the destination replica. File system blocks associated with the set of replicas are de-allocated. A subset of the file system blocks are de-allocated concurrently.

A method is used in managing processing tasks in storage systems. A set of tasks is received for processing. A type of the set of tasks is identified. Based on the type of the set of tasks, a determination is made as to whether to add data objects associated with the set of tasks to a used object list managed in conjunction of a cache of the system for accessing the data objects associated with the set of tasks. The cache is configured to store the data objects of the storage system. A portion of a memory of the storage system is reserved as the cache.

Growing community of inventors

Chongqing, China

Fenghao Zhang