Huawei Technologies Co., Limited

Other

Pak-Ching Lee

Xi Zhang

Qun Huang

Cheng He

Yong Huang

Gong Zhang

Lujia Pan

Jiajin Zhang

Runhui Li

Matt M T Yiu

Shakeel Salamat Ullah

Hoi-Wan Chan

Lu Tang

Ng-Kwok Sing

A distributed storage system includes one or multiple proxies, a metadata store, multiple agents, and multiple data repositories, which provisions storage space within a cloud or a cloud region from one or multiple cloud storage services. A proxy encodes data objects to generate coded chunks for fault tolerance and security, stores coded chunks in data repositories via agents, and persists the states and metadata of data objects in the metadata store. An agent manages the coded chunks in one or multiple data repositories and encodes coded chunks during data repair. The system utilizes data processing and management methods for improved data repair and management, including (i) hierarchical data repair for reducing both the cross-cloud network bandwidth usage and the data repair time, (ii) various data management configurations across objects, and (iii) extensions for new data management mechanisms and new cloud storage services.

Methods and apparatus for data processing and management in erasure-coded multi-cloud storage systems

In a data processing method, a worker node in a distributed data processing system receives first data from an upstream worker node. The first data has been stored in a buffer of the upstream worker node. The worker node sends a first portion of the first data to a persistent storage device of the distributed data processing system for persistent backup, and performs computational processing on the first data to generate second data. Prior to completing performing computational processing on the first data, the worker node sends acknowledgement information to the upstream worker node to instruct the upstream node to delete the first data from the buffer of the upstream worker node. The worker node then sends the second data to a downstream worker node in the distributed data processing system for further processing by the downstream worker node.

Data processing method and apparatus

One example packet processing device includes a buffer, and the packet processing device obtains a to-be-measured packet. In response to determining that occupied storage space in the buffer is less than a preset threshold, the packet processing device reads the to-be-measured information from the buffer, and modifies, based on the to-be-measured information and a first algorithm, a pieces of data in first measurement data corresponding to the to-be-measured packet, where a is a positive integer. In response to determining that occupied storage space in the buffer is greater than or equal to a preset threshold, the packet processing device modifies, based on to-be-measured information and a second algorithm, w pieces of data in second measurement data corresponding to the to-be-measured packet, where w is a positive integer, and w is less than a.

Traffic measurement method, device, and system

A data processing method is disclosed, and the method includes: encoding a data chunk of a predetermined size, to generate an error-correcting data chunk corresponding to the data chunk, where the data chunk includes a data object, and the data object includes a key, a value, and metadata; and generating a data chunk index and a data object index, where the data chunk index is used to retrieve the data chunk and the error-correcting data chunk corresponding to the data chunk, the data object index is used to retrieve the data object in the data chunk, and each data object index is used to retrieve a unique data object.

Data processing method, system, and apparatus

A method, an apparatus, and a system for identifying an abnormal IP data stream, which are used to improve identification accuracy. The method provided by the embodiments of the present invention includes: receiving Y elements sent by a data collection node; mapping the Y elements to N buckets; acquiring a bucket in the N buckets as a target bucket; acquiring r upper traffic limits of a first object in r buckets within the current time interval, the first object is any object mapped to the target bucket; and identifying, according to a preset abnormal object type and the r upper traffic limits within the current time interval, whether the first object is an abnormal object, where the preset abnormal object type is a heavy hitter or a heavy changer.

Method, apparatus, and system for identifying abnormal IP data stream

Growing community of inventors

Hong Kong, China

Pak-Ching Lee