Huawei Technologies Co., Limited

Future Wei Technologies, Inc.

Huawei Cloud Computing Technologies Co., Ltd.

Daohui Wang

Wei Zhang

Jiongjiong Gu

Xiaoyong Min

Wei Fan

Yongqiang Zeng

Zhile Zhang

Xian Liu

Chi Song

Yu Yin

James Prescott Hughes

Yan Wang

Qiang Yu

Mingchang Wei

Ajit Verma

John Plocher

Deping Yang

Xuyou Liu

Rong Rao

Tonglei Wang

Yun Zhan

Zhihui Chen

Zhenxing Dai

Shengbing Shi

A data processing method in a storage system and the storage system, and relates to the field of data storage technologies. A client first sends a data block in a stripe and metadata of the data block to a data storage node. The client then sends the metadata of the data block and a parity block to a parity storage node, to back up the metadata of the data block on the parity storage node. When the metadata of the data block is successfully backed up, the data block is written into a corresponding storage device based on a storage location indicated by the metadata. This reduces write operation time.

Data processing method and apparatus in storage system, and storage system

A data storage method implemented by a storage node in a distributed storage system includes receiving first data of a first strip in a first stripe, wherein the first stripe comprises a first plurality of strips, receiving second data of a second strip in a second stripe, wherein the second strip comprises a second plurality of strips, wherein a first logical address of the first data is the same as a second logical address of the second data, and generating a record indicating that the first data reaches the storage node before the second data.

Data storage method and apparatus in distributed storage system, and computer program product

Embodiments of the present invention provide a cluster system with calculation and storage converged, including: a resource server group, where the resource server group includes at least two resource servers, each resource server in the resource server group has at least one of: a calculation resource and a storage resource, at least one resource server in the resource server group has a calculation resource and a storage resource, and the storage resource includes a persistent disk and a cache that corresponds to the persistent disk; and a storage resource control module, configured to construct a shared storage resource pool by using storage resources in the resource server group, so as to provide a storage service, where storage resources included in each storage resource pool are from at least two resource servers in the resource server group.

Cluster system with calculation and storage converged

A distributed storage and replication system includes a MDC module, multiple IO routing modules, and multiple OSD nodes. The MDC module is adapted to configure at least two partition, the IO routing module is adapted to route an IO request to an OSD node, and the OSD node is adapted to execute storage of data corresponding to the IO request. The MDC is configured to determine a faulty OSD node, update a partition view of a partition group that includes a partition on the faulty OSD node, and send an updating notification to a primary OSD node in the updated partition view. The primary OSD node is adapted to process replication of the data corresponding to the IO request. According to embodiments of the present disclosure, processing performance, fault tolerance, and availability of consistency replication are improved.

Distributed storage and replication system and method

Embodiments of the present invention provide a cluster system with calculation and storage converged, including a resource server group comprising at least two resource servers. Each resource server in the resource server group has at least one of a calculation resource and a storage resource. At least one resource server in the resource server group has a calculation resource and a storage resource, and the storage resource includes a persistent disk and a cache that corresponds to the persistent disk. A storage resource control module is configured to construct a shared storage resource pool by using storage resources in the resource server group, so as to provide a storage service, where storage resources included in each storage resource pool are from at least two resource servers in the resource server group.

A data storage method is used to improve storage consistency of a distributed storage system. The method includes: a primary storage node performs EC coding on a to-be-stored data segment to obtain a target EC stripe; determines in a storage node group to which the primary storage node belongs, m+k target storage nodes used to store m+k target EC blocks of the target EC stripe; sends a preparation message to the target storage nodes; receives a response message sent by a target storage node; and sends an execution message to the target storage nodes to instruct the target storage nodes to write target EC blocks that are in preparation logs.

Data storage method, data recovery method, related apparatus, and system

Embodiments of the present invention provide a method and a system for sharing a storage resource, where storage resources of a hard disk and a storage array are divided into multiple storage partitions and the multiple storage partitions form a shared storage resource pool, a read-write control module is allocated to each storage partition, and global partition information is generated to record a correspondence between each storage partition in the shared storage resource pool and the read-write control module. The embodiments of the present invention implement fast and simple integration of heterogeneous storage resources, and therefore can efficiently use various storage resources, save cost, and avoid a resource waste.

Method and system for pooling, partitioning, and sharing network storage resources

A method for data reconstruction includes: obtaining a data migration list, where the data migration list indicates a partition with data to be migrated on a storage node with data to be migrated; generating a data migration task according to the data migration list and routing table information, where the routing table information indicates correspondence between a storage node in a cluster and a partition; and concurrently executing data migration tasks of a storage node in the cluster to reconstruct data. In addition, a device for data reconstruction and a storage system are further provided accordingly.

Method, device, and system for data reconstruction

A data recovery method to improve storage consistency of a distributed storage system includes: receiving a recovery message from a storage node after a fault recovery of the faulty storage node; obtaining N lost erasure coding (EC) blocks of the faulty storage node through computation according to EC blocks stored by non-faulty storage nodes; sequentially sending N lost EC blocks to the faulty storage node for storage. In the process of sequentially sending the N lost EC blocks to the faulty storage node, when it is determined that for a first EC block that is to be stored by the faulty storage node, a second EC block having a same key value as the first EC block exists in the lost EC blocks, after the faulty storage node stores the second EC block, the first EC block is sent to the faulty storage node to store.

The present disclosure relates to the field of information technology, and in particular, to a block storage-based data processing method, apparatus, and system. The block storage-based data processing method provided in embodiments of the present disclosure is applied in a system including at least two storage nodes, each storage node including a CPU, a cache medium, and a non-volatile storage medium, and the cache medium in all the storage nodes forming a cache pool. According to the method, after receiving a data operation request sent by a client, a service processing node sends the data operation request to a corresponding storage node in the system according to a logical address carried in the data operation request, so that the data operation request is processed in the cache medium of the storage node under control of the CPU of the storage node.

Block storage-based data processing methods, apparatus, and systems

System and methods for storage object distribution using a universal distributed storage (UDS) system. An embodiment UDS includes server nodes and a header server in communication with the server nodes. The header server is configured to divide an object received from a client into chunks, to assign a key to each of the chunks, to calculate a hash value for each of the keys to identify which of the server nodes will store each of the chunks, and to instruct one of the server nodes to store each of the chunks in accordance with the hash value calculated. In an embodiment, the server nodes include a single central processing unit, a single storage device, a primary network interface, and a redundant network interface.

Self-organizing disk (SoD)

The storage method according to the present invention includes: partitioning raw data into several data blocks; calculating, according to a preset mapping algorithm, three consecutive virtual storage partitions to which a first data block is mapped, where a virtual storage partition is a partition obtained by dividing a virtual storage area, each of the virtual storage partitions is mapped to one physical disk, physical disks have consecutive sequence numbers, and sequence numbers of three physical disks that are mapped to the three consecutive virtual storage partitions are in an arithmetic sequence with a non-zero common difference; and storing, according to the calculated three consecutive virtual storage partitions and the three physical disks that are mapped to the three consecutive virtual storage partitions, the first data block and a first copy and a second copy of the first data block in the three physical disks in sequence.

Method and apparatus for storing data using a data mapping algorithm

The present disclosure provides a storage system for data read and write. One embodiment of the storage system includes an origination device that is configured to receive a request for a logical block addressing-based operation on a volume, convert the logical block addressing-based operation request into a key addressing-based operation request carrying a key corresponding to data to be operated, and send the key addressing-based operation request to a routing library; the routing library is configured to receive the key addressing-based operation request, hash the key corresponding to the data to be operated, determine that a storage node taking charge of a hash region in which the hashed key is located is the master storage node, send the key addressing-based operation request to the master storage node of the data to be operated.

Growing community of inventors

Shenzhen, China

Daohui Wang