Alibaba Group Holding Limited

Wenhui Yao

Jiaji Zhu

Haiyong Zhang

Junfeng Liu

Chengyu Dong

Jing Lu

Yong Wang

Yang Wu

Yuanyuan Dong

Dongzheng Wu

Feng Cao

Shuo Huang

Junping Wu

Wei Xie

The present invention discloses an intermediate file processing method, client, server, and system, including: receiving, from a first client, a message of writing an intermediate file to a first server; requesting a second server to create cluster information of the intermediate file; after the cluster information is created, receiving the cluster information returned by the second server, wherein the cluster information includes a priority level; and sending the cluster information to the first client and a second client, wherein the intermediate file is uploaded to the first server by the first client, so that the intermediate file is written by the first server according to a local disk load and the priority level of the cluster information, and the intermediate file is read from the first server by the second client according to the cluster information. It is ensures that tasks of jobs of higher priority levels can be processed promptly, jobs of higher priority levels are prevented from being delayed, and disk load balance is maintained.

Intermediate file processing method, client, server, and system

Embodiments of the disclosure provide a data write-in method and apparatus. The data write-in method includes: selecting a target replica server from a plurality of replica servers managed by a metadata server according to remaining capacity of the plurality of replica servers; selecting a write-in disk in the target replica server according to remaining capacity and load of disks managed by the target replica server; and storing write-in data into the write-in disk through the target replica server.

Data write-in method and apparatus in a distributed file system

The disclosure discloses a method for adjusting distribution of partitioned data. Access frequency information of to-be-processed partitioned data is acquired when an adjustment time corresponding to the to-be-processed partitioned data has arrived; revenue data of each of data centers corresponding to the partitioned data is then determined according to the access frequency information and a preset revenue function; and finally, optimal distribution information is determined according to the revenue data of each of the data centers and a quantity of copies, and a position of each of the copies in each of the data centers is adjusted according to the optimal distribution information. Therefore, without even having to provide additional memory or hard disk for storage, this disclosure dynamically optimizes the distribution status of the partitioned data according to access frequencies and characteristics of the partitioned data, thus reducing transmission bandwidth requirements among data centers.

Methods and apparatuses for adjusting the distribution of partitioned data

Provided in the disclosed embodiments are methods and apparatuses for switching service nodes in a distributed storage system, the service nodes comprising a current service node and backup service nodes. The method comprises: monitoring response states of the service nodes to a service request; and if a response state of the current service node is abnormal, stopping communication between the current service node and the backup service nodes, and triggering a switching process for the current service node. In the case of a service timeout, service unavailability, service abnormality, or the like caused by a hardware failure or software defect, logic is checked with a service node that performs logical determination and data statistics collection for multiple factors affecting response states of service nodes, thereby achieving autonomous switching and recovery of the service nodes and improving service availability.

Method and apparatus for switching service nodes in a distributed storage system

The present disclosure provides routing table synchronization methods, apparatuses, and systems. One exemplary method comprises: acquiring modified routing table data if it is detected that a routing table in a default first data management group is modified; and synchronizing the modified routing table data to at least one non-default second data management groups to update a routing table in the at least one non-default second data management group. The first data management group and the at least one second data management group are deployed in a distributed file system. The technical solutions provided in present disclosure can address the technical problem that when any routing table in a distributed file system is modified, routing tables in the system may not be synchronized because routing tables in the distributed file system are static tables.

Routing table synchronization method, apparatus, and system

Growing community of inventors

Hangzhou, China

Wenhui Yao