Huawei Technologies Co., Limited

Tencent Technology (shenzhen) Company Limited

Qun Yu

Jun Xu

Yuangang Wang

Guanyu Zhu

Licheng Chen

Yuanyuan Zhang

Haiyan Liu

Xiao Hua

Lei Xiong

Wengsan Cheong

Fei Zhan

Journal cover

Notebook cover

A storage system includes a management node and a plurality of storage nodes forming a redundant array of independent disks (RAID). When the management node determines that not all data in an entire stripe is updated based on a received write request, the management node sends an update data chunk obtained from to-be-written data to a corresponding storage node. The storage node does not directly update, based on the received update data chunk, a data block stored in a storage device of the storage node, but store the update data chunk into a non-volatile memories (NVM) cache of the storage node and send the update data chunk to another storage node for backup. According to the data updating method, write amplification problems caused in a stripe update process can be reduced, thereby improving update performance of the storage system.

Data updating technology

A storage system includes a management node and a plurality of storage nodes forming a redundant array of independent disks (RAID). When the management node determines that not all data in an entire stripe is updated based on a received write request, the management node sends update data chunk obtained from to-be-written data to corresponding storage node. The storage node do not directly update, based on the received update data chunks, data block stored in storage device of the storage node, but store the update data chunk into non-volatile memories (NVM) cache of the storage node and send the update data chunk to another storage node to backup. According to the data updating method, write amplification problems caused in a stripe update process can be reduced, thereby improving update performance of the storage system.

An access request processing method is performed by a computer device that includes a processor, a dynamic random-access memory (DRAM), and a non-volatile memory (NVM). When receiving a write request, the processor may identify an object cache page according to the write request. The processor obtains the to-be-written data from a buffer according to a buffer pointer in the write request, the to-be-written data including a new data chunk to be written into the object cache page. The processor then inserts a new data node into a log chain of the object cache page, where the NVM stores data representing the log chain of the object cache page. The new data node includes information regarding the new data chunk of the object cache page. The computer device provided in this application can reduce system overheads while protecting data consistency.

Access request processing method and apparatus, and computer device

An access request processing apparatus comprises, a processor determines an object cache page according to a write request when receiving the write request. After determining that the NVM stores a log chain of the object cache page, the processor inserts, into the log chain of the object cache page, a second data node recording information about a second log data chunk. The log chain already includes a first data node recording information about the first log data chunk. The second log data chunk is at least partial to-be-written data of the write request. Then, the processor sets, in the first data node, data that is in the first log data chunk and that overlaps the second log data chunk to invalid data.

Access request processing method and apparatus, and computer system

Embodiments of this application disclose a method for retrieving similar videos performed at a computing device. The computing device obtains video information of a video for which similar videos are to be retrieved, the video information including a video tag and a video title, and trains the video information by using a preset text depth representation model, to convert the video information into a word vector. After selecting, from a video library according to a preset knowledge graph, videos matching the video information, to obtain a first candidate video set, the computing device screens, in the video library, videos similar to the video information according to the word vector, to obtain a second candidate video set and then determines a similar video for the video information from the first candidate video set and the second candidate video set.

Method and apparatus for retrieving similar video and storage medium

An access request processing method and apparatus, and a computer device are disclosed. The computer device includes a processor, a dynamic random-access memory (DRAM), and a non-volatile memory (NVM). When receiving a write request, the processor may identify an object cache page according to the write request. The processor obtains the to-be-written data from a buffer according to a buffer pointer in the write request, the to-be-written data including a new data chunk to be written into the object cache page. The processor then inserts a new data node into a log chain of the object cache page, where the NVM stores data representing the log chain of the object cache page. The new data node includes information regarding the new data chunk of the object cache page. The computer device provided in this application can reduce system overheads while protecting data consistency.

An access request processing method and apparatus, and a computer system is disclosed. The computer system includes a processor and a non-volatile memory (NVM). When receiving a write request, the processor determines an object cache page according to the write request. After determining that the NVM stores a log chain of the object cache page, the processor inserts, into the log chain of the object cache page, a second data node recording information about a second log data chunk. The log chain already includes a first data node recording information about the first log data chunk. The second log data chunk is at least partial to-be-written data of the write request. Then, the processor sets, in the first data node, data that is in the first log data chunk and that overlaps the second log data chunk to invalid data.

A file data access method and a computer system, where the method includes accessing a page global directory (PGD) of the process using PGD space when accessing first file data by a process, determining, based on access to the PGD and according to a first virtual address of the first file data in file system space, a first PGD entry in the PGD, linking a file page table of the process to the first PGD entry, where the file page table points to a physical address of the file data such that a processor retrieves a first physical address of the first file data in a memory according to the first virtual address using the PGD and the file page table, and accessing the first file data according to the first physical address.

Time-restricted access to file data

A data processing method and apparatus are provided. A system includes a CPU pool and a storage pool, a mapper node and a reducer node respectively run on different CPUs in the CPU pool, and a remote storage area shared by the mapper node and the reducer node is delimited in the storage pool. In this method, the mapper node executes a map task, and stores a data segment obtained by executing the map task, into the remote storage area; and the reducer node directly obtains a data segment to be processed by the reducer node from the remote storage area, and executes a reduce task on the data segment. The method omits some actions in the prior art, and shortens a time for executing the Map/Reduce task.

Data processing method and apparatus, and system

A file access method and apparatus, and a storage system are provided. After receiving a file access request including a file identifier, first physical address space is accessed according to first virtual address space and a first mapping relationship between the first virtual address space and the first physical address space storing a file system. After obtaining, from the first physical address space, an index node of an object file indicated by the file identifier, a file page table is obtained according to information included in the index node, where the file page table records second physical address space of the object file. Then, second virtual address space is allocated to the object file. After establishing a second mapping relationship between the second physical address space and the second virtual address space, the object file in the second physical address space is accessed according to the second virtual address space.

File access method and apparatus, and storage system

A file access method and apparatus, and a storage system are provided. After receiving a file access request from a process, a first physical address space is accessed according to a preset first virtual address space and a preset first mapping relationship between the first virtual address space and the first physical address space, where the first physical address space stores a file system. After obtaining an index node of a target file from the first physical address space according to a file identifier of the target file carried in the file access request, a file page table of the target file is obtained according to file page table information. The file page table records a second physical address space in the first physical address space. The target file is accessed according to the second physical address space.

Growing community of inventors

Beijing, China

Qun Yu