Alibaba Group Holding Limited

Yu Du

Rui Wang

Peng Xu

Yikang Xu

A client device may transmit a data stream including an object (such as a data file or record) to a storage system for storing the object in a storage device. In response to receiving the data stream, the storage system may store or write data of the object into a plurality of logical blocks of the storage device with an end-to-end data protection based at least in part on a comparison of a combination of check codes of a header of the object, the object and padding data with a combination of a plurality of check codes that are generated separately for metadata associated with the object and corresponding parts of the object stored in the plurality of logical blocks. The storage system may further provide an end-to-end data protection for reading data of an object stored in multiple logical blocks of a storage device.

End to end check code protection in storage engine

Methods and systems for in-memory metadata reduction in cloud storage system are provided. According to an aspect, a method comprises receiving a first command to write a data stream to a storage device; writing the data stream into a plurality of fragments having logical addresses corresponding to physical addresses on the storage device; and generating an index for individual fragment of the plurality of fragments, the index indicating information to locate the physical addresses of the individual fragment. Individual records in the individual fragment have a same pre-set logical size and all individual records in the individual fragment are continuous, and the index indicates the information including at least: an offset value of the individual record in the individual fragment; the pre-set logical size of the individual record; and a pre-set physical size of the individual record.

Method and system for in-memory metadata reduction in cloud storage system

A storage engine may obtain one or more object access properties of an object to be received, and determine a type of storage device that is suitable or desirable for storing the object from among different types of storage devices based at least in part on the one or more object access properties of the object to be received. In response to determining the type of storage device, the storage engine may allocate a storage device of such type for the object. The storage engine may then receive the object, and store the object into the allocated storage device.

Using object properties for facilitating object storage

When performing a recycling operation on a storage device, a storage system may use or create a data buffer in the storage device, and designate the data buffer to temporarily store data of data blocks to be recycled in the storage device using direct memory access (DMA) operations that are performed internally in the storage device, without the need of reading the data of the data blocks from the storage device and writing the data into a host memory of the storage system, thereby saving or reducing the consumptions of the communication bandwidth of a communication channel between the storage system and the storage device, and the memory bandwidth of the host memory.

Growing community of inventors

Hangzhou, China

Yu Du