Lemon Inc.

Xi'an Jiaotong University

Beijing Volcano Engine Technology Co., Ltd.

Douyin Vision Co., Ltd.

Fei Liu

Ping Zhou

Peng Xu

Hui Zhang

Chaohong Hu

Kan Frankie Fan

Longxiao Li

Kyoungryun Bae

Sheng Qiu

Wei Tang

Prathamesh Amritkar

Jinhyuk Kim

Shan Xiao

Ming Lin

Bo Wang

Bo Li

Hao Wang

Yang Liu

Chul Lee

Changyou Xu

Dengcheng Zhu

Fangxing Yu

Chi Zhang

Kyung Ho Kim

Menghan Li

Weiguo Wu

Hyungjin Im

Bhanu Gogineni

Jinwei Xie

Di Xu

Ken Hu

A computing device for verifying data integrity is provided, comprising a memory controller configured to receive a plurality of original data blocks. Each original data block has an associated initial CRC value. The memory controller then segments and recombines the received data blocks into logic blocks, and calculates a new logic block CRC value for each logic block. The logic blocks are transmitted with their respective new logic block CRC values to a storage device, and the logic blocks are written to non-volatile memory of the storage device in a write operation. After the write operation, a combined CRC value is calculated for the logic blocks and a combined CRC value for the original data blocks, and compare the combined CRC values. The memory controller determines whether the combined CRC values match. When they match, the memory controller generates a verification response verifying the integrity of the write operation.

Device and methods for managing the data integrity of read and write operations

A computing system for verifying data integrity is provided, including a host device comprising a memory controller, and a storage device. The memory controller receives original data blocks, each having an associated initial CRC value. The memory controller then segments and recombines the data blocks into logic blocks. The storage device is configured to write the logic blocks to its non-volatile memory of the storage device in the write operation, calculate a new logic block (LB) CRC value for each logic block, and calculate a combined LB CRC value. After the write operation, the memory controller combines duplicated copies of a given CRC value for each of the original data blocks, then combines them together to calculate a combined segments CRC value, and compares the combined segments CRC value to the combined LB CRC value. When the combined CRC values match, a verification response is generated verifying data integrity.

System and methods for managing the data integrity of read and write operations

Systems and methods for accessing block storage devices are provided. In particular, a computing device may receive a write request including an uncompressed data and an uncompressed block address associated with the uncompressed data, generate compressed data by compressing the uncompressed data, determine a plurality of mapping candidates of compressed data blocks in the block storage devices based on the uncompressed block address, select a compressed data block from the plurality of mapping candidates that has sufficient capacity to store the compressed data, write the compressed data to the selected compressed data block, update metadata of the selected compressed data block to link the uncompressed block address to a compressed block address of the selected compressed data block, and write the selected compressed data block back to a respective block storage device of the block storage devices.

Transparent block device compression using hash-table

Methods, systems, and devices for performing an acceleration process by offloading an operation. The system includes a hardware offloading engine that includes a hardware accelerator for performing the acceleration process. The hardware accelerator has a processor configured to receive a hardware offloading command, the hardware offloading command including an operation code, an input pointer, and an output pointer, in which at least one of the input pointer or the output pointer includes a unified data pointer that includes one or more bits of memory for identifying the source location for the input data or the destination location of the output data, parse the operation code, the input pointer, and the output pointer from the hardware offloading command, retrieve the input data based on the input pointer, and execute an offloaded operation on the input data based on the operation code.

Method and system for acceleration or offloading utilizing a unified data pointer

This disclosure describes a non-volatile storage device includes a non-volatile storage and a controller. The controller includes a data evaluator to evaluate each block of data stored in the non-volatile storage, the evaluation being made in accordance with multiple integrity factors; a factor evaluator to provide a weight to each of the multiple integrity factors for each of the multiple blocks of data, aggregate the multiple weighted integrity factors for each of the multiple blocks of data, and identify one of the multiple blocks of data stored in the non-volatile storage for which the aggregated weight integrity factors is highest; and a scanner to perform at least one media scan of the identified block of data stored in the non-volatile storage, e.g., NAND flash memory.

Weighted non-volatile storage media scans

A method is directed to storing data in a non-volatile storage. The method includes writing sequentially at least a main portion of a data record in one or more logic blocks of an append-only storage zone of the non-volatile storage until the one or more first logic blocks are written full. The method also includes generating a first journal including a remainder portion of the first data record and a first journal entry for the remainder portion. A storage device control system includes a non-volatile storage and a controller for the non-volatile storage.

Journal based data storage in block-level storage

A method for adaptive mapping for data compression includes determining an input/output (I/O) request pattern, dynamically switching between a segment mapping mode and a flat hash table mapping mode based on the determined I/O request pattern, updating a shared mapping table for the segment mapping mode and the flat hash table mapping mode, and adjusting an entry of the mapping table based on the determined I/O request pattern and a status of the entry.

Adaptive mapping for transparent block device level compression

A method for accessing blocks of a solid state drive is described. A starting position is received, where the starting position identifies a first block of a contiguous block region within a namespace of the solid state drive. The contiguous block region is to be accessed according to a single input/output operation and the namespace comprises two dimensions of logical address space with respective indices for indexing blocks within a corresponding dimension of the logical address space. A first dimensional identifier that identifies a size of the contiguous block region in a first dimension of the namespace is received. A second dimensional identifier that identifies a size of the contiguous block region in a second dimension of the namespace is received. Blocks of the contiguous block region are accessed in response to the single input/output operation according to the starting position, the first dimensional identifier, and the second dimensional identifier.

Multi-dimensional solid state drive block access

The present disclosure provides a data processing method, a device, a computer apparatus and a storage medium, wherein the method includes: in response to a target disk receiving at least one write request within a preset time period, determining a size threshold value for classifying a data update type according to a size of write data respectively indicated by each write request; determining a data update type corresponding to each write request according to a size of each write request and the size threshold value; dividing write data of the write request to obtain a data block according to a preset data block size, and caching the data block in a cache region of the target disk corresponding to the data update type, the target disk has multiple types of cache regions configured therein, different cache regions are configured to support caching of data with different update frequencies.

Data processing method, device, computer apparatus and storage medium

The present disclosure describes techniques of metadata management for transparent block level compression. A first area may be created in a backend solid state drive. The first area may comprise a plurality of entries. The plurality of entries may be indexed by addresses of a plurality of blocks of uncompressed data. Each of the plurality of entries comprises a first part configured to store metadata and a second part configured to store compressed data. Each of the plurality blocks of uncompressed data may be compressed individually to generate a plurality of compressed blocks. Metadata and at least a portion of compressed data associated with each of the plurality of compressed blocks may be stored in one of the plurality of entries based on an address of a corresponding block of uncompressed data. A second area may be created in the backend solid state drive for storing the rest of the compressed data.

Metadata management for transparent block level compression

Systems and methods for space allocation for block device compression are provided. In particular, a computing device may receive an allocation request to write the compressed data, select a range list adequate for serving the allocation request from a plurality of range list, dequeue a range entry from the selected range list to allocate free space for the compressed data, and allocate the free space corresponding to the range entry to the compressed data to serve the allocation request.

Space manager for transparent block device compression

A system and method are described to efficiently allocate memory space with low latency overhead by allocating blocks of non-volatile memory on a storage device according to a tree data structure comprising a plurality of counter sets, each counter set including one or a plurality of counters indicating numbers of unallocated blocks of memory space within the non-volatile memory.

System and method for allocating memory space

Described are examples for storing data on a storage device, including storing, in a live write stream cache, one or more logical blocks (LBs) corresponding to a data segment, writing, for each LB in the data segment, a cache element of a cache entry that points to the LB in the live write stream cache, where the cache entry includes multiple cache elements corresponding to the multiple LBs of the data segment, writing, for the cache entry, a table entry in a mapping table that points to the cache entry, and when a storage policy is triggered for the cache entry, writing the multiple LBs, pointed to by each cache element of the cache entry, to a stream for storing as contiguous LBs on the storage device, and updating the table entry to point to a physical address of a first LB of the contiguous LBs on the storage device.

Growing community of inventors

Los Angeles, CA, United States of America

Fei Liu