Emc IP Holding Company LLC

Emc Corporation

Yining Si

Philippe Armangau

Christopher Alan Seibel

Ye Zhang

Jean-Pierre Bono

Xiangping Chen

William C Davenport

Ivan Bassov

Yubing Wang

Qi Mao

Miles Aram De Forest

Kamakshi Viswanadha

Alexander Daniel

Junping Zhao

Ahsan Rashid

Marc A De Souter

Alexander S Mathews

Ajay Karri

Chao Zheng

Ivan Basov

Michael C Brundage

Bruce E Caram

Srinivasa Rao Vempati

Qin Tao

Yaming Kuang

Pranit Sethi

Maher Kachmar

Fenghao Zhang

Gang Xie

Marshall Hansi Wu

Yan Xu

Piers Yu

Alex Zhongbing Yang

Steve Zhuxiong Ai

Pu Wang

A method for file system reorganization in the presence of inline compression includes obtaining a virtual block pointer for an original compressed segment to be reorganized, the original compressed segment comprising compressed allocation units of data stored in a storage system, wherein the virtual block pointer comprises an extent list identifying the compressed allocation units in the original compressed segment and a pointer to where the original compressed segment is stored; copying only the referenced compressed allocation units in the original compressed segment to a new compressed segment in a substantially contiguous manner; updating the extent list to identify the referenced compressed allocation units in the new compressed segment, and the pointer to where the new compressed segment is stored; and freeing the original compressed segment.

File system reorganization in the presence of inline compression

A technique for performing writes in a storage system that supports data compression tests incoming writes to determine whether the amount of data to be written is less than a predetermined threshold. If so, the storage system avoids compression and stores the amount of data as received without compressing it. If not, the storage system performs compression and stores the incoming data as a compressed extent.

Inline compression with small-write compression avoidance

A technique for managing storage of compressed data includes generating and enforcing a minimum slot size requirement. The minimum slot size is based at least in part on collected performance metrics that indicate a degree of compressibility of data received, compressed, and written by the data storage system. As new data arrive, the new data are compressed and stored in slots at least as big as the minimum slot size, in many cases effecting an over-allocation of storage space and improving the likelihood that subsequent overwrites will fit into existing slots.

Adaptively over-allocating storage space for compressed data

Methods and apparatus are provided for reporting fragment filling in storage systems. An exemplary method comprises obtaining at least one compressed allocation unit of data in a storage system; reserving space for the at least one compressed allocation unit in a compressed segment based on a greater of (i) a size of the at least one compressed allocation unit, and (ii) a minimum target fragment length specified for at least one write operation class; and writing the at least one compressed allocation unit to the reserved space. An attempt is optionally first made to fill the at least one compressed allocation unit of data in a free extent and/or a hidden fragment of at least one compressed segment prior to reserving the space. The reserving the space is optionally only performed if the attempt to fill the at least one compressed allocation unit of data is not successful.

Fragment filling for storage system with in-line compression

A technique for performing in-line compression includes receiving data into a data log that temporarily holds the data and aggregating the data into batches, where each batch includes multiple blocks of received data. For each batch of data, a storage system performs a compression operation, which proceeds block-by-block, compressing each block and comparing a total compressed size of all blocks compressed so far against a budget. The storage system increments the budget for successive blocks, such that a per-block budget is greater for a first block in the batch than it is for a last block in the batch, thus allowing earlier blocks to meet budget even if they are relatively incompressible.

Compressing data in line using weighted compression budgets

A technique for storing data selectively tags write requests received by a data storage system based on whether each write request is directed to a file having a file type on an exclusion list, thereby producing a set of tagged write requests directed to a file type on the exclusion list and a set of untagged write requests directed to a file type not on the exclusion list. The method further includes persistently storing tagged data specified by the set of tagged write requests without first compressing the tagged data, and persistently storing untagged data specified by the set of untagged write requests after compressing the untagged data.

Write tagging for selective avoidance of inline compression

A method, computer program product, and computer system for receiving, at a computing device, an I/O request directed to a compressed data portion of a file system. A field associated with the compressed data portion may be determined to be invalid. An error may be generated in response to determining that the field associated with the compressed data portion is invalid. The file system may be maintained in an online status while the compressed data portion is invalid.

Fencing for zipheader corruption for inline compression feature system and method

A storage technique includes selectively tagging a set of write requests by applying tags to write requests that are pursuant to copy operations and not applying tags to at least some write requests that are not pursuant to copy operations. For each of the set of write requests, the technique further includes performing a decision operation configured to (i) direct the write request down a first processing path that supports inline deduplication in response to the write request satisfying a set of criteria and (ii) direct the write request down a second processing path that does not support inline deduplication in response to the write request failing to satisfy the set of criteria. The set of criteria include whether the write request has a tag applied, such that the decision operation preferentially directs write requests down the first processing path in response to the write requests being tagged.

Tagging write requests to avoid data-log bypass and promote inline deduplication during copies

A method is used in managing deletion of replicas of files. A request to delete a set of replicas of a file of a file system is received. A replica of the file represents a state of the file at a particular prior point in time. A destination replica is identified. A set of file system blocks shared between a replica of the set of replicas and the destination replica is de-allocated by updating metadata of the destination replica. File system blocks associated with the set of replicas are de-allocated. A subset of the file system blocks are de-allocated concurrently.

Managing deletion of replicas of files

Managing a circular transaction log in a data storage system includes receiving requests specifying data to be written to a data object served by the data storage system, and adding log entries to the log containing data specified by the requests at or after a tail of the log and before a head of the log. In response to a trigger event, a size of the log is dynamically increased by inserting additional storage space into the log at or after the tail and before the head, to account for differences between the rate at which entries are added to the log and the rate at which entries are flushed from the log. The disclosed technique may further operate to find the slot position in which each log entry is stored, based on the identifier of the log entry, after inserting the additional storage space into the log.

Dynamically adjustable transaction log

A method is used in managing data relocation in storage systems. Metadata of a slice of a storage tier in a data storage system is evaluated for migrating the slice from the storage tier to another storage tier. The data storage system includes a first storage tier and a second storage tier configured such that performance characteristics associated with the first storage tier is superior to the second storage tier. Based on the evaluation, relocation of the slice of the storage tier is effected. The metadata of the slice indicates whether the slice includes user data.

Managing relocation of slices in storage systems

A method is used in managing data relocation in storage systems. A request is received to write to a slice of storage tier in a data storage system where the slice is being relocated. The relocation of the slice includes copying data of the slice to another slice in the data storage system. Location within the slice at which the request to write to the slice has been received is determined. Based on the determination, relocation of the slice of the storage tier is effected.

Managing data relocation in storage systems

A technique for storing data in a log receives a set of sequential requests, each request specifying data to be written to a data object served by a data storage system. The data specified by respective ones of the requests is placed into respective lockable units of storage for the log, such that the data of no two sequential requests are stored within the same lockable unit. Each lockable unit is locked for access by a writer when being written, and unlocked when not being written, and is configured to store data specified by multiple requests. Completion of each of the requests is acknowledged in response to placing the data specified by the respective request in one of the lockable units of storage. The log may also be partitioned, with individual partitions being used to store data specified by requests having specific request sizes.

Transaction logging using round-robin block allocation and I/O size based partitions

A method is used in managing storage space in storage systems. Metadata of a slice of a storage pool in a data storage system is evaluated for returning the slice as free storage to the storage pool. Based on the evaluation, returning of the slice to the storage pool is affected. The metadata of the slice indicates whether the slice includes user data.

Managing storage space in storage systems

A method, computer program product, and computer system for instantiating, by a computing device, a slice-object associated with a slice when the slice-object is accessed. The slice-object is released to a slice object cache when accessing is complete. It is determined whether the slice is accessed within a threshold period of time. If the slice is accessed within the threshold period of time, the slice-object is retrieved from the slice-object cache. If the slice is not accessed within the threshold period of time, memory used for the slice-object is released.

Growing community of inventors

Shrewsbury, MA, United States of America

Yining Si