Netapp, Inc.

Rajesh Khandelwal

Ananthan Subramanian

Sandeep Yadav

Kevin Daniel Varghese

Rohit Shankar Singh

Parag Sarfare

Suhas Urkude

Jayalakshmi Pattabiraman

Blake H Lewis

Subramaniam Periyagaram

Nikhil Mattankot

James Robert Morefield

Deekshith Belchapada

Satya Suresh Chouta Naga Veera

Dnyaneshwar Nagorao Pawar

Kiran Nenmeli Srinivasan

Alok Sharma

Kartik Rathnakar

Mukul Sharma

David A Grunwald

Venkateswarlu Tella

Venkateswarlu Telia

Jungsook Yang

Methods and systems for co-locating journaling and data storage are provided. Separate journal and volume partitions may be maintained within each logical storage unit (e.g., Logical Unit Number (LUN)) of a distributed storage system. Journaling of metadata associated with write requests received from one or more clients may be distributed by identifying a destination logical storage unit to which data associated with a given write request is to be stored and causing the data and metadata to be persisted to disk by journaling the metadata and the data to respective portions of an active log within the journal partition of the destination logical storage unit. By using the same logical storage unit for both journaling of write requests and writing the data associated with such write requests, the bottleneck due to there being only a single device or storage unit handling all metadata for all write requests can be avoided.

Co-located journaling and data storage for write requests

A method and system for co-locating journaling and data storage based on write requests. A write request that includes metadata and data is received from a client. A logical storage unit for storing the metadata and the data is identified. The logical storage unit is divided into a journal partition and a volume partition. The journal partition includes a first log and a second log. Which of the first log and the second log is an active log and which of the first log and the second log is an inactive log are identified. The metadata is recorded in a first location in the active log and the data is recorded in a second location in the active log during a single I/O operation. A reply is sent to the client after the metadata and the data are recorded in the journal partition.

A first plurality of block identifiers is sorted based, at least in part, on a measure of spatial locality. A second plurality of block identifiers is sorted based, at least in part, on the measure of spatial locality. At least the first plurality of block identifiers and the second plurality of block identifiers are incrementally merged into a third plurality of block identifiers based, at least in part, on the measure of spatial locality. A block of data corresponding to metadata associated with a plurality of block identifiers of the third plurality of block identifiers is updated.

Efficient modification of storage system metadata

One or more techniques and/or computing devices are provided for inline deduplication. For example, a checksum hash table and/or a block number hash table may be maintained within memory (e.g., a storage controller may maintain the hash tables in-core). The checksum hash table may be utilized for inline deduplication to identify potential donor blocks that may comprise the same data as an incoming storage operation. Data within an in-core buffer cache is eligible as potential donor blocks so that inline deduplication may be performed using data from the in-core buffer cache, which may mitigate disk access to underlying storage for which the in-core buffer cache is used for caching. The block number hash table may be used for updating or removing entries from the hash tables, such as for blocks that are no longer eligible as potential donor blocks (e.g., deleted blocks, blocks evicted from the in-core buffer cache, etc.).

Inline deduplication

Storage systems track free blocks using various data structures and maps. For instance, free block maps may contain data blocks with values that indicate whether a block is free or not. When an operation results in a block being freed, the relevant data block in the maps must be written during an I/O operation to update the value. Large numbers of updates my occur after an operation that frees a large numbers of blocks, which can lead to performance degradation. Accordingly, disclosed are systems and methods for deferring updating of free block data tracking structures using logs.

Resource reservation for storage system metadata updates

A technique for efficiently scanning various datasets in a storage system to generate metadata for shared data blocks is provided. In one embodiment, data containers in an active file system of a storage object are first scanned. Remaining images (e.g., snapshots of the file system) are arranged in a chronological order based on the age of such images in the storage object. In some instances, to perform a metadata scan of a given image, the storage server compares each data container in the given image to a corresponding data container in a prior image (from the chronological order) to identify any non-identical data blocks (i.e., blocks that are not common between the data containers of the two images) in each data container. The metadata scanner then generates metadata only for the non-identical data blocks in each data container and skips metadata generation for any remaining data blocks.

Efficient file system scan for shared data blocks

Growing community of inventors

Sunnyvale, CA, United States of America

Rajesh Khandelwal