Netapp, Inc.

Mark Walter Storer

Shankar Pasupathy

Timothy Clayton Bisson

Gokul Soundararajan

Lakshmi Narayanan Bairavasundaram

Jiri Schindler

Yiying Zhang

Michael G Wilson

Examples are disclosed for identifying duplicated media content in a plurality of media files. In some examples, according to a media file format, media content sequences may be located and duplicated media content sequences identified. For these examples, at least a portion of the identified duplicated media content sequences may then be deleted or not stored at a storage system. Other examples are described and claimed.

Techniques for deduplication of media content

Techniques for a data storage cluster and a method for maintaining and updating reliability data and reducing data communication between nodes, are disclosed herein. Each data object is written to a single data zone on a data node within the data storage cluster. Each data object includes one or more data chunks, and the data chunks of a data object are written to a data node in an append-only log format. When parity is determined for a reliability group including the data zone, there is no need to transmit data from other data nodes where the rest of data zones of the reliability group reside. Thus, inter-node data communication for determining reliability data is reduced.

Methods for managing storage in a data storage cluster with distributed zones based on parity values and devices thereof

Methods and apparatuses for updating members of a data storage reliability group are provided. In one exemplary method, a reliability group includes a data zone in a first storage node and a checksum zone in a second data storage node. The method includes updating a version counter associated with the data zone in response to destaging a data object from a staging area of the data zone to a store area of the data zone without synchronizing the destaging with the state of the checksum zone. The method further includes transmitting, from the data zone to the checksum zone, an update message indicating completion of the destaging of the data object, wherein the update message includes a current value of the version counter.

Decoupled reliability groups

A storage system provides highly flexible data layouts that can be tailored based on reliability considerations. The system allocates reliability values to logical containers at an upper logical level of the system based, for example, on objectives established by reliability SLOs. Based on the reliability value, the system identifies a specific parity group from a lower physical storage level of the system for storing data corresponding to the logical container. After selecting a parity group, the system allocates the data to physical storage blocks within the parity group. In embodiments, the system attaches the reliability value information to the parity group and the physical storage units storing the data. In this manner, the underlying physical layer has a semantic understanding of reliability considerations related to the data stored at the logical level. Based on this semantic understanding, the system has the capability to prioritize data operations on the physical storage units according to the reliability values attached to the parity groups.

Reliability based data allocation and recovery in a storage system

Described herein is a system and method for an efficient cache warm-up. The system and method may copy data blocks from a primary storage device to a cache memory device. The system and method may identify a subset of data blocks stored on the primary storage device as candidate data blocks for copying to the cache memory device during a cache warm-up period. A cost effectiveness for copying the candidate data blocks to the cache memory device may be determined. In some embodiments, the cost effectiveness may be calculated based on one or more latency values associated with the primary storage device and the cache memory device. The candidate data blocks may be copied to the cache memory device based on the cost effectiveness.

System and method for an efficient cache warm-up

System and method for processing cache metadata from a plurality of cache devices. One or more storage systems may store data blocks to a set of storage devices, each storage system hosting and operating one or more cache devices for caching data blocks of the storage devices. Each cache device may host and implement an agent engine that collects and stores cache metadata for the cache device, the cache metadata describing caching operations of the cache device. A monitor engine receives and aggregates 'raw cache metadata' from each of a plurality of cache devices and processes the raw cache metadata to produce processed cache metadata comprising normalized cache metadata (based on activity rate of a cache device), weighted cache metadata, or normalized weighted cache metadata. The processed cache metadata is used by a cache prediction engine to select data blocks for caching to one or more of the cache devices.

Caching data using multiple cache devices

Using append only log format in data storage cluster with distributed zones for determining parity of reliability groups

Techniques for a data storage cluster and a method for deduplicating data in the data storage cluster in a scalable manner, by (among other things) using an epoch-based global chunk data structure, are disclosed herein. A global chunk data structure for an epoch is distributed and maintained at a plurality of metadata nodes within the data storage cluster. Fingerprints and identifiers of data chunks are written to the cluster after a particular epoch are written to delta chunk data structures stored in different metadata nodes of the cluster. When the data storage cluster advances to the next epoch, the global chunk data structure is updated using the delta chunk data structures. At any given time, data deduplication in the data storage cluster can be conducted based on the global chunk data structure for the current epoch.

Growing community of inventors

Walnut Creek, CA, United States of America

Mark Walter Storer