Intertrust Technologies Corporation

Nimble Storage, Inc.

Hewlett Packard Enterprise Development LP

Emc Corporation

Data Domain, Inc.

Emc IP Holding Company LLC

Umesh Maheshwari

W Olin Sibert

Robert E Tarjan

Lesley R Matheson

Andrew K Wright

Susan S Owicki

James Jay Horning

William G Horne

R Hugo Patterson

Radek Vingralek

Edward Kihyen Lee

Sazzala Reddy

Varun Mehta

Vanco Buca

Kai Yi Li

Senthil Kumar Ramamoorthy

Ming Benjamin Zhu

Tomasz Barszczak

Gurunatha Karaje

Ajay Gulati

R Hugo Patterson, Iii

William G Home

Pradeep Shetty

Sandeep Karmarkar

Sudhanshu Goswami

Vinay Manivel

Sonam Mandal

James J Homing

Rajat Sharma

Anagha Barve

Murali Krishna Vishnumolakala

Nitin Muppalaneni

Srikant Varadan

Hy Dinh Vu

Chun Liu

Xiaoshan Zuo

Zheng Yang

Suresh Vasudevan

Thomas P McKnight

Susan K Owicki

Steve Rodrigues

Lavina Jain

Kai Li

In some examples, a system performs data deduplication using a deduplication fingerprint index in a hash data structure comprising a plurality of blocks, wherein the hash data structure is stored in persistent storage, and a block of the plurality of blocks comprises fingerprints computed based on content of respective data units. The system uses an indirect block in a memory to access a given block of the plurality of blocks in the hash data structure, the indirect block containing references to blocks of the hash data structure containing the deduplication fingerprint index, and the references indicating storage locations of the plurality of blocks in the persistent storage.

Indirect block containing references to blocks of a persistent fingerprint index

In some examples, a system receives, in a synchronization buffer stored in a memory, fingerprint index entries for respective data units received by the system, each fingerprint index entry of the fingerprint index entries comprising a fingerprint computed for a corresponding data unit, and a storage location indicator for the corresponding data unit. The system merges the fingerprint index entries in the synchronization buffer with fingerprint index entries of a persistent fingerprint index in a hash data structure stored in persistent storage, the hash data structure comprising a plurality of buckets.

Merging buffered fingerprint index entries

According to examples, a storage node may include storage devices and a controller that may determine whether all of a plurality of data chunks of a first intra-node portion of a stripe have been stored on the storage node. Based on a determination that all of the data chunks have been stored, a first intra-node parity chunk may be stored at a second one of the storage devices, in which the first intra-node parity chunk may be determined from at least one of the data chunks of the first intra-node portion. Based on a determination that at least one of the data chunks has not been stored, storage of a first intra-node parity chunk of the stripe on the storage node may be delayed until a determination is made that all of the data chunks of the first intra-node portion have been stored at the storage node.

Storing data in a distributed storage system

Methods and storage arrays are described. One example method is for storage processing on a storage array. The method includes executing a storage application on the storage array. The storage application is configured to process write commands and read commands to and from storage of the storage array, and the write commands and the read commands being from an application. The method includes, processing, by the storage application, a write command from the application. The processing includes storing data blocks of the write command to a write buffer of the storage array. The method also includes processing said data blocks, while processing said storing, to identify data blocks and/or metadata to store in a read cache-on-write buffer. In this example, only a sub-set of said data blocks are temporarily stored in said read cache-on-write buffer. The method includes draining data blocks, from time to time, from said read cache-on-write buffer to end storage of said storage. The draining includes producing data segments that include two or more of said data blocks.

Storage system with read cache-on-write buffer

Methods, systems, and computer programs are presented for allocating CPU cycles in a storage system. One method includes operations for receiving requests to be processed, and for associating each request to one task. A foreground task is for processing input/output requests, and the foreground task includes one or more flows. Each flow is associated with a queue and a flow counter value, where each queue is configured to hold requests. The method further includes an operation for selecting one task for processing by the CPU based on an examination of the number of cycles processed by the CPU for each task. When the selected task is the foreground task, the flow having the lowest flow counter is selected. The CPU processes a request from the queue of the selected flow, and the flow counter of the selected flow is increased based on the data consumption of the processed task.

Quality of service implementation in a networked storage system with hierarchical schedulers

Systems and methods are disclosed for protecting a computer program from unauthorized analysis and modification. Obfuscation transformations can be applied to the computer program's local structure, control graph, and/or data structure to render the program more difficult to understand and/or modify. Tamper-resistance mechanisms can be incorporated into the computer program to detect attempts to tamper with the program's operation. Once an attempt to tamper with the computer program is detected, the computer program reports it to an external agent, ceases normal operation, and/or reverses any modifications made by the attempted tampering. The computer program can also be watermarked to facilitate identification of its owner. The obfuscation, tamper-resistance, and watermarking transformations can be applied to the computer program's source code, object code, or executable image.

Software self-defense systems and methods

Cluster storage comprises an interface and a processor. The interface is to send a tag to a selected node and receive tags from the selected node. The tags received from the selected node comprise tags for likely similar segments stored on the selected node. The processor is to break a segment into subsegments, calculate subsegment tags for each subsegment, identify one or more references to one or more previously stored subsegments and/or one or more segment data using the tags from the selected node and the subsegment tags, and send the one or more references to the one or more previously stored subsegments and/or segment data and associated tags to the selected node.

Cluster storage using subsegmenting for efficient storage

A storage device made up of multiple storage media is configured such that one such media serves as a cache for data stored on another of such media. The device includes a controller configured to manage the cache by consolidating information concerning obsolete data stored in the cache with information concerning data no longer desired to be stored in the cache, and erase segments of the cache containing one or more of the blocks of obsolete data and the blocks of data that are no longer desired to be stored in the cache to produce reclaimed segments of the cache.

Identification of blocks to be retained in a cache based on temperature

A storage device includes a flash memory-based cache for a hard disk-based storage device and a controller that is configured to limit the rate of cache updates through a variety of mechanisms, including determinations that the data is not likely to be read back from the storage device within a time period that justifies its storage in the cache, compressing data prior to its storage in the cache, precluding storage of sequentially-accessed data in the cache, and/or throttling storage of data to the cache within predetermined write periods and/or according to user instruction.

Methods and systems for reducing churn in a caching device

Methods, systems, and computer programs are presented for allocating CPU cycles and disk Input/Output's (IOs) to resource-creating processes based on dynamic weights that change according to the current percentage of resource utilization in the storage device. One method includes operations for assigning a first weight to a processing task that increases resource utilization of a resource for processing incoming input/output (IO) requests, and for assigning a second weight to a generating task that decreases the resource utilization of the resource. Further, the method includes an operation for dynamically adjusting the second weight based on the current resource utilization in the storage system. Additionally, the method includes an operation for allocating the CPU cycles and disk IOs to the processing task and to the generating task based on their respective first weight and second weight.

Network storage device using dynamic weights based on resource utilization

Methods, systems, and computer programs are presented for deduplicating data in a storage device. One method includes an operation for identifying multiple deduplication domains for a storage system. A fingerprint index is created for each deduplication domain, where each data block stored in the storage system is associated with one of the plurality of deduplication domains. The method also includes operations for receiving a first data block the storage system, and for identifying a first deduplication domain from the plurality at of deduplication domains corresponding to the first data block. The first data block is deduplicated within the first deduplication domain utilizing a first fingerprint index associated with the first deduplication domain.

Multiple deduplication domains in network storage system

Systems and methods are disclosed for providing a trusted database system that leverages a small amount of trusted storage to secure a larger amount of untrusted storage. Data are encrypted and validated to prevent unauthorized modification or access. Encryption and hashing are integrated with a low-level data model in which data and meta-data are secured uniformly. Synergies between data validation and log-structured storage are exploited.

Trusted storage systems and methods

Methods and systems are presented for allocating CPU cycles among processes in a storage system. One method includes operations for maintaining segments in a first memory, each segment including blocks, and for maintaining a block temperature for each block in a second memory. The first memory is a read-cache where one segment is written at a time, and each block is readable from the first memory without reading the corresponding complete segment. The block temperature is based on the frequency of access to the respective block, and a segment temperature is based on the block temperature of its blocks. Additionally, the segment with the lowest segment temperature is selected for eviction from the second memory, and blocks in the selected segment with a block temperature greater than a threshold temperature are identified. The selected segment is evicted, and a segment with the identified blocks is written to the first memory.

Access-based eviction of blocks from solid state drive cache memory

Methods, systems, and computer programs are presented for storing data in a solid state drive (SSD). One method includes an operation for detecting a plurality of streams writing to the SSD, each stream writing in sectors, a page including a plurality of sectors and a block including a plurality of pages. A write operation includes writing at least one complete page, and an erase operation includes erasing at least one complete block. The method further includes operations for allocating a write buffer for each stream in RAM memory, and for storing each received sector of a stream in the corresponding write buffer. When a write buffer stores enough sectors to fill a page, content of the write buffer is written to a page in flash memory such that the page is filled. Further, the write buffer is freed after writing the content of the write buffer to the flash memory.

Low write amplification in solid state drive

Methods, systems, and computer programs are presented for deduplicating data. One method includes an operation for receiving a data block having a logical address. The storage system includes a logical address mapping index for mapping logical addresses to block numbers, a block index for mapping block numbers to physical locations, and a fingerprint index for mapping fingerprints of data blocks to block numbers. Additionally, the method includes an operation for inline processing the data block. Inline processing the data block includes determining a fingerprint of the data block, examining the fingerprint index to determine if the fingerprint is already mapped to an existing data block in the storage system, if the fingerprint is already mapped then adding a mapping of the logical address to the existing data block in the logical address mapping index, and if the fingerprint is not already mapped then creating the corresponding entries in the indices.

Processing of incoming blocks in deduplicating storage system

Growing community of inventors

San Jose, CA, United States of America

Umesh Maheshwari