Nec Laboratories America, Inc.

Nec Corporation

Other

Cezary Dubnicki

Cristian Ungureanu

Krzysztof Lichota

Erik Kruus

Michal Welnicki

Lukasz Heldt

Jerzy Szczepkowski

Ricardo G Bianchini

Wojciech Kilian

Michal Kaczmarczyk

Benjamin Atkin

Marcin Barczynski

Jakub Sakowicz

Lukasz Slusarczyk

Jaroslaw Wrona

Bartlomiej Romanski

Leszek Gryz

Christian Toelg

Rekha N Bachwani

Piotr Skowron

Leszek R Gryz

Marek Biskup

Grzegorz Calkowski

Eduardo Pinheiro

Przemyslaw Strzelczak

Elzbieta Adamczyk

Urszula Herman-Izycka

Ricardo Bianchini

Cristian Ungurenau

Rekha Bachwani

A storage device has: a data storage part storing deduplicated block data; a temporary data storage part temporarily storing block data acquired from the data storage part; a data retrieval control part retrieving the block data stored by the data storage part, storing the block data into the temporary data storage part, and retrieving the block data from the temporary data storage part; and a temporary data control part controlling the storage state of the block data stored by the temporary data storage part. The storage device also has a retrieval turn information storage part storing retrieval turn information which is information about a turn to be retrieved of the block data. The data retrieval control part causes the temporary data storage part to store the block data acquired from the data storage part on the basis of the retrieval turn information acquired from the retrieval turn information storage part, and the temporary data control part controls the storage state of the block data in the temporary data storage part on the basis of the retrieval turn information.

Storage device, program, and information processing method

A storage system according to the present invention includes: a block detecting means for checking whether respective block data obtained by division are already stored in a storage device; and a data writing means for storing the respective block data obtained by division with duplicated storage eliminated into the storage device. The block detecting means detects a share rate representing a rate of a shared portion between a plurality of consecutive block data configuring a predetermined range in write target data among the block data obtained by division and a plurality of block data in a predetermined range already stored consecutively in the storage device. The data writing means, depending on the detected share rate, newly stores the block data obtained by division, into the storage device.

Storage system and deduplication control method therefor

The storage system includes a data storage controlling unit that stores a plurality of units of block data, generated by dividing storage target data, in a distributed manner in a plurality of storage devices, and performs duplicate storage elimination. The data storage controlling unit stores a plurality of continuous units of block data of the storage target data, generated by dividing the storage target data, in a particular storage device among the plurality of storage devices, stores, in the particular storage device, feature data based on the data content of the block data and storing position information representing the storing position in the particular storage device of the block data in association with each other as a storing position specifying table, and stores storage device identifying information for identifying the particular storage device and the feature data of the block data stored in the particular storage device in association with each other as a storage device specifying table.

System and method for deduplication of distributed data

Duplicate storage elimination is performed in units of block data generated by dividing a data stream into arbitrary-sized blocks. Further, sub-block data is generated by further dividing the block data into a plurality of pieces of data, and sub-address data based on the data content of each of the pieces of sub-block data is stored in a predetermined storage device. As such, duplicate storage elimination is also performed in sub-block data units based on the sub-address data.

Storage system

The storage system includes a progress status detection unit that detects respective progress statuses representing proportions of the amounts of processing performed by respective processing units to the amount of processing performed by the entire storage system, each of the processing units being implemented in the storage system and performing a predetermined task; a target value setting unit that sets target values of processing states of the processing units, based on the detected progress statuses of the respective processing units and ideal values of the progress statuses which are preset for the respective processing units; and a processing operation controlling unit that controls the processing states of the processing units such that the processing states of the processing units meet the set target values.

Controlling throughput of processing units associated with different load types in storage system

In a content address storage system, storage target data or address data is stored in a storage device with respect to each time zone divided in a time-series manner, and a storage region in the storage device storing a piece of data, which is not pointed to by other address data, of the storage target data or the address data stored in the storage device in a past time zone before a current time zone, is released.

Storage system for storing target data and address data

The storage system includes a first auxiliary storage device, a second auxiliary storage device, and a main storage device, and also includes a data management unit which stores and keeps, in the main storage device, index data based on feature data by referring to the feature data of storage target data stored in the first auxiliary storage device, and if the index data stored and kept in the main storage device reaches a preset amount, stores and keeps, in the second auxiliary storage device, the index data stored and kept in the main storage device, and deletes the index data stored and kept in the second auxiliary storage device from the main storage device.

Storage system, computer-readable medium, and data management method having a duplicate storage elimination function

An online data storage system () is disclosed in which users may store a variety of personal documents in accordance with a common document structure that can include tag structures that are tailored to specific types of documents. The document and tag structures permit convenient access of the documents and enable providers of goods and services to generate targeted conditional offers while ensuring the privacy of individuals storing the documents.

Structured analysis and organization of documents online and related methods

In accordance with exemplary embodiments of the present invention, a content addressable data structure system may include directed acyclic graphs (DAGs) of data content that are addressed using both a user-defined search key and content of data blocks. Internal keys of retention roots of the DAGs may be derived from the user-defined search key while the remaining blocks may be content addressed. As opposed to using a content address, the user may provide the search key when retrieving and deleting DAGs retaining the data content. In addition, the internal keys may be implemented using internal content addressable storage operations, such as applying a hash function and employing a distributed hash table.

Content addressable storage systems and methods employing searchable blocks

System(s) and method(s) are provided for data management and data processing. For example, various embodiments may include systems and methods relating to relatively larger groups of data being selected with comparable or better performing selection results (e.g., high data redundancy elimination and/or average chunk size). In various embodiments, the system(s) and method(s) may include, for example a data group, block, or chunk combining technique or/and a data group, block, or chunk splitting technique. Various embodiments may include a first standard or typical data grouping, blocking, or chunking technique and/or data group, block, or chunk combining technique or/and a data group, block, or chunk splitting technique. Exemplary system(s) and method(s) may relate to data hashing and/or data elimination. Embodiments may include a look-ahead buffer and determine whether to emit small chunks or large chunks based on characteristics of underlying data and/or particular application of the invention (e.g., for backup).

Methods and systems for quick and efficient data management and/or processing

A fixed prefix peer to peer network has a number of physical nodes. The nodes are logically divided into a number of storage slots. Blocks of data are erasure coded into original and redundant data fragments and the resultant fragments of data are stored in slots on separate physical nodes such that no physical node has more than one original and/or redundant fragment. The storage locations of all of the fragments are organized into a logical virtual node (e.g., a supernode). Thus, the supernode and the original block of data can be recovered even if some of the physical nodes are lost.

Method for operating a fixed prefix peer to peer network

The present invention generally provides a method for grid storage including balancing read and write requests from applications across a first group of nodes in a grid storage system for avoiding hot spots and optimizing performance through smart caching; balancing storage capacity across a second group of nodes in the grid storage system, nodes in the first and second groups being at least one of hardware interchangeable online, capable of being added to change performance or capacity of the grid storage system and capable of being removed to change performance or capacity of the grid storage system; and self managing of the first and second groups of nodes for providing at least one of scalability, self healing after failure of components in the grid storage, non-disruptive upgrades to the grid storage system, and eliminating duplicate data on an object or sub-object level in the grid storage system.

Method and system for a self managing and scalable grid storage

Data is stored in a distributed data storage system comprising a plurality of disks. When a disk fails, system reliability is restored by executing a set of reconstructions according to a schedule. System reliability is characterized by a dynamic Normalcy Deviation Score. The schedule for executing the set of reconstructions is determined by a minimum intersection policy. A set of reconstructions is received and divided into a set of queues rank-ordered by redundancy level ranging from a lowest redundancy level to a highest redundancy level. For reconstructions in each queue, an intersection matrix is calculated. Diskscores for each disk are calculated. The schedule for the set of reconstructions is based at least in part on the intersection matrices, the Normal Deviation Scores, and the diskscores.

Dynamically quantifying and improving the reliability of distributed data storage systems

Exemplary systems and methods in accordance with embodiments of the present invention may provide a plurality of data services by employing splittable, mergable and transferable redundant chains of data containers. The chains and containers may be automatically split and/or merged in response to changes in storage node network configurations and may be stored in erasure coded fragments distributed across different storage nodes. Data services provided in a distributed secondary storage system utilizing redundant chains of containers may include global deduplication, dynamic scalability, support for multiple redundancy classes, data location, fast reading and writing of data and rebuilding of data due to node or disk failures.

Scalable secondary storage systems and methods

The present invention is directed to improving utilization in a peer-to-peer network. Each node in a peer-to-peer network is assigned a number of preferably fixed-size storage slots proportional to the node storage, a first portion of which is allocated to storage zones and a second portion of which is allocated as a free slot reserve. In accordance with an embodiment of an aspect of the invention, this split of a storage zone is delayed until the storage zone becomes full, referred to herein as 'lazy splitting'. In accordance with an embodiment of another aspect of the invention, it is advantageous to allocate more storage slots at a node than its actual physical capacity referred to herein as 'oversubscription'. In accordance with an embodiment of another aspect of the invention, each physical node is allocated at least one storage zone, even as a new node is added to the system.

Growing community of inventors

Warsaw, Poland

Cezary Dubnicki