Hitachi Data Systems Corporation

Hitachi Vantara Corporation

Alan Bryant

Trek Skyward Palmer

David B Pinkney

Kevin S Grimaldi

Nathan W Clark

Richard Rogers

Vitaly Zolotusky, Jr

Richard D Bramante, Jr

Benjamin Isherwood

Walter Wohler

Marty Kulma

Robert Cooper

David Charubini

Alexander Vladimirov Kostadinov

In some examples, a system may use content classes to manage indexing of object data. A content class may include a set of one or more content properties. Each content property may include a name, an expression for extracting data, and an expression type. When object data is received, such as for indexing, the expression type of each content property may be compared with the data type of the received data. Based at least in part on determining that the expression type matches the data type, the system may extract a data value from the received data in accordance with the expression. The system may save the extracted data value to a data structure in association with the name of the content property, and may subsequently use the data value and the name of the content property when creating an index for the object data.

Content classes for object storage indexing systems

In some examples, a plurality of nodes may each host a respective logical database (LDB) that includes database data that references external resources stored in a storage system. Further, each node may maintain, for each external resource referenced by the respective LDB on that node, a reference count of a number of references in the respective LDB to each external resource and a timestamp associated with the reference count. In response to addition of a reference to a particular external resource in a particular LDB, a node that hosts the particular LDB may be programmed to increment the reference count and update the timestamp for the particular external resource. Similarly, in response to removal of the reference to the particular external resource in the particular LDB, the node hosting the particular LDB is programmed to decrement the reference count and update the timestamp for the particular external resource.

Deduplication and garbage collection across logical databases

Example implementations are directed to restoration of data not only from the servers, but also from the clients connected to the server. Algorithms are incorporated to identify content that was created or modified after the last backup. This algorithm also identifies and resolves changes in the mount points for shared folders, preventing information leakage. When the server recovers from a failure, it notifies the clients about the recovery on their next connection. Each client then determines the current state of its mount points and file paths and compares them with the server's mount points and file paths. After the comparison, the client comprehends and indicates the mount point differences by renaming them and sends entire local data (all files, folders, mount points) to the server. The server reconciles the differences.

Distributed disaster recovery file sync server system

A method, of managing storage of content of a system in storage units including run units which do not allow spin-down functionality and spin-down units which allow spin-down functionality, comprises: establishing a storage tiering rule which sets a policy indicating what content is eligible to be stored on a run unit and what content is eligible to be stored on a spin-down unit, the storage tiering rule being applicable to at least a group of the contents in the content system to determine stored content eligibility thereof for storage in the storage units and migration between the storage units; and identifying candidates of contents to migrate between the storage units, based on the storage tiering rule, state of the content system, and the stored content eligibility of the at least a group of the contents.

Content selection for storage tiering

Example implementations described herein are directed to a transaction query engine involving a token that is generated by the server and sent to the corresponding the client. The client uses the token to query transactions made on the server. The token can include mountpoint information detailing shared and private information of the filesystems, transaction identifiers identifying transactions of a filesystem, and a filesystem identifier to identify the corresponding filesystem. The server processes this token and updates the token based on the transaction information of the filesystem, and sends the token back to the corresponding client.

Transaction query engine

An 'operate with missing region' feature allows a cluster to continue servicing reads for available regions even when some regions are missing. Upon a given node failure condition, the cluster is placed in an effective read-only mode for all regions. The node failure condition typically is one where there has been a failure of an authoritative region copy and no backup copy is then available. Typically, 'read-only' means no client write or update requests succeed while the cluster is in this state. Those requests should fail with an error message that indicates failure because of the read-only status. Preferably, such requests are re-tried. In this mode, regions are only allowed to perform read operations but the cluster continues to operate with missing regions, and missing regions are entered on a region map. The cluster then automatically recovers returning missing regions, after which it leaves the read-only state.

System and method for enhancing availability of a distributed object storage system during a partial database outage

A method, of managing storage of content of a system () in storage units () including run units which do not allow spin-down functionality and spin-down units which allow spin-down functionality, comprises: establishing a storage tiering rule () which sets a policy indicating what content is eligible to be stored on a run unit and what content is eligible to be stored on a spin-down unit, the storage tiering rule being applicable to at least a group of the contents in the content system to determine stored content eligibility thereof for storage in the storage units and migration between the storage units; and identifying candidates of contents to migrate between the storage units (), based on the storage tiering rule, state of the content system, and the stored content eligibility of the at least a group of the contents.

A redundant array of independent nodes are networked together. Each node executes an instance of an application that provides object-based storage. The nodes are grouped into systems each having multiple nodes. Objects in the systems have access control lists (ACLs) associated therewith. A method of providing a search key to each ACL in any one system comprises: selecting a region for each metadata object pertaining to the objects in the system; and generating, for each ACL associated with an object in the system, an ACL ID (identifier) as a search key which includes the region number of the region selected for the metadata object pertaining to the object, a region map level corresponding to the region selected, and a monotonically increasing counter value that is automatically incremented each time an ACL ID is generated. The ACL ID remains unique during reconfiguration via splitting/merging existing distributed portions of the database.

Cluster-wide unique ID for object access control lists

This disclosure enables a user to incrementally increase or decrease the size of the cluster and, in particular, to incrementally increase or decrease the number of map regions implemented in a metadata manager associated with the cluster while maintaining availability of the cluster. In a representative embodiment, a configurable target map size is identified, either manually (e.g., by the user) or programmatically. If the target map size is found to be within an acceptable value (given the available node configuration), a procedure to create a new map is triggered. The new map preferably is generated as follows. Initially, one or more new temporary region(s) are created either to split or join the existing regions (depending on whether the map is being increased or decreased). For each temporary region, the process also generates MDPL backup regions. Metadata from an original region (or regions) is then migrated to the temporary region(s). Once the migrations have been completed, the old region or regions (from which the metadata is migrated) are removed and what were the temporary regions are, in effect, promoted. The result is the new map.

System and method for managing scalability in a distributed database

Cluster recovery time is reduced by an incremental refresh technique that is described herein. The goal of the technique is to salvage the portion of the redundant distributed database lost (e.g., during a fault) by performing an incremental refresh of only the updates occurring to that portion of the database since the time of loss.

System and method for managing integrity in a distributed database

An 'operate with missing region' feature of this disclosure allows the cluster to continue servicing reads for available regions even when some regions are missing. In particular, upon a given node failure condition, the cluster is placed in an effective read-only mode for all regions. The node failure condition typically is one where there has been a failure of an authoritative region copy and no backup copy is then available. As used herein, 'read-only' means that no client write or update requests will succeed while the cluster is in this state. In particular, those requests should fail with an error message (or the like) that indicates failure because of the read-only status. Preferably, such requests are then re-tried. In this mode, all regions are only allowed to perform read operations. During the read-only state, the cluster continues to operate with missing regions, and missing regions are entered on the region map. The cluster then automatically recovers returning missing region(s), after which is leaves the read-only state.

Growing community of inventors

Walpole, MA, United States of America

Alan Bryant