Hitachi Vantara Corporation

Hitachi Data Systems Corporation

Benjamin Isherwood

Yury Kats

Matthew Michael McDonald

Donald P Pannese

David B Pinkney

Richard Rogers

Rich Rogers

Christopher S Lacasse

Jack A Orenstein

Alan Bryant

Kevin Canuette Grimaldi

Jeffrey M Crump

Vitaly Zolotusky, Jr

Vitaly Zolotusky

Vitaly Zolutusky

David P Pinkney

Steven Haapaoja

Cody Lee

David Charubini

Kevin Scott Grimaldi

In some examples, a system may use content classes to manage indexing of object data. A content class may include a set of one or more content properties. Each content property may include a name, an expression for extracting data, and an expression type. When object data is received, such as for indexing, the expression type of each content property may be compared with the data type of the received data. Based at least in part on determining that the expression type matches the data type, the system may extract a data value from the received data in accordance with the expression. The system may save the extracted data value to a data structure in association with the name of the content property, and may subsequently use the data value and the name of the content property when creating an index for the object data.

Content classes for object storage indexing systems

In a namespace distributed across storage devices, collisions of objects stored in the namespace can occur. To manage collisions, first, a collision flag is provided in system metadata of the objects, and second, a predetermined naming convention is applied to the pathnames of collided objects within the namespace. A winner object of a collision is stored with the pathname of the actual object (e.g., '/object') in the namespace, and the loser object is stored with a pathname derived from the actual object (e.g., '/object.collision') and the collision flag of the loser object is set to mark the loser object within the namespace. Accordingly, winners and losers of collisions in the namespace can be identified by searching system metadata for collision flags which indicate collision losers, and associated winners can be identified based on the pathname of the losers.

System and method of collision management in a namespace of a storage system

In some examples, a plurality of content properties may be used for managing indexing of object data, such as a plurality of files. Each content property may include an expression for extracting data values from specific data fields in the object data. Further, each content property may include a name that may be used for indexing the objects using the extracted data values. For instance, multiple content properties may have the same name, but may each have different expressions for extracting different data values from different data fields. Accordingly, the content properties may be used to extract desired data values from the objects for indexing. When a search query is received including a content property name and a specified value, one or more of the objects that are indexed with the specified value may be searched.

Content class for object storage indexing system

A storage system comprises: a controller; a memory; one or more objects, each object having content data and metadata which includes system metadata; and a policy which governs writing of the content data and metadata of the one or more objects from the storage system to an external storage. The policy includes a configurable criterion for decoupling of the content data and the metadata of a given object and a rule for determining storage locations for the content data and the metadata between the storage system and the external storage in a decoupled manner if the content data and the metadata of the object are decoupled. The configurable criterion is based on an attribute of the metadata of the object. The controller is operable to execute the policy against the objects and to apply the rule to determine storage locations for the content data and metadata of a decoupled object.

Decoupled content and metadata in a distributed object storage ecosystem

A redundant array of independent nodes are networked together. Each node executes an instance of an application that provides object-based storage. The nodes are grouped into systems each having multiple nodes. A search index is provided in each system for locating objects within the system. A method of managing the search index comprises splitting the search index into shards to be divided among a plurality of regions (). The regions are larger in number than the nodes in the system. Each region has a master and at least one backup copy of one of the shards. A shard core is a master or backup copy of a shard. The method further comprises distributing the shard cores to the nodes, and providing a region mapping table in each node in the system to map the shard cores to the nodes in which the shard cores are stored ().

Highly available search index with storage node addition and removal

A storage system stores objects and copies of objects on the storage system and other storage systems external to the storage system. The storage system stores the copies in storage pools of volumes, which are organized into one or more tiers. The configuration settings of each tier and each pool within the tier may be configured according to a user's preferences. In one example, the number of copies of data content and the number copies of metadata associated with the data content an individual pool stores may be specified. When objects are migrated between tiers, the objects are stored among the storage pools of the tiers. If the number of data content copies or metadata copies is increased, the data or metadata is copied from a determined copy source and if the number of copies decreases, the data is removed from the volumes in the pools.

Service plan tiering, protection, and rehydration strategies

A method and apparatus for metadata favored replication in an active/active topology. When replicating data objects between multiple storage systems, metadata is sent first while the associated data content is sent at a subsequent time. Metadata is stored and made visible on a target storage system thereby reserving the object path in a namespace to reduce collisions in the namespace. After metadata is transferred to the target storage system, the target system performs collision processing to determine if an object already exists in the namespace prior to receiving the transferred metadata. After collision processing is performed, the winning object is made visible for access on the target storage system regardless of whether the associated data content has been transferred to the target system.

Metadata favored replication in active topologies

A redundant array of independent nodes are networked together. Each node executes an instance of an application that provides object-based storage. The nodes are grouped into a plurality of systems each having multiple nodes. The systems have one or more replication links each being formed to indicate replication of data from one system to another system in a replicated environment where each system is configured as a sub-domain in a Domain Name System (DNS) infrastructure. A DNS alias synchronization method comprises maintaining updated information, within each system, of all replication links involving that system and of DNS aliases of other systems associated with all replication links involving that system (S-S). This enables that system to process network-based requests, on behalf of the other systems, without redirecting the requests from the other systems to that system.

DNS alias synchronization in replication topology

A storage system comprises a controller; a memory; and one or more objects each including content data and metadata. The metadata is used to construct user-defined content properties, each content property providing an ability to extract a specific metadata field from the objects by referring to a user-defined content property name of the content property. The content properties are organized into user-defined content classes, each content class grouping a set of content properties into a named category under a user-defined content class name. The controller is operable to index the content properties of the content classes to create an index. The content properties being indexed are identified by the content property names. In some embodiments, the controller is operable to de-dupe the index, for content properties having same values expressed differently in different metadata formats, by directing the differently expressed values to same index field under same content property name.

A redundant array of independent nodes are networked together. Each node executes an instance of an application that provides object-based storage. The nodes are grouped into a plurality of systems each having multiple nodes. A data protection level method comprises setting a numerical value of a system dynamic data protection level for each system of nodes; and in response to user input, setting a value of a data protection level for one or more data objects in a system, the value being a fixed numerical value or a dynamic value. If the one or more data objects have a dynamic value of the data protection level in a system, the one or more data objects inherit the system dynamic data protection level of the system in which the one or more data objects reside. The value of the data protection level represents a number of copies of data of the one or more data objects to be kept where the one or more data objects reside.

System and method for optimizing protection levels when replicating data in an object storage system

A content platform cluster that comprises an array of nodes is logically partitioned. Using a web-based interface, an administrator defines 'tenants' within the cluster, wherein a tenant has a set of attributes: namespaces, administrative accounts, data access accounts, and a permission mask. A namespace is a logical partition of the cluster that serves as a collection of objects typically associated with at least one defined application. Each namespace has a private file system with respect to other namespaces. This approach enables a user to segregate cluster data into logical partitions. Tenant information, including all associated namespaces, in the cluster (the 'source cluster') is replicated to at least one target cluster over a replication link. Preferably, replication is available for multiple tenants in the source cluster, and a replication algorithm ensures that a particular tenant can be added to the replication link without stalling the progress of other tenants.

Fixed content storage within a partitioned content platform, with replication

A redundant array of independent nodes are networked together. Each node executes an instance of an application that provides object-based storage. The nodes are grouped into a plurality of systems each having multiple nodes. An object recovery method comprises: receiving, by a first system of the plurality of systems from a client application, a read request for an object, the object having been replicated to/from at least one second system among the plurality of systems; if the object of the read request is available in the first system, returning by the first system the object of the read request to the client application; and if the object of the read request is not available in the first system, performing a read from replica process by the first system to access a replica of the object from a second system among the plurality of systems and using the replica of the object to return the object of the read request to the client application.

Growing community of inventors

Tewksbury, MA, United States of America

Benjamin Isherwood