International Business Machines Corporation

Google Inc.

Motorola Mobility LLC

Motorola Solutions, Inc.

Pure Storage, Inc.

Harsha Hegde

Wesley B Leggette

Jason K Resch

Manish Motwani

Yogesh Ramesh Vedpathak

Daniel J Scholl

Rohan P Shah

Andrew D Baptist

Venkata Giri Badanahatti

Greg R Dhuse

Ilya Volvovski

Bart Richard Cilfone

Ethan S Wozniak

Tarun Agarwal

Robert T Veitch

Michael Edward Factor

Adam Michael Gray

Jay P Jayapalan

Mark Duane Seaborn

Amit H Lamba

Nicholas Georg Lange

Lakshmi Krishna Priya Kala

David M Beischer

A method for execution by a processing system includes determining to retrieve index data for a set of objects stored by an object storage system. An index type is selected from a plurality of index. A request message indicating a request for the index data of one of a plurality of indices stored by the object storage system corresponding to the index type is generated, and the request message is transmitted to the object storage system. The index data for the set of objects is received in a response message from the object storage system. The index data includes a plurality of pairs that each include a key and a value, where the keys correspond to the index type, and where the value of each pair includes metadata of one of the set of objects. The index data is stored in local memory in response to receiving the index data.

Retrieving index data from an object storage system

A method for execution by an access layer of an object storage system includes In various embodiments, a processing system of an access layer of an object storage system includes at least one processor and a memory that stores operational instructions, that when executed by the at least one processor cause the processing system to receive a request message from a requesting entity via a network, where the request message includes a pre-signed URL. A set of custom policy parameters are extracted from the pre-signed URL. Policy verification data is generated by comparing each attribute of a determined set of attributes of the access request to a corresponding custom policy parameter of the set of custom policy parameters. An access indicated in the request message is executed in response to the policy verification data indicating that each attribute compares favorably to the corresponding custom policy parameter.

Pre-signed URLs with custom policies for data access in an object storage system

Systems and methods for replicating containers in object storage using intents are disclosed. A DS processing unit, upon reception of a write request may determine the location of replicated instances for the container. DS processing unit can then generate an intent for each fork, to indicate a specific write request received. The DS processing unit may save or persist intent objects within one of the containers, or in a dispersed data structure. A DS unit or DS processing unit may then check for intents whose operation has not been completed, and when resources to perform the operation are available, perform the operation specified in the intent. DS processing unit can remove a pending intent once the job or operation is complete. A clean-up agent could run periodically to complete unfinished jobs that are pending within DS processing unit or DS unit due to containers not being available for whatever reason.

Replicating containers in object storage using intents

A storage system access point receives a first access request from a client device, wherein the first access request specifies first data. The storage system access point attempts to execute the first access request. In response to successfully accessing the first data, a first message is transmitted to an external audit system indicating that information corresponding to the first access request is to be recorded by the external audit system. In response to that the first data has been successfully accessed, and that the information corresponding to the first access request has been successfully recorded by the external audit system, notifying the client device that the first access request has been successfully completed.

Atomically tracking transactions for auditability and security

A system and method for co-locating a containerized application and an associated cloud storage bucket are provided. In embodiments, a method includes determining, by a computing device, a location of a containerized application in a data storage network; determining, by the computing device, that a cloud storage bucket does not yet exist in a geographic zone associated with the location of the containerized application; selecting, by the computing device, a data storage location for the cloud storage bucket from a plurality of data storage locations in the data storage network based on the geographic zone; and sending, by the computing device, a request to create the cloud storage bucket to the data storage location and causing creation and co-location of the cloud storage bucket within a predetermined distance of the containerized application.

Co-location of storage buckets with containerized applications

A method includes detecting, by a security module of a dispersed storage network (DSN), a potentially compromised encoded data slice (EDS) of a set of EDSs. The potentially compromised EDS is stored in a storage unit of a set of storage units of the DSN. The method further includes monitoring other storage units of the set of storage units to detect one or more other potentially compromised EDSs of the set of EDSs. When the one or more other potentially compromised EDSs are detected, the method includes determining a data compromise threat level based on the potentially compromised EDSs and the one or more other potentially compromised EDSs and enabling an automatic data preservation protocol based on the data compromise threat level. The automatic data preservation protocol includes one or more of: one or more data preservation options, one or more data tracking options, and one or more notification options.

Automatic data preservation for potentially compromised encoded data slices

A method for execution by a computing device of an access layer of an object storage system includes receiving, via a network, a request message from an edge node, wherein the request message indicates a requested access of a data object. An update frequency of the data object is determined in response to receiving the request message, and a cache time to live (TTL) is determined based on the update frequency. Performance of the requested access of the data object is facilitated in response to receiving the request message, and a response message that includes the cache TTL is generated in response to performing the requested access. The response message is transmitted to the edge node via the network.

Determining cache time to live in an object storage system

A method for execution by one or more processing modules of one or more computing devices of a dispersed storage network (DSN), the method begins by receiving a dispersed storage network (DSN) access request and determining an endpoint address from which the DSN access request is originating. The method continues by determining a first geographic location based on the endpoint address. The method continues by determining a target bucket and object the DSN access request is associated with and determining a second geographic location of the target bucket. For differing first and second geographic locations, the method continues by creating a mirror bucket in the first geographic location and configuring an access layer to route future DSN access requests to the mirror bucket based on TTL needs.

Automated cache buckets using mirrors for content distribution networks (CDN)

A method begins by receiving a data access request regarding a data object. The method continues by determining whether the data object is a primary data object. When the data object is the primary data object, the method continues by identifying one or more other data objects based on information associated with the data object. The method continues by sending a modified data object retrieval request regarding the data object and the one or more other data objects to a storage entity, where the modified data object retrieval request includes a modified header. The method continues by receiving a modified data object retrieval response from the storage entity that includes a modified response header and a modified response body that includes the primary data object and the one or more other data objects. The method continues by outputting the data object and the one or more other data objects.

Dynamically linking objects in an object storage during uploads to cloud

A computer readable storage device includes a first memory section that stores operational instructions that, when executed, cause a computing device to, as data accesses occur for a plurality of data objects of a storage container, update object values to produce updated object values, update object retention costs to produce updated object retention costs, adjust a dynamic retention threshold based on the updated object values and the updated object retention costs and update a data object retention policy for a data object based on the dynamic retention threshold to produce an updated retention policy for the data object. The computer readable storage device includes a second memory section that stores operational instructions that, when executed, cause the computing device to, when a data access is a deletion request, utilizing a current updated data object retention policy to determine and execute a deletion-retention option for the data object.

Dynamic retention policies and optional deletes

A method for execution by a dispersed storage and task (DST) processing unit includes determining to determine to overwrite an original data object stored in a plurality of storage units with an updated data object. Validation level data can be determined, where the validation level data indicates a data object overwrite level, a data region overwrite level, or a data segment overwrite level. Checksum metadata associated with the original data object can be retrieved in response to determining to overwrite an original data object. Overwriting of a subset of data regions or data segments of the original data object can be foregone in response to generating validation data that indicates their checksums in the checksum metadata compare favorably to corresponding overwrite checksum values.

Overwriting data objects in a dispersed storage network

A method for execution by a container instance manager (CIM) includes determining to create a new instance of a first data container, where the first data container is stored in a first memory location. Creation of the new instance of the first data container for storage in a second memory location is facilitated in response to the determining to create the new instance. The method further includes determining to remove a duplicate instance of a second data container. Deletion of the duplicate instance of the second data container from memory is facilitated in response to the determining to remove the duplicate instance.

Managing data container instances in a dispersed storage network

In a data storage network, a data storage transfer is initiated over a communications link between a source endpoint and a storage endpoint. The data storage transfer is configured to transmit at least one data object from the source endpoint to the storage endpoint using a first transfer technology protocol of a plurality of transfer technology protocols supported by the storage endpoint. The source endpoint monitors at least one network performance parameter associated with the communication link between the source endpoint and the storage endpoint. During transfer of the at least one data object, the source endpoint automatically switches from the first transfer technology protocol to a second transfer technology protocol based, at least in part, on a result of the monitoring.

Dynamically switching between object storage transfer protocols

Systems and Methods for encrypting and decrypting data in a dispersed storage network are disclosed. A data object may be encrypted using a data object specific encryption key, a container specific encryption key, a tenant account specific encryption key, or a time based encryption key. This specific, or more generally, secondary encryption key can be derived from a master or primary encryption key. Encryption key metadata pertaining to the master encryption key and the specific encryption key is also created and stored in the DSN. When reading an encrypted data object, the master encryption key can be retrieved and, along with the encryption key metadata, used to derive the specific encryption key. The specific encryption key can then be used to decrypt the encrypted data object to recover the data object.

Use of key metadata during write and read operations in a dispersed storage network memory

A method for responding to a read request from a user for a set of encoded data slices (EDSs) in a distributed storage network begins with a processing module determining that a threshold number of encoded data slices is not available and continues with the processing module determining that one or more copies are available for the set of EDSs. The method continues with the processing module determining whether a combination of the one or more additional EDSs within the copy of the set of EDSs and the available EDSs from the set of EDSs is at least a read threshold number of EDSs, and when a read threshold is available based on the combination the processing module responds to the request using the combination.

Growing community of inventors

Hawthorn Woods, IL, United States of America

Harsha Hegde