Amazon Technologies, Inc.

Michael Arthur Ten-Pow

Jason George McHugh

Praveen Kumar Gattu

Derek Ernest Denny-Brown, Ii

Han Wu

Rohan Gurappagouda Patil

Himangshu Ranjan Borah

Pedro Henrique Almeida Tanaka

A distributed storage system may store data object instances in persistent storage and may store keymap information for those data object instances in a distributed hash table on multiple computing nodes. Each data object instance may include a composite key containing a user key. The keymap information for each data object instance may map the user key to a locator and the locator to the data object instance. A request to store or retrieve keymap information for a data object instance may be routed to a particular computing node based on a consistent hashing scheme in which a hash function is applied to a portion of the composite key of the data object instance. Thus, related entries may be clustered on the same computing nodes. The portion of the key to which the hash function is applied may include a pre-determined number of bits or be identified using a delimiter.

System and method for clustering distributed hash table entries

Systems and methods for providing object versioning in a storage system may support the logical deletion of stored objects. In response to a delete operation specifying both a user key and a version identifier, the storage system may permanently delete the specified version of an object having the specified key. In response to a delete operation specifying a user key, but not a version identifier, the storage system may create a delete marker object that does not contain object data, and may generate a new version identifier for the delete marker. The delete marker may be stored as the latest object version of the user key, and may be addressable in the storage system using a composite key comprising the user key and the new version identifier. Subsequent attempts to retrieve the user key without specifying a version identifier may return an error, although the object was not actually deleted.

System and method for logical deletion of stored data objects

Data security and privacy are improved by a client providing a hashed version of collected data to a remote analysis service, and having the analysis service determine the relevancy of the data from the hashes before requesting the plaintext of the data. In one example, a browser plug-in obtains data which is divided into overlapping three-character sequences, and the sequences are hashed to produce a sequence of hashes. The sequence of hashes is sent by the plug-in to the remote service, which uses the hashes to determine if the associated data is relevant to the analysis performed by the remote service, without requiring access to the associated plaintext. After making the determination, the remote service may request that relevant data be provided to the service in plaintext form, while data that is not relevant need not be sent to the remote service.

Cryptographic data encoding method with enhanced data security

A distributed storage system may store data object instances in persistent storage and may cache keymap information for those data object instances. The system may cache a latest symbolic key entry for some user keys of the data object instances. When a request is made for the latest version of stored data object instances having a specified user key, the latest version may be determined dependent on whether a latest symbolic key entry exists for the specified user key, and keymap information for the latest version may be returned. When storing keymap information, a flag may be set to indicate that a corresponding latest symbolic key entry should be updated. The system may delete a latest symbolic key entry for a particular user key from the cache in response to determining that no other requests involving the keymap information for data object instances having the particular user key are pending.

System and method for fetching the latest versions of stored data objects

Systems and methods for providing object versioning in a storage system may support the logical deletion of stored objects through the use of delete marker objects. In response to a delete operation specifying a key, but not a version identifier, the storage system may create a delete marker object that is stored as the latest object version of the key. In response to performing a subsequent mutating operation, or at pre-determined time intervals, the storage system may perform a local and/or a global analysis of the delete marker objects stored in the system (and/or metadata associated with those delete marker objects) to determine whether any of them are no longer needed to ensure correct operation of the system. The analysis may apply one or more reap conditions to delete marker objects and/or metadata to identify extraneous delete marker objects, removing those that meet one or more reap conditions from storage.

Growing community of inventors

Seattle, WA, United States of America

Michael Arthur Ten-Pow