Amazon Technologies, Inc.

James Christopher Sorenson, Iii

Allan Henry Vermeulen

John David Cormie

Yun Lin

Eric Michael Wagner

David M Barth

Nicholas Gordon

Ami K Fischman

Nishanth Alapati

Alan B Atlas

Hao He

Mrithyunjaya Kumar Annapragada

David Carl Salyers

Ankur Khetrapal

Adam Douglas Morley

Steve Huynh

Owen Cummings

Karl Gohde

Antonio Gabriel Perez De Tejada Martinez

Graham Pedersen

Time and value ordering may be applied for items stored in data backups. A change log that persists changes to a data set may be updated with changes and used to update an in-memory table for the data set, which describes changes to items up to a current time. An event may be detected to seal the in-memory table from subsequent updates and a persistent data object that orders the items in the in-memory according to both keys of the respective items and the respective time values of the items, as stored in the change log, may be generated and stored as part of a backup for the data set.

Time and value ordered data objects for a backup of a data set

A lightweight filesystem may be provided for remote storage caching. A filesystem may maintain a persistent cache for a data set stored as data files in immutable data objects in a remote data store. Filesystem metadata may be evaluated to determine whether a portion of a data file is stored in the persistent cache according to an offset and length specified in a request. If in the persistent cache, data obtained from a data block in the persistent cache may be returned. If not in the persistent cache, then the remote data store may be accessed and the data file in the immutable data object read to obtain the portion of the data file.

Lightweight filesystem for remote storage caching

A storage gateway serves as an interface between processes on a customer network and a service provider. The storage gateway is located on-premise with the customer processes. To customer processes, it appears that data is stored locally. However, the storage gateway interfaces with a remote storage service to store the data. For cached gateways, the primary data store is a remote data store, while frequently accessed data may be locally cached by the gateway. Reads may be satisfied from the local cache or from virtual data storage; writes are handled so as to appropriately update data blocks in the local cache and/or in virtual data storage. For shadowing gateways, the primary data store is the local data store; reads are passed through to a local data store, and writes are shadowed to virtual data storage as well as being sent to local data store.

Remote storage gateway shadow or cache configuration

A system capable of identifying and locking resources (e.g., items, tickets, etc.) may map resources to a bit vector and use bit vectors to efficiently identify and lock the resources. For example, tickets to a concert may be mapped to a bit vector and the system may generate an availability bit vector indicating an availability of the seat locations. Thus, the tickets may be reserved by changing a binary value for bits corresponding to individual tickets. In addition, the system may generate bit vectors corresponding to metadata associated with the seat locations (e.g., aisle seat, handicap accessible, first ten rows, back ten rows, etc.), allowing the system to easily identify available seat locations that satisfy the search request. As the bit vectors are relatively small and can be processed easily, the system may improve a latency and reduce an error rate associated with searching for and reserving seat locations.

Method for selling and searching for items

Techniques are described for managing access to data storage in a plurality of bitstore nodes. In some situations, a data storage service uses multiple bitstore nodes to store data accessible via a network, such as the Internet. In some situations, multi-level cache systems are employed by each bitstore node and managed, for example to increase throughput of the data storage service.

Multi-level page caching for distributed object store

Techniques are described for automatically managing the use of a buffer cache in volatile memory, including in at least some situations to distribute the buffer cache across the volatile memory of a pool of multiple computing devices. Furthermore, the data that is stored in such a distributed buffer cache may in some situations be stored in a redundant manner, such that the stored data may be recovered from a subset of the computing devices in the pool, such as by using erasure coding techniques. In some situations, the described techniques are used to provide a distributed database buffer cache to support operations of a database server program for a database&#x2014;thus, as database data is retrieved from a non-volatile storage device, that database data may also be redundantly stored across the distributed database buffer cache for later retrieval and use.

Managing data stored in distributed buffer caches

Methods, apparatus, and computer-accessible storage media for restoring data from a snapshot to a data volume. The blocks in the volume may be treated as an implicit tree structure, for example a binary tree; each local block corresponds to a block on the snapshot. A local block on the volume may be marked, for example fingerprinted with metadata, to indicate that the local block has not been restored. Initially, the local block at the root node is marked. To restore a local block, the restore process may generate a list indicating all local blocks on a path from the root node of the tree to the target node that have not been restored. The marks in the local blocks are used in generating the list. For each block indicated in the list, children of the block are fingerprinted, and the block is restored from the snapshot.

Methods and apparatus for data restore and recovery from a remote data store

Techniques are described for managing data storage and retrieval, such as by generating a hash value for a beginning and/or ending portion of a new data group to be stored, attempting to match each generated hash value to a plurality of hash values associated with other existing data groups that are already stored, and comparing the new data group and an identified existing data group to determine any identical extended initial or final portion of the two data groups. The new data group may then be stored as the portion unique to the new data group, along with a reference to the existing identical extended portion of the existing data group. The new stored data group may be later retrieved using the unique portion of the data group, along with the existing extended portions corresponding to other stored data groups for any stored references.

Reducing head and tail duplication in stored data

A distributed, web-services based storage system. A system may include a web services interface configured to receive, according to a web services protocol, a given client request for access to a given data object, the request including a key value corresponding to the object. The system may also include storage nodes configured to store replicas of the objects, where each replica is accessible via a respective unique locator value, and a keymap instance configured to store a respective keymap entry for each object. For the given object, the respective keymap entry includes the key value and each locator value corresponding to replicas of the object. A coordinator may receive the given client request from the web services interface, responsively access the keymap instance to identify locator values corresponding to the key value and, for a particular locator value, retrieve a corresponding replica from a corresponding storage node.

Growing community of inventors

Seatlle, WA, United States of America

James Christopher Sorenson, Iii