Amazon Technologies, Inc.

James Zuber

Nikhil Shah

Vishwas Narendra

Phillip H Pruett, Iv

Sameer Choudhary

Arvinth Ravi

Ernest S Cohen

Nishant Jain

Tyler Michael Jung

Bryan T Wilkinson

Abhishek Kannan

Pierre Vigneras

Janko Jerinic

Vijayasarathy Kannan

Nicholas Jacob Essenburg

Liming Ye

Suchindra Yogendra Agarwal

Christopher Andrew Stephens

Prathiban Mohanasundaram

Hardik Navin Nagda

Slavcho Georgiev Slavchev

Rajesh Sampath

John Pender

Addison Joseph Burns

Preetham Kowshik

RaghuKishore Balivada

Harishkumar Katagal

David Grant Emeny-Smith

Wilson Li

Mehak Mehta

Systems and methods are provided for efficiently maintaining a transaction sequence witness service to ensure that requests to read data provide consistent results across all storage nodes in a distributed system. Each storage node that stores data in response to a particular update event may first update the transaction sequence witness service with the sequence number for the update event. Thus, any other storage node that services a subsequent request for the data will be able to see, via the transaction sequence witness service, the sequence number that must be available on the storage node to respond to the request.

Data event management for monotonic read consistency in a distributed storage system

Systems and methods are provided for efficiently maintaining a transaction sequence witness service to ensure that requests to read data provide consistent results across all storage nodes in a distributed system. When a storage node receives a request for a data object, the node may obtain&#x2014;from transaction sequence witness service&#x2014;the latest sequence number associated with the object. If the sequence number indicates the object has been modified in the period of time since the currently stored version was stored, the node can obtain the current version from persistent storage.

Witness service for ensuring data consistency in a distributed storage system

Systems and methods are provided for managing data partitions in a distributed storage system and, in particular, the routing data used by the distributed storage system to route requests to the proper caching layers, persistent storage nodes, etc. Data items may be managed in a multi-tier configuration in which they are grouped into different partitions based on their key prefixes, and partitions are grouped into different cells based on key ranges. When partitions are moved from cell-to-cell, or when cells are split, the routing data is changed accordingly. In order to ensure that the correct routing data is used throughout the distributed storage system, a change to routing data may be accompanied by a special barrier record being written to the transaction log of affected partitions.

Request routing management for a distributed storage system

Systems and methods are provided for managing the order of data written to a transaction log in a distributed storage system. In a system with multiple nodes, if sequencers are naively generated without taking into consideration inconsistencies among the different nodes generating the sequencers, then the sequencers may not increase for each data transaction. To alleviate this problem, the node committing the transaction to the transaction log may perform a consistent read and verifies that the sequencer advances. If the sequencer does not advance, the node can perform a context-dependent operation such as adjusting the sequencer, acknowledging the write without committing it to the transaction log, or rejecting the write altogether.

Transaction management for monotonic write consistency in a distributed storage system

Systems and methods are provided for using a distributed cache architecture with different methods to load balance requests depending upon whether a requested data item is a freely-requested item (e.g., a 'hot key'). The cache may be implemented as a consistent hash ring, and most keys may be assigned to particular node based on a consistent hash. For hot key requests, the requests may be distributed among a subset of nodes rather than being assigned to a specific node using consistent hashing. When a witness service is used to ensure that cached data is fresh, verification requests for data regarding hot keys may be batched to avoid overloading the witness service with hot key requests.

Multi-tier cache for a distributed storage system

Growing community of inventors

Seattle, WA, United States of America

James Zuber