Amazon Technologies, Inc.

Seth William Markle

Andrew Kent Warfield

Munif M Farhan

Keun Soo Jo

Alazel Acheson

Christopher Andrew Stephens

Roey Rivnay

James Alexander Bornholt

Colin Laird Lazier

Bryan James Donlan

Paul David Franklin

Jason George McHugh

Douglas Stewart Laurence

Thomas Grant Slatton

Andrew Christopher Schleit

Eric Yves Theriault

James Caleb Kirschner

Jacob Taylor Wires

Leon Thrane

Gregory Scott Benjamin

Qianyong Yu

Grant B Fresen

Valentin Flunkert

Enrico Sartorello

Mihir Dharamshi

Robert Devers Wilson

Alyssa H Henry

David J Carroll

Miroslav Miladinovic

Rajesh Sampath

Steven Michael Hershey

Michael A Uhlar

Jessie E Felix

Christoph Bartenstein

Rajeev Joshi

Jonathan Herman

Shreeranga Puttur Ramakrishna

Rameez Sajwani

Subhakanta Kar

Anup Talwalkar

An object storage data store may be implemented using storage devices providing multiple extents, where individual ones of the extents include multiple independently addressable, contiguous blocks and where updating an extent is performed by updating the respective independently addressable, contiguous blocks sequentially using a series of write operations. The storage devices may include shingled magnetic recording devices and may include conventional persistent storage devices. The key value data store may implement a superblock structure, index structure, and data store structure using respective portions of the multiple extents, where updates to each of the structures is performed sequentially using a series of write operations. The key value data store may further implement a cache for the index structure to minimize input-output (IO) density of the object storage data store.

Persistent object storage with sequential updates

An object storage system includes mass storage devices that implement general storage for objects stored in the object storage system and additionally includes other storage devices, such as solid-state drives, that provide higher performance storage access. The object storage system implements a common access interface for accessing both accelerated access objects (who are eligible to have cached copies stored on the higher performance storage devices) and non-accelerated access objects stored in the general storage. The cache is fully managed by the service and no changes are required for client applications to receive accelerated access to objects that are classified as accelerated access objects per a customer configurable acceleration policy for the object or for a bucket in which the object is stored.

Object storage service with managed cache and common interface for accessing accelerated and non-accelerated access objects

Systems and methods are provided for management of data access latency in a distributed storage system by dynamically replicating data objects experiencing a high request volume. When a storage node is unable to meet a data access latency target (e.g., due to one or more data objects hosted by the storage node being the subject of a large quantity of requests), the storage node may initiate a process by which a second storage node is employed to reduce the request processing load. The second storage node may cache a copy of the requested data object, and may service at least a portion of requests for the data object using the cached copy.

Data retrieval latency management system

Systems and methods are described herein for detecting the inadvertent modification to or deletion of data in a data store and taking automated action to prevent the deletion of data from becoming permanent. The described techniques may also be utilized to detect anomalous changes to a policy or affecting storage of data and taking automated action to mitigate the effects of those changes. In one example, events generated as a result of requests to perform operations on data objects in a data storage service may be obtained, where at least some of the events indicate a failure to fulfill respective requests. Data from the events may be input into a model to detect an anomaly indicative of inadvertent modification of data. As a result of detection of the anomaly, a set of operations may be initiated or performed to prevent the inadvertent modification of data from becoming permanent.

Detecting durability issues with anomaly detection

Technologies are provided for supporting multi-actuator storage device access using logical addresses. Separate sets of storage locations on a storage medium of a storage device can be associated with different actuators of the storage device. For example, a first set of storage locations can be assigned to a first actuator of the storage device and a second set of storage locations can be assigned to a second actuator of the storage device. The storage locations of the storage medium can be associated with logical addresses. The storage device can receive a data access request containing a logical address and can identify a storage location associated with the logical address. The storage device can identify a storage location set to which the storage location belongs and can use an actuator associated with the storage location set to access the storage location associated with the logical address.

Hard disk access using multiple actuators

Technologies are provided for supporting multi-actuator storage device access using logical addresses. Separate sets of logical addresses (such as logical block addresses) can be associated with different actuators of a storage device. For example, a first set of logical addresses can be assigned to storage locations on one or more storage media that is/are accessible using a first actuator of the storage device and a second set of logical addresses can be assigned to storage locations on one or more storage media that is/are accessible using a second actuator of the storage device. The storage device can receive a data access request containing a logical address and can identify a logical address set to which the logical address belongs. The storage device can use an actuator associated with the logical address set to access a storage location assigned to the logical address.

Multi-actuator storage device access using logical addresses

Technologies are provided for supporting storage device write barriers. A host computer can be configured to transmit a write barrier command to a storage device to indicate that one or more data access commands should be processed before one or more other data access commands are processed. For example, a host computer can transmit one or more data access commands to a storage device. The host computer can then transmit a write barrier command to the storage device. The storage device can be configured to receive the write barrier command and to associate a write barrier with the one or more data access commands. The host computer can continue to transmit additional data access commands to the storage device. However, the storage device will not process the additional data access commands until after the one or more data access commands associated with the write barrier have been processed.

Storage device write barriers

Technologies are provided for storing data in a storage device based on an associated attribute or attributes. A storage device can be configured to write data to a storage location of the storage device based on an associated attribute. The attribute can describe one or more storage-related requirements of the data. The storage device can identify one or more storage locations where the data can be stored that meet the storage-related requirements described by the attribute. A host computer can transmit an updated attribute for the data to the storage device to reflect new storage-related requirements for the data. The storage device can write the data to a new storage location that meets the new requirements. A mapping table can be maintained that associates a logical identifier for the data with the actual storage location where the data is stored.

Attribute-driven storage for storage devices

Technologies are provided for a storage device data move command. A storage device can be configured to receive a data move (or garbage collection) command and, responsive to receiving the command, move data from one zone of the storage device (or range of storage locations within the storage device) to another zone (or another range of storage locations) within the storage device. The command can comprise a source zone identifier and a target zone identifier. The storage device can read data from a storage zone associated with the source zone identifier and write the data to another storage zone associated with the target zone identifier. The identifiers can include ranges of storage location addresses within the separate storage zones. In at least some embodiments, a host bus adapter can be configured to support the data move (or garbage collection) command for a storage device attached to the host bus adapter.

Efficient storage device data move operation based on priority of garbage collection command

Technologies are provided for supporting storage device write barriers. A storage device can be configured to associate a data access command with a write barrier. The write barrier can be used to indicate that one or more data access commands should be processed before one or more other data access commands are processed. For example, a host computer can transmit one or more data access commands to a storage device. The storage device can determine that the one or more data access commands are associated with a write barrier. The host computer can continue to transmit additional data access commands to the storage device. However, the storage device will not process the additional data access commands until after the one or more data access commands associated with the write barrier have been processed.

Described are systems and methods for storing a data object using a hierarchical erasure encoding to store a physical representation of the data object across a plurality of fault domains. A first erasure encoding is applied to the data object to generate a first set of shards of the data object. Individual shards of the set of shards may then be distributed across the fault domains for storage. Within the fault domains a second erasure encoding may be applied to the individual shards to generate a second set of shards. Finally, a manifest may be generate in order to reconstruct the data object from the first set of shards and the second set of shards.

Hierarchical erasure coding for multi-region storage

Technologies are provided for accessing a physical location of a storage medium reader of a storage device. A computing device can transmit a request for the storage medium reader's physical location. The storage device can determine the physical location of the storage medium reader and transmit the location to the computing device. The computing device can use the physical location of the storage medium reader to determine an expected latency for retrieving one or more stored data items. The computing device can transmit a command to change the physical location of the storage medium reader, for example by changing a location of the storage medium reader with respect to a given storage medium and/or changing a storage medium that is accessed by the storage medium reader. The computing device can control the placement of the storage medium reader in order to optimize retrieval of data items from the storage device.

Storage medium reader physical location access

A multi-actuator storage device includes separate actuators that can be used to access a storage medium. A property of a request can be analyzed to determine which actuator to use to access data associated with the request. For example, high-priority requests can be handled by an actuator designated for random I/O requests, whereas low-priority requests can be handled by an actuator designated for sequential I/O requests. Writes can be treated as low-priority requests. In a particular embodiment, a priority bit can be used to determine which actuator is used to access the data.

Multi-actuator storage device with actuator selection

A data storage service stores a dataset on a set of storage nodes in accordance with a first encoding. A set of shards constituting quorum, and one or more additional shards, are stored on the storage nodes. The data storage system determines to store the dataset according to a second encoding, in which the second encoding has fewer total shards. The data storage system reconfigures the storage of the dataset in accordance with the second encoding, such that the reconfigured storage comprises subsets of shards from the first encoding that were not re-encoded in forming the second encoding.

Growing community of inventors

Seattle, WA, United States of America

Seth William Markle