Amazon Technologies, Inc.

University of Washington

Paul David Franklin

Bryan James Donlan

James Caleb Kirschner

Colin Laird Lazier

Claire Elizabeth Suver

Swaminathan Sivasubramanian

Rajesh Sudhakar Sheth

James Horsley

Grant Alexander Macdonald McAlister

Darin Lee Frink

Jeremiah David Brazeau

James Raymond Allard

David Levy

Samuel Rubin Barrett

Brent James Lutz

Jeffrey Allen Dzado

Peter George Ross

Kevin M Bailey

Ryan Charles Schmitt

Andrew Brenden Tinka

Mary Crys Calansingin

Marvin Michael Theimer

James Christopher Sorenson

Seth William Markle

Rishabh Animesh

Adam Frederick Brock

Jonathan Robert Collins

Masataka Kubo

Mark Robert Sieklucki

Kevin Christopher Miller

Kestutis Patiejunas

Anil M Kumar

Frank Charles Paterra

Grant A M McAlister

Eric Neilsen

Naveen Anand Subramaniam

William H Ebeling

Robert J Czarnecki

Brian Gouldsberry

Qingqing Xiao

Mastaka Kubo

Andrew Brendan Tinka

Darren Charles Cronquist

Aspects of a data environment, such as the creation, provisioning, and management of data stores and instances, are managed using a separate control environment. A user can call into an externally-facing interface of the control environment, the call being analyzed to determine actions to be performed in the data environment. A monitoring component of the control plane also can periodically communicate with the data environment to determine any necessary actions to be performed, such as to recover from faults or events in the data environment. A workflow can be instantiated that includes tasks necessary to perform the action. For each task, state information can be passed to a component in the data environment operable to perform the task, until all tasks for an action are completed. Data in the data environment can be accessed directly using an externally-facing interface of the data environment, without accessing the control plane.

Control service for data management

A switching device is implemented in a network-attachable data transfer device to provide data storage access to other such devices. In some embodiments, network-attachable data transfer devices are arranged in a clustered configuration to provide various computational and storage services. When one or more devices of the cluster fails, various implementations associated with the switching device, via an external data interface, provide operational mitigation, optimized data recovery, and efficient reinstatement of normal operation of the cluster.

Data access interface for clustered devices

Systems and methods are provided for utilizing rules for placement of objects in storage in a manner that improves retrieval times relative to a default ordering utilized by an object storage system. For example, a request to store an object in a persistent storage of a data storage system may be received, metadata associated with the request may then be parsed to identify a signal for placement of the object within the persistent storage, and a rule may be identified for placement of objects associated with that signal, such as by indicating a desired grouping or ordering of objects associated with the signal. A particular storage location for the object may then be determined within the persistent storage based at least in part on the signal, the rule, and previously determined storage locations of one or more other data objects associated with the signal.

Placement of data objects in storage for improved retrieval

First audit information corresponding to a first set of log entries associated with a transformation performed on first data is obtained, where the first audit information includes a first commutative result produced by applying a commutative function to object identifiers associated with the first set of log entries. Second audit information corresponding to a second set of log entries associated with the transformation performed on second data is obtained, with the second data being a different representation of the first data and where the second audit information includes a second commutative result produced by applying the commutative function to object identifiers associated with the second set of log entries. The first commutative result is compared with the second commutative result to validate whether the second data matches the first data. One or more actions are performed depending on whether the second data is the successful transformation of the first data.

Audit correctness of asynchronous batch transformation systems

A processing device receives a request to identify whether a file is a complete file. The processing device determines a first size of the file. The processing device determines whether the first size satisfies a size criterion. The processing device determines that the file is a complete file responsive to the determination that the first size satisfies the size criterion.

File size as an indicator of file properties

Techniques and systems for storing and retrieving data storage devices of a data storage system are disclosed. In some embodiments, inventory holders are used to store data storage devices used by a data storage system. When data is to be transacted with the data storage devices, mobile drive units locate appropriate inventory holders and transport them to a device reading station, where an appropriate device retrieval unit transacts the data. After the data has been transacted, the data storage devices are returned to the appropriate inventory holders, and the inventory holders are placed by the mobile drive units in locations where they may be accessed in response to further data transactions.

Optimization of data request processing for data storage systems

A data center may include a tape library rack module along with rack computer systems. The rack computer systems may be configured to provide computing capacity within a data center environment. In some embodiments, the tape library rack module may include an enclosure encompassing an interior of the tape library rack module, a rack within the interior, and a tape library unit mounted on the rack. The tape library rack unit may include tape cartridges configured to store data within a tape environment that is different than the data center environment. The tape library rack unit may be within a portion of the interior that is enclosed such that it is environmentally isolated from the data center environment. In some examples, the tape library rack module may include a cooling unit and/or a humidifier unit, which may provide the tape environment to the environmentally isolated portion of the interior of the tape library rack module.

Tape library rack module with environmentally isolated interior

A data storage service stores a dataset on a set of storage nodes in accordance with a first encoding. A set of shards constituting quorum, and one or more additional shards, are stored on the storage nodes. The data storage system determines to store the dataset according to a second encoding, in which the second encoding has fewer total shards. The data storage system reconfigures the storage of the dataset in accordance with the second encoding, such that the reconfigured storage comprises subsets of shards from the first encoding that were not re-encoded in forming the second encoding.

Storage system with inter-stretch transcoding

Cohorts may be created on storage nodes in an object-redundant storage system that uses replication and/or a redundant encoding technique. In a cohort with N nodes, M data elements (replicas or shards) of an object are stored to M of the nodes that are randomly (or otherwise) selected from the N nodes. Metadata for locating other data elements for an object in the cohort may be stored with one or more of the data elements in the cohort. To retrieve an object from a cohort, a subset of the nodes may be queried; the subset may be randomly or otherwise selected. If enough data elements are retrieved for the object from the queried nodes, the object is provided to the requester. Otherwise, additional data elements may be retrieved according to the metadata returned with a data element or by querying additional nodes.

Volume cohorts in object-redundant storage systems

Techniques and systems for storing and retrieving data storage devices of a data storage system are disclosed. In some embodiments, inventory holders are used to store data storage devices used by a data storage system. When data is to be transacted with the data storage devices, mobile drive units locate appropriate inventory holders and transport them to a device reading station, where an appropriate device retrieval unit transacts the data. In some embodiments, each inventory holder includes a heterogenous mix of data storage device types, the layout of which may be calculated according to the specific mix allocated to a given inventory holder. After the data has been transacted, the data storage devices are returned to the appropriate inventory holders, and the inventory holders are placed by the mobile drive units in locations where they may be accessed in response to further data transactions.

Heterogenous media storage and organization in automated data storage systems

Adaptive data recovery for clustered data devices

A system and method for dynamically implementing a resource budget based at least in part on receiving information that prompts a determination of whether to adjust a maximum amount of resources available for utilization at least in part by data storage operations. As a result of the determination, the system and method produce, based at least in part on the information, a resource budget that reflects an adjustment to the maximum amount of resources available for utilization at least in part by the data storage operations, and implements the resource budget such that performance of the data storage operations is adjusted in accordance with the adjustment to the maximum amount of resources available for utilization.

Dynamic power budgets

A multilayered index is implemented for data stored in a data storage system. The multilayered index may include a granular index that specifies the location of specific files or abstractions within the data storage system, while a skip table specifies segments of the abstractions, and may point to both the granular index and the data storage devices of the data storage system. In redundancy coded systems, the use of multilayered indices may enable asynchronous retrieval of data during normal or adverse operation (such as during periods of varying data availability).

Layered indexing for asynchronous retrieval of redundancy coded data

Growing community of inventors

Seattle, WA, United States of America

Paul David Franklin