Amazon Technologies, Inc.

Rishabh Animesh

James Caleb Kirschner

Paul David Franklin

Sandesh Doddameti

Umar Farooq

Siddharth Shah

Kevin Christopher Miller

Anil M Kumar

Mark Christopher Seigle

Adam Frederick Brock

Ryan Charles Schmitt

Naveen Anand Subramaniam

Anusha Dasarakothapalli

Jan Dean Larroza Catarata

Brian Gouldsberry

Robert J Czarnecki

Qingqing Xiao

A data storage system implements techniques to efficiently store and retrieve structured data. For example, structured data is transformed into correlated segments, which are then redundancy coded and archived in a correlated fashion. The characteristics of the redundancy code used enable flexible handling of the archived data without excessive latency.

Archival data storage for structured data

A computing system may generate a directed graph to access data stored in multiple locations or blocks of a data storage device or system. Cost values may be determined for each of multiple paths between nodes, representing the blocks or subsets of data. In some cases, nodes having a cost value between them that is less than a threshold may be combined into a single node. A master path, linking at least two of the multiple paths, between a start node and an end node, may be generated by iteratively selecting paths with a lowest cost. The number of paths considered for determining the lowest path cost may be limited by a complexity parameter, so as to optimize the path to access the data without introducing unbeneficial computational complexity.

Optimal-path finding algorithm for data on storage media

Systems and methods are provided for utilizing rules for placement of objects in storage in a manner that improves retrieval times relative to a default ordering utilized by an object storage system. For example, a request to store an object in a persistent storage of a data storage system may be received, metadata associated with the request may then be parsed to identify a signal for placement of the object within the persistent storage, and a rule may be identified for placement of objects associated with that signal, such as by indicating a desired grouping or ordering of objects associated with the signal. A particular storage location for the object may then be determined within the persistent storage based at least in part on the signal, the rule, and previously determined storage locations of one or more other data objects associated with the signal.

Placement of data objects in storage for improved retrieval

An archival data storage service identifies, in response to a request to retrieve data previously archived by the service, a set of data storage devices that collectively include the data. From the set of data storage devices, the archival data storage service identifies a set of bins, where a subset of the set of bins includes the data. Based on a deadline for fulfillment of the request, the archival data storage service generates a schedule for retrieval of the set of bins for obtaining the data. The schedule is provided to cause retrieval of the set of bins in accordance with the schedule.

Decoupling data request rate from hardware medium for archival data storage devices

A data storage system implements techniques for efficient retrieval of data stored thereon, using time of upload or another monotonically increasing variable as a key or identifier for the data to be stored and/or retrieved. Data is sorted according to, e.g., upload time, and the data is addressed with respect to time of upload and byte offset within the archive.

Monotonic time-based data storage

First audit information corresponding to a first set of log entries associated with a transformation performed on first data is obtained, where the first audit information includes a first commutative result produced by applying a commutative function to object identifiers associated with the first set of log entries. Second audit information corresponding to a second set of log entries associated with the transformation performed on second data is obtained, with the second data being a different representation of the first data and where the second audit information includes a second commutative result produced by applying the commutative function to object identifiers associated with the second set of log entries. The first commutative result is compared with the second commutative result to validate whether the second data matches the first data. One or more actions are performed depending on whether the second data is the successful transformation of the first data.

Audit correctness of asynchronous batch transformation systems

A computer system, such as a data storage system, implements techniques for managing unpredictable and/or bursty system demand. For example, a database service implements a schema and/or a database table rotation that allows for events to be allocated substantially evenly across the available keyspace. In some embodiments, asynchronous and synchronous components of a data processing system are accordingly isolated so as to improve the operational efficiency and utilization of both.

Techniques and systems for scalable request handling in data processing systems

Techniques described and suggested herein include implementations of caches and scalers to handle data storage requests, and storage event status requests associated with data storage requests, in a scalable fashion. For example, a data storage system, such as a data storage system implemented by a computing resource service provider in connection with providing an archival storage service or other data storage service, may be implemented to maintain a consistent response time and backend capability for incoming data storage requests, which may be a component of ensuring a consistent customer experience for customers of an associated service, with little or no regard to peaky or high data storage request rates observed by the implementing data storage system.

Growing community of inventors

Seattle, WA, United States of America

Rishabh Animesh