Amazon Technologies, Inc.

John Luther Guthrie

Danny Wei

Marc Stephen Olson

Nachiappan Arumugam

Benjamin Arthur Hawks

Norbert Paul Kusters

James Michael Thompson

Kerry Quintin Lee

Madhuvanesh Parthasarathy

Jianhua Fan

John Robert Smiley

Brian Todd Burruss

Jeevan Shankar

Norbert P Kusters

Jiahua He

Nandakumar Gopalakrishnan

Systems and methods for provisioning a slave copy for redundant data storage and for writing data to persistent storage in a block-based storage system using sequential operation numbers are provided. In one embodiment, the method includes maintaining a master copy and a slave copy of a data volume, the master copy including data generated by a plurality of operations having respective sequential operation numbers, receiving a write instruction for second data to be added to the master copy, and recording the second data as a note that is not readable. The method may further include sending a copy of the note from the master copy to the slave copy, committing the note to the master copy with a sequential operation number, and committing the copy of the note to the slave copy based in part on the sequential operation number. A B-tree may be created based at least in part on an offset for a write instruction associated with the second data, a length, and an operation number included in the note.

Systems and methods including committing a note to master and slave copies of a data volume based on sequential operation numbers

Systems and methods are described for a storage processing service that processes multiple storage commands. The storage processing service uses tags from test storage commands to determine whether a test storage service is to be instantiated that reflects a corresponding production service. Test storage commands with the same tag are tested on that test service. Additionally, the storage processing service determines a strategy for testing processes on production services when the storage system is overloaded. In one embodiment, the test service manager can determine to stop testing processes for a period of time, and issue a shed command that queues or sheds test storage commands. Advantageously, a shed command, while active at a storage processing service, may alleviate the overload on production services. The test service manager can continue to monitor the storage system to determine whether the overload continues to exist.

Managing test services in a distributed production service environment

Persistent storage for a master copy is provided using operation numbers. A master copy can include a persistent key-value store such as a B-tree with references to corresponding data. When provisioning a slave copy, the master copy sends a point-in-time copy of the B-tree to the slave copy, which stores a copy of the B-tree, allocates the necessary space, and updates the references of the B-tree to point to a local storage before the data is transferred. When writing the data to persistent storage, a snapshot created on the master copy is an operation that is replicated to the slave copy. The snapshot is generated using a volume view that includes changes to chunks of data of the master copy since a previous snapshot, as determined using the operation number for the previous snapshot. Data (and metadata) for the snapshot is written to persistent storage while new input/output operations are processed.

Data replication snapshots for persistent storage using operation numbers

Systems and methods are described providing detailed input/output (I/O) metric information that is collected and gathered by an agent of the storage volume. An I/O request is received by a storage volume, and the agent associates primary and secondary identifiers with that I/O request. For example, a trace may be associated with that I/O request. The agent may store this I/O metric information in a ring buffer. Further, after collection and aggregation, statistics may be published by an I/O metric service that further processes the data provided by the agent. Advantageously, interdependent relationships associated with the I/O request or I/O operations of that request may be included in those statistics. This may allow an operator to evaluate the performance of I/O requests for a network.

Managing I/O requests

Systems and methods are described for dynamically detecting outliers in a set of input/output (I/O) metrics collected and aggregated by a storage volume network. An I/O request is received by a storage volume network, and an agent of the storage volume network associates primary and secondary identifiers with that I/O request. For example, a trace may be associated with a request to write data to a storage volume network, and spans may be associated with the individual operations required to fulfill that request. Once gathered, I/O metrics may be aggregated based on the associated identifiers. I/O metric information regarding outliers may be received from the storage volume network, processed, and published by an I/O metrics service to identify the outliers among the primary and secondary identifiers. These outliers may then be stored for further analysis, and may be utilized to determine improvements to the performance of a storage volume network.

Computer system for managing I/O metric information by identifying one or more outliers and comparing set of aggregated I/O metrics

Interchangeable garden tool

Growing community of inventors

Bellevue, WA, United States of America

John Luther Guthrie