Amazon Technologies, Inc.

Hitachi Data Systems Engineering UK Limited

Bluearc UK Limited

Fiorenzo Cattaneo

Tate Andrew Certain

Sachin Jain

Matthew Scott Garman

James Robert Hamilton

Danny Wei

David Nolan Sunderland

Roland Paterson-Jones

Christopher J Aston

Francesco Lacapra

Trevor Edward Willis

Simon L Benham

Kerry Quintin Lee

Matthew S Garman

Trevor Willis

David N Sunderland

Customers of a shared-resource environment can provision resources in a fine-grained manner that meets specific performance requirements. A customer can provision a data volume with a committed rate of Input/Output Operations Per Second (IOPS) and pay only for that commitment (plus any overage), and the amount of storage requested. The customer will then at any time be able to complete at least the committed rate of IOPS. If the customer generates submissions at a rate that exceeds the committed rate, the resource can still process at the higher rate when the system is not under pressure. Even under pressure, the system will deliver at least the committed rate. Multiple customers can be provisioned on the same resource, and more than one customer can have a committed rate on that resource. Customers without committed or guaranteed rates can utilize the uncommitted portion, or committed portions that are not being used.

Committed processing rates for shared resources

Commitments against various resources can be dynamically adjusted for customers in a shared-resource environment. A customer can provision a data volume with a committed rate of Input/Output Operations Per Second (IOPS) and pay only for that commitment (plus any overage), for example, as well as the amount of storage requested. The customer can subsequently adjust the committed rate of IOPS by submitting an appropriate request, or the rate can be adjusted automatically based on any of a number of criteria. Data volumes for the customer can be migrated, split, or combined in order to provide the adjusted rate. The interaction of the customer with the data volume does not need to change, independent of adjustments in rate or changes in the data volume, other than the rate at which requests are processed.

Managing committed request rates for shared resources

Managing committed processing rates for shared resources

A set of virtualized computing services may include multiple types of virtualized data store differentiated by characteristics such as latency, throughput, durability and cost. A sequence of captures of a data set from one data store to another may be scheduled to achieve a variety of virtualized computing service user and provider goals such as lowering a probability of data loss, lowering costs, and computing resource load leveling. Data set captures may be scheduled according to policies specifying fixed and flexible schedules and conditions including flexible scheduling windows, target capture frequencies, probability of loss targets and/or cost targets. Capture lifetimes may also be managed with capture retention policies, which may specify fixed and flexible lifetimes and conditions including cost targets. Such data set capture policies may be specified with a Web-based administrative interface to a control plane of the virtualized computing services.

Data set capture management with forecasting

Techniques are described for facilitating interactions with device driver modules. In at least some situations, the techniques include managing interactions between device driver modules and other programs or hardware devices so as to minimize disruptions related to the device driver modules, including when changes to existing device driver modules are made. Such device driver module changes may have various forms and may occur for various reasons, including to install new versions of device driver modules or otherwise upgrade existing device driver modules. Furthermore, the interactions with device driver modules may be managed in various manners, including to allow changes to occur to a device driver module while that device driver module is in use on a computing system, but without causing other programs on the computing system to be restarted or to lose existing connections to the device driver module being changed.

Facilitating device driver interactions

Systems and methods are disclosed that facilitate the collection, transmission and processing of information related to the storage processes initiated in conjunction with virtual machine instances instantiated on a hosted virtual network. A virtual machine manager component, such a hypervisor component, receives a storage request from one or more virtual machine instances hosted on a host computing device. Additionally, the virtual machine manager component can further request via the storage protocol that the storage subsystem should collect and return storage process metric information. If the storage subsystem supports the collection of the storage process metric information, the storage subsystem will provide a response to the storage request and at least some of the requested storage process metric information. The virtual machine manager component can then process the response to the storage request as appropriate and forward any storage process metric information to a storage process metrics management component.

Storage process metrics

Techniques, including systems and methods, for capturing data sets include performing a client-side two-phase commit to ensure one or more data consistency conditions. A logical volume may represent a data set that is distributed among a plurality of physical storage devices. One or more client devices are instructed to block at least acknowledgment of write operations. When the one or more client devices have blocked at least acknowledgment of write operations, one or more servers in communication with the physical storage devices are instructed to capture corresponding portions of the data set. When the servers have been instructed to capture corresponding portions of the data set, the client devices are instructed to resume at least acknowledgment of write operations.

Techniques for capturing data sets

A distributed file system and method distributes file system objects across multiple self-contained volumes, where each volume is owned by a unique file system node. Logical links are used to reference a file system object between volumes. Each file system node includes a relocation directory in which is maintained hard links to locally-stored file system objects that are referenced from another file system node using logical links. Various file system operations that involve multiple volumes are performed without having to place a write lock on more than one volume at a time. Various caching schemes allow the various file system nodes to cache file system object data and metadata.

Growing community of inventors

Snoqualmie, WA, United States of America

Fiorenzo Cattaneo