Amazon Technologies, Inc.

Danny Wei

Norbert Paul Kusters

John Luther Guthrie, Ii

Marc John Brooker

Nachiappan Arumugam

Jianhua Fan

James Michael Thompson

Marc Stephen Olson

Christopher Magee Greenwood

Kerry Quintin Lee

Benjamin Arthur Hawks

David R Richardson

Christopher Nathan Watson

Tate Andrew Certain

Kiran-Kumar Muniswamy-Reddy

Nandakumar Gopalakrishnan

James Robert Hamilton

Madhuvanesh Parthasarathy

Sachin Jain

Wells Lin

Fiorenzo Cattaneo

David Nolan Sunderland

Surya Prakash Dhoolam

Tobias L Holgers

Anthony Nicholas Liguori

James T Pinkerton

Matthew Scott Garman

Divya Ashok Kumar Jain

Igor A Kostic

Roland Paterson-Jones

Colin Williams

Sandeep Kumar

Fan Ping

Pieter Kristian Brouwer

Avram Israel Blaszka

John Luther Guthrie

Lalit Jain

Andrew Boyer

Thomas Tarak Mathew Veppumthara

Oleksandr Chychykalo

Sebastiano Peluso

Arvind Chandrasekar

Yu Li

Shuvabrata Ganguly

Sriram Venugopal

Shobha Agrawal

Kristina Kraemer Brenneman

Tarun Goyal

Arpit Tripathi

Jiahua He

Michael Thacker

Andre Podnozov

Vijay Prasanth Rajavenkateswaran

Shreyas Ramalingam

Ravi Nankani

Leah Shalev

Yan Valerie Leshinsky

Craig Wesley Howard

Brad Eugene Marshall

Wayne William Duso

Varun Verma

Asawaree Kalavade

Charles Louis Ward

John Robert Smiley

Yi Li

Mahmood Miah

Lakshmi N Pallikila

David I Gellman

James Andrew Trenton Lipscomb

William Zaharchuk

Rahul Upadhyaya

Romain Benoit Seguy

Oscar Allen Grim Courchaine

Valentin Gabriel Priescu

Robert Howard Sawers

Vaibhav Tyagi

Jeevan Shankar

Fnu Amit Anand Amleshwaram

Shiow-wen Wendy Cheng

Ryan Michael Eccles

Siddhartha Roy

Paul C Reed

Mitchell Gannon Flaherty

Colin Harrison Brace

Gary Michael Herndon

Marc Levy

Peng Wu

Dragos Diaconescu

Anandh Mahalingam

Shailesh Bhagwat

Ravi Mantrala

Shweta Joshi

Pavan Kumar Korlepara

Ii John Luther Guthrie

A data storage system includes a head node and mass storage devices. The head node is configured to flush data stored in a storage of the head node to a set of the mass storage devices of the data storage system. A head node may flush both current version data and point-in-time version data to the set of mass storage devices. Also, the data storage system maintains an index that indicates storage locations of data for particular portions of a volume before and after the data is flushed to the set of mass storage devices. In some embodiments an index includes a current version reference for a volume or portion of a volume and one or more point-in-time snapshot references.

Data system with flush views

Systems and methods are described herein for routing data by transferring a physical storage device for at least part of a route between source and destination locations. In one example, a computing resource service provider, may receive a request to transfer data from a customer center to a data center. The service provider may determine a route, which includes one or more of a physical path or a network path, for the data loaded onto a physical storage device to reach the data center from the customer center. Determining the route may include associating respective cost values to individual physical and network paths between physical stations between the customer and end data centers, and selecting one or more of the paths to reduce a total cost of the route. Route information may then be associated with the physical storage device based on the route.

Dynamic physical data transfer routing

A block-based storage system hosts logical volumes that are implemented via multiple replicas of volume data stored on multiple resource hosts in different failure domains. Also, the block-based storage service allows multiple client computing devices to attach to a same given logical volume at the same time. In order to prevent unnecessary failovers, a primary node storing a primary replica is configured with a health check application programmatic interface (API) and a secondary node storing a secondary replica determines whether or not to initiate a failover based on the health of the primary replica.

Block-storage service supporting multi-attach and health check failover mechanism

Systems and methods are described for use of reserved storage capacity at a cloud provider network extension, sometimes referred to as a provider substrate extension ('PSE') or “outpost”, as a data cache, thereby improving performance of the PSE. The reserved storage capacity may be reserved for a primary purpose, such as data replication in the event of failure of an in-use portion of the PSE's storage capacity. During non-failover operation, the reserved storage capacity may be otherwise unused. Embodiments of the present disclosure enable the reserved storage capacity, when not used for its primary purpose, to be used as a data cache to store information potentially used at the PSE.

Data caching in reserved storage capacity of cloud provider network extensions

Customers of a shared-resource environment can provision resources in a fine-grained manner that meets specific performance requirements. A customer can provision a data volume with a committed rate of Input/Output Operations Per Second (IOPS) and pay only for that commitment (plus any overage), and the amount of storage requested. The customer will then at any time be able to complete at least the committed rate of IOPS. If the customer generates submissions at a rate that exceeds the committed rate, the resource can still process at the higher rate when the system is not under pressure. Even under pressure, the system will deliver at least the committed rate. Multiple customers can be provisioned on the same resource, and more than one customer can have a committed rate on that resource. Customers without committed or guaranteed rates can utilize the uncommitted portion, or committed portions that are not being used.

Committed processing rates for shared resources

A data storage system includes multiple head nodes and data storage sleds. Volume data is replicated between a primary and one or more secondary head nodes for a volume partition and is further flushed to a set of mass storage devices of the data storage sleds. Volume metadata is maintained in a primary and one or more secondary head nodes for a volume partition and is updated in response to volume data being flushed to the data storage sleds. Also, the primary and secondary head nodes store check-points of volume metadata to the data storage sleds, wherein in response to a failure of a primary or secondary head node for a volume partition, a replacement secondary head node for the volume partition recreates a secondary replica for the volume partition based, at least in part, on a stored volume metadata checkpoint.

Data storage system with metadata check-pointing

Commitments against various resources can be dynamically adjusted for customers in a shared-resource environment. A customer can provision a data volume with a committed rate of Input/Output Operations Per Second (IOPS) and pay only for that commitment (plus any overage), for example, as well as the amount of storage requested. The customer can subsequently adjust the committed rate of IOPS by submitting an appropriate request, or the rate can be adjusted automatically based on any of a number of criteria. Data volumes for the customer can be migrated, split, or combined in order to provide the adjusted rate. The interaction of the customer with the data volume does not need to change, independent of adjustments in rate or changes in the data volume, other than the rate at which requests are processed.

Managing committed request rates for shared resources

One or more configuration parameters for an object gateway instance are received at an interface to a provider network, the parameters including an identifier of a first object store of the provider network for which to cache objects in a first object cache of the object gateway instance and an indication of a data transfer mode that controls when objects written to the first object cache are written to the first object store. The one or more configuration parameters are stored in a data store of the provider network and sent to the object gateway instance. A read request that includes the identifier of the first object store and a first object identifier is received from the object gateway instance, and a first object associated with the first object identifier and stored in the first object store is sent to the object gateway instance.

Data caching in provider network substrate extensions

A data storage manager may manage storage locations for blocks of a storage volume. The blocks of the storage volume may be assigned to a logical volume exposed to a computing instance supported by a host. Furthermore, the data storage manager may also generate and maintain a set of rules that specify the locations of blocks of the storage volume, and provides the set of rules to the host. The set of rules may be included in a data structure enabling the host to access the blocks based at least in part on the information included in the set of rules.

Enabling host computer systems to access logical volumes by dynamic updates to data structure rules

Generally described, one or more aspects of the present application correspond to techniques for creating encrypted block store volumes of data from unencrypted object storage snapshots of the volumes. These encryption techniques use a special pool of servers for performing the encryption. These encryption servers are not accessible to users, and they perform encryption and pass encrypted volumes to other block store servers for user access. The encryption context for the volumes can be persisted on the encryption severs for as long as needed for encryption and not shared with the user-facing servers in order to prevent user access to encryption context.

Encryption by default in an elastic computing system

A data storage system includes multiple head nodes and multiple data storage sleds mounted in a rack. For a particular volume or volume partition one of the head nodes is designated as a primary head node for the volume or volume partition. The primary head node is configured to store data for the volume in a data storage of the primary head node and cause the data to be replicated to a secondary head node. The primary head node is also configured to cause the data for the volume to be stored in a plurality of respective mass storage devices each in different ones of the plurality of data storage sleds of the data storage system.

Data storage system with multiple durability levels

A data storage system includes multiple head nodes and data storage sleds. The data storage sleds include multiple mass storage devices and a sled controller. Respective ones of the head nodes are configured to obtain credentials for accessing particular portions of the mass storage devices of the data storage sleds. A sled controller of a data storage sled determines whether a head node attempting to perform a write on a mass storage device of a data storage sled that includes the sled controller is presenting with the write request a valid credential for accessing the mass storage devices of the data storage sled. If the credentials are valid, the sled controller causes the write to be performed and if the credentials are invalid, the sled controller returns a message to the head node indicating that it has been fenced off from the mass storage device.

Data storage system with enforced fencing

A data storage system includes a rack, multiple head nodes, multiple data storage sleds, and at least two networking devices. The at least two network devices are configured to implement at least two redundant networks within the data storage system. Also, each of the head nodes is assigned at least two network addresses for communication with the data storage sleds of the data storage system via the at least two networking devices. The data storage sleds each include multiple mass storage devices and a sled controller that is configured to couple with the at least two network switches. IO In some embodiments, the data storage system further includes redundant power systems within a rack in which the head nodes, the data storage sleds, and the at least two networking devices are mounted.

Data storage system with redundant internal networks

Growing community of inventors

Seattle, WA, United States of America

Danny Wei