Amazon Technologies, Inc.

Surya Prakash Dhoolam

Christopher Magee Greenwood

Mitchell Gannon Flaherty

Marc John Brooker

Gary Michael Herndon

Danny Wei

Marc Stephen Olson

James Michael Thompson

Nishant Satya Lakshmikanth

Anthony Nicholas Liguori

Madhuvanesh Parthasarathy

Sriram Venugopal

Marc Levy

Bradley Nathaniel Burkett

Hans-Philipp Anton Hussels

Iain Michael Christopher Peet

Haoyu Huang

Rahul Karnik

Nachiappan Arumugam

Divya Ashok Kumar Jain

Joshua Dawie Mentz

Michael Groenewald

Steven J Kruy

Mustafa Ugur Torun

James Paul Kardos

Tarun Goyal

Cecilia Deng

Shashanka Krishnaswamy

Jonathan Patrick Lim

Alireza Sahraei

Gyanendra Mishra

Mark Danoher

Gabriël De Vos Smit

Kaili Xu

A data storage manager may manage storage locations for blocks of a storage volume. The blocks of the storage volume may be assigned to a logical volume exposed to a computing instance supported by a host. Furthermore, the data storage manager may also generate and maintain a set of rules that specify the locations of blocks of the storage volume, and provides the set of rules to the host. The set of rules may be included in a data structure enabling the host to access the blocks based at least in part on the information included in the set of rules.

Enabling host computer systems to access logical volumes by dynamic updates to data structure rules

A computing resource monitoring service obtains a request from a third-party partner system to generate a first bus. In response to the request, the computing resource monitoring service updates an account associated with a customer of a remote service to indicate presence of the first bus. The computing resource monitoring service activates the first bus to establish an association with a second bus, obtain a set of events generated by a resource of the third-party partner system, and provides the set of events to the first bus. The computing resource monitoring service publishes the set of events to cause the second bus to distribute the set of events to another remote service.

Hybrid system event processing

A distributed system may implement evaluating placement configurations for distributed resource placement. Placement requests for a partition of a distributed resource may be received. An evaluation of prospective placement configurations of the distributed resource is performed that locates the partition at different resource hosts. In some embodiments, placement configurations may be analyzed with respect to infrastructure zone locality. Multiple infrastructure zone localities may be analyzed and combined to evaluate prospective placement configurations. Prospective placement configurations may be analyzed with respect to other criteria, such as resource host utilization data. Based, at least in part, on the evaluation of the prospective placement, a resource host is identified for placing the partition.

Evaluating placement configurations for distributed resource placement

A distributed system may implement analyzing resource placement fragmentation for capacity planning. Capacity planning may determine when, where, and how much capacity to implement for a distributed system that hosts resources. Placement constraints for resources may, over time, create fragmentation or stranded capacity which is available yet unusable to host new resources. Analyzing capacity fragmentation across a distributed system may allow a determination of available capacity that is actually available to host additional resources. In some embodiments, future resource placements may be estimated in order to perform capacity fragmentation analysis to determine available capacity.

Analyzing resource placement fragmentation for capacity planning

Data replication groups may be used to store data in a distributed computing environment. A data replication group may include a set of nodes executing a consensus protocol to maintain data durably. In order to increase efficiency and performance of the data replication group a placement system may be used to determine the placement of node of the data replication groups. The nodes may be executed by a set of host computer systems within a distributed computing environment. The placement system may determine the placement of the nodes within the set of host computer systems based at least in part on one or more constraints. A selection algorithm may be used to determine a set of node for a data replication group satisfying the one or more constraints.

Node placement for replication groups

Various embodiments of dynamically switching between query and scan for optimizing table reads are described. A particular client of a shared table can use various criteria in determining whether to query for individual entries in a table, or whether to scan the entire table. Criteria can be, in some embodiments, whether there are fewer than a threshold number of other clients accessing the same portion of the table as the particular client, or the size of the portion accessed by the client. If there are fewer than a threshold number of other clients accessing the same portion of the table, and the size of the portion is below a size threshold, for example, then querying individual entries in the table might be appropriate in those circumstances. By querying instead of scanning, for example, clients can reduce the total number of I/O operations per second on the shared table.

Dynamically switching between query and scan for optimizing table reads

A polling resource management system and methodology for facilitating the management of polling resources for polling message queues. A user interacts with customer applications within a web services system. A polling manager within the web services system utilizes polling resources, such as polling threads, polling assignments, and polling workers to access and implement messages in the message queues. A polling resource manager automatically scales the polling assignments, polling threads, and invoke threads to better utilize the polling resources and to reduce latency experienced by the user when interacting with the customer applications.

Polling resource management system

Notice of migration of a portion of a data volume from a first location to a second location is received by a first computer system from a second computer system, where the data volume is separated over a network from the first computer system. A third computer system, separated over a network from the first computer system, is caused to invalidate a mapping between the portion and the first location. An indication that the third computer system seeks access to the portion is identified. A third computer system is enabled, by providing a mapping between the portion and the second location, to access portion at the second location.

Rule invalidation for a block store management system

Data volumes hosted for customers in a multi-tenant environment can be moved advantageously throughout the environment to improve performance and reduce cost. A data volume can serve I/O for a virtual machine instance, and it can be advantageous for the virtual machine and the data volume to be in the same network locality, or share at least some of the same network state and interconnection. Since there is limited capacity in a network locality, data volumes not attached to virtual machines can be moved to other locations in the environment. This can include moving data volumes to other local network topologies or snapshotting data volumes and writing the snapshots to another storage service. If the data volume is again needed for I/O, the data volume can moved, allocated, or reattached as necessary.

Moving storage volumes for improved performance

First information about regions of storage space in a storage environment available for a volume is provided to a service provider, with the storage environment being external to the service provider. The service provider is notified that information usable to locate a storage destination of a portion of the volume is unavailable. Second information that includes the storage destination in the storage environment is obtained from the service provider. A data operation is performed at the storage destination, with the storage destination determined based at least in part from the second information.

Block store management for remote storage systems

A distributed system may implement preventing concentrated selection of resource hosts for placing resources within the distributed system. When a request to place a resource at a resource host is received, an evaluation of load metrics for the resource host is made with respect to rejection criteria. If the rejection criteria is satisfied, then the resource host may reject the request to place the resource. If the rejection criteria is not satisfied, then the resource host may not reject the placement of the resource. Rejection criteria may be provided to resource hosts along with the placement request, in some embodiments. Rejection criteria may be changed, weakened, or removed entirely if initial attempts to place a resource are unsuccessful.

Preventing concentrated selection of resource hosts for placing resources

Data replication groups may be used to store data in a distributed computing environment. A data replication group may include a set of nodes executing a consensus protocol. The nodes may be executed by a set of host computer systems within a distributed computing environment. In addition, the data replication group may include a number of slots used to store data. In this manner, a single data replication group may maintain a plurality of stored data objects.

Replication group data management

A distributed system may implement network locality-based throttling for automatic resource migration. Utilization of a distributed system that includes multiple resources at multiple resource hosts may be monitored. New migration operations to migrate a resource in the distributed system may be evaluated with respect a migration operation limit for one or more network localities of the distributed system. If the migration operation limit for the one or more network localities is exceeded, then the new migration operations may be throttled. Different network localities may have different migration operation limits.

Network locality-based throttling for automated resource migration

A distributed system may implement opportunistic resource migration to optimize resource placement. Resources may be placed amongst different resource hosts of a distributed system. An evaluation of the current placement may be performed according placement criteria that improve placement of the resources at the distributed system. Based on the evaluation, the prospective migration of resources that exceed an improvement threshold may be identified as candidate resources to migrate. Migration for the candidate resources may be opportunistically performed. In some embodiments, a priority may be assigned to the candidate resources according to which the candidate resources are selected for performing migration.

Opportunistic resource migration to optimize resource placement

A customer of a computing resource service provider submits a request to a virtual computer system service to instantiate a virtual computer system. In response to the request, the service obtains physical network information in order to identify any hardware groupings that may be used, at a later time, to provision one or more data storage volumes that may be associated with the virtual computer system. The service assigns a score to each of these hardware groupings based at least in part on the available capacity for provisioning of these data storage volumes. Based at least in part on these assigned scores, the virtual computer system service selects a hardware grouping and instantiates the virtual computer system within the selected hardware grouping.

Growing community of inventors

Seattle, WA, United States of America

Surya Prakash Dhoolam