Amazon Technologies, Inc.

Pejus Manoj Das

Wei Xiao

Kiran Kumar Muniswamy Reddy

David Craig Yanacek

Stefano Stefani

Xianglong Huang

Shirley Xue Yi Ly

Jai Vasanth

Stuart Henry Seelye Marshall

Lokendra Singh Panwar

Mohammad Zeeshan Qureshi

Aanchal Gupta

Matthew Shawn Wilson

Akshat Vig

Sudipta Sengupta

Yijun Lu

Dumanshu Goyal

Poorna Chand Srinivas Perumalla

Adam Douglas Morley

Chase Kernan

Sanjay Govindrao Kulkarni

Lewis Bruck

Haifeng He

Shaun M Vickers

Andrew McGoogan

Christopher Elliott Browder

Soumyadip Banerjee

Jeremy Brian Hanni

Jiacheng Guo

Nirmal Mehta

Huang-Chun Yu

Shirley Xue Yi Leung

Vladimir Mitrovic

Yongcong Luo

A distributed data storage system may implement determining indexing progress for a table stored in the distributed data storage system. A table may be stored in multiple table partitions. When a secondary index is created for the table, each partition may independently index the items stored within the table partition in order to identify those items in the partition that should be stored in the secondary index. During creation of the secondary index, creation progress points that indicate the progress of the indexing across the table partitions may be determined. The creation progress points may be provided via a programmatic, textual, or graphical interface to the distributed data storage system.

Determining indexing progress for a table in a distributed data store

A system that implements a data storage system may store data for a database table in multiple replicated partitions on respective storage nodes. An export of an active database partition may generate an exported copy of the database partition. A first exported copy may be stored locally on a source node and a second exported copy of the database partition may be sent to a recipient node. The source node may validate the first copy responsive to sending the second copy and may perform the validation while the second copy is being sent and processed at the recipient node. If the first copy cannot be validated, the source node may cause the second copy to be invalidated and abort further processing of the second copy. The source node may also cause the second copy to be marked as invalid to prevent reliance on an invalid database partition copy.

Early detection of corrupt data partition exports

Implementations detailed herein include description of a computer-implemented method to migrate a machine learning model from one accelerator portion (such as a portion of a graphical processor unit (GPU)) to a different accelerator portion. In some instances, a state of the first accelerator portion is persisted, the second accelerator portion is configured, the first accelerator portion is then detached from a client application instance, and at least a portion of an inference request is performed using the loaded at least a portion of the machine learning model on the second accelerator portion that had been configured.

Accelerator based inference service

A data storage system may implement implicit checkpoint for generating a secondary of a table. Indexing updates may be replicated and maintained across a replica group storing a table for a data store. Upon detection of a restart event for generating a secondary index, a replica in the replica group may evaluate the indexing updates to determine an index creation restart point according to an order for indexing the table. The generation of the secondary index may be resumed at the index creation restart point. In this way, secondary index generation may continue whether or not a previously indexing replica in the replica group, such as a master replica, is available to continue generating the secondary index.

Implicit checkpoint for generating a secondary index of a table

Methods and apparatus for connection redistribution in load-balanced systems that include multiple load balancers each serving multiple nodes. In the connection redistribution method, each node estimates a connection close rate, which may be based on an estimation of the percentage of the overall client traffic received by the respective load balancer that is being handled by the node. The node generates close requests for connections between the respective load balancer and clients according to the connection close rate. The node sends the close requests to its load balancer, which forwards the close requests to the appropriate clients. Upon receiving a close request, a client may close the connection(s) indicated by the request, obtain a public IP address for a load balancer, and initiate new connection(s) to the respective load balancer via the public IP address.

Connection redistribution in load-balanced systems

Methods and apparatus for configurable-capacity time-series tables are disclosed. A schedule of database table management operations, including at least an operation to change a throughput constraint associated with a table in response to a triggering event, is generated. The table is instantiated with an initial throughput constraint in accordance with the schedule. Work requests directed to the table are accepted based on the initial throughput constraint. The throughput constraint is modified in response to the triggering event. Subsequent work requests are accepted based on the modified throughput constraint.

Configurable-capacity time-series tables

A data storage system may implement generating creation performance metrics for a secondary index of a table. Metrics may be collected from different partitions of a table and different partitions of a secondary index in a table store upon initiating creation of the secondary index for the table that utilize provisioned resources for creating the secondary index. The metrics may be analyzed to generate performance metrics for the creation of the table. The performance metrics may be provided to clients of the data store or to monitoring systems. Monitoring systems may send performance event notifications to clients or automatically perform modifications to the provisioned throughput capacity.

Generating creation performance metrics for a secondary index of a table

A data storage system may implement rate-limiting secondary index creation for an online table. A secondary index may be generated for a table stored in a data store. The table may be incrementally indexed, maintaining the updates determined according to indexing different portions of the table in a queue of pending updates that are subsequently applied at the secondary index. Prior to indexing a portion of the table, an evaluation of a current number of pending updates in the queue of pending updates may be performed with respect to a throttle threshold. If the current number of pending updates exceeds the throttle threshold, then indexing the portion of the table may be throttled. Received updates to previously indexed portions of the table, may be applied to the table and placed in the queue of pending updates without an evaluation of the current number of pending requests.

Rate-limiting secondary index creation for an online table

A non-relational data store may implement validating and non-validating secondary indexes for a table. Operations at a table for a given item may be performed when indexing the item to create a secondary index or when updates to the given item are received. Attribute values of a given item may be validated with respect to an indexing schema for the secondary index. For a non-validating secondary index, validation errors detected for the attribute values may be ignored so that the operation at the table may be performed. For a validating secondary index, validation errors detected for the attribute values may result in denying performance of the operation. In some embodiments, a secondary index from may be changed from validating to non-validating, or non-validating to validating.

Validating and non-validating secondary indexes for a table in a non-relational data store

A data storage system may implement locking item ranges for creating a secondary index of an online table. A secondary index may be generated for a table of items stored in a non-relational data store. Different ranges of items in the data store may be locked while a corresponding portion of the secondary index is generated. Upon generating the corresponding portion of the secondary index, a range of items may be unlocked. While generating the secondary index, the table may be made available for servicing access requests. For a request to update the table received during the generation of the secondary index, a determination may be made as to whether the update is included within a locked range of the table. If locked, the request may be delayed until the range is unlocked.

Locking item ranges for creating a secondary index from an online table

A merchant system is disclosed that can provide multiple online marketplaces that utilize marketplace product catalog layering. Product catalogs associated with each marketplace may be defined and layered such that products sold by one marketplace may be displayed on and purchased by way of another marketplace. The merchant system might also provide a unified electronic shopping cart for storing purchases made at multiple marketplaces, a unified checkout user interface for completing the purchase of items from multiple marketplaces, and a unified order management user interface for managing purchases made from multiple marketplaces.

Marketplace product catalog layering

Growing community of inventors

Shoreline, WA, United States of America

Pejus Manoj Das