Amazon Technologies, Inc.

Nomura Holdings, Inc.

Anthony A Virtuoso

Rahul Sharma Pathak

Turkay Mert Hocanin

Mehul A Shah

Jonathan Andrew Fritz

Santosh Chandrachood

Bijay Singh Bisht

Anurag Windlass Gupta

Ramkumar Kamalapuram Sugavanam

Eric Mills

Gitesh Tyagi

Wesley Blumenthal

Nachimuthu Govindasamy

Stefano Stefani

Tate Andrew Certain

Allan Henry Vermeulen

Rui Liu

Ashok Kumar

Christopher Richard Jacques De Kadt

Benjamin Albert Sowell

Dallas Lamont Willett

Jason Douglas Denton

Robin Alan Golden

Richard Shawn Bice

Jian Fang

Anoop Kochummen Johnson

Linchi Zhang

Sachin Suresh Bhat

Rahul Pathak

Andrew Shapira

Rajul Mittal

Konstantin Milyutin

Vignesh Rajamani

Matthew Fata

Akila Tennakoon

Seth Thomas Denney

Philip Simko

Shashank Bhardwaj

Roman Gavrilov

Brian Scott Ross

Linan Zheng

Montana Norman Wong

Carey Michael Crook

Mahmoud Salem

Yugandhar Maram

Balaji Kannan

Shiyu Sun

Yu Yan

Nivetha Purusothaman

Aditya Manohar

Parag Pramod Chaudhari

Jalpan Randeri

Matt Bachman

Yishan Yang

Jeff Thomas Carter

Udit Mehrotra

Sneha Bharadwaj

Ajay Baliram Jadhav

Jason Denton

Techniques for adaptive sleep virtual machine management are described. A service of a provider network receives a parameter indicating that a first virtual machine can be slept and determines to sleep the first virtual machine based at least in part on the parameter. A state of the first virtual machine is captured, and the first virtual machine is terminated. The service determines to resume the captured state of the first virtual machine based at least in part on an indication. A second virtual machine is launched using the captured state of the first virtual machine and resumes execution of the captured state of the first virtual machine. A proxy server sends traffic to the first virtual machine before the termination of the first virtual machine and to the second virtual machine after the resumption of execution of the captured state of the first virtual machine by the second virtual machine.

Adaptive sleep virtual machines in a cloud provider network

Techniques for self-tuning an analytics system via observed execution optimization are described. Upon a need for execution resources, a resource manager can select a type of executor from multiple candidate executor types based at least in part on one or more of current execution data associated with the execution of tasks of a user application and/or historic execution data associated with one or more other applications. The current execution data may include event log data originated by the driver application based on the execution of the user application and/or metric data describing characteristics of one or more worker nodes involved with executing the user application or characteristics of one or more other executors implemented by the one or more worker nodes in executing the user application.

Self-tuning analytics system with observed execution optimization

Techniques for a distributed data processing application service in a cloud provider network are described. A virtual machine bundle is obtained, the virtual machine bundle including a copy of a memory and one or more registers of a paused virtual machine executing a distributed data processing application runtime. A request to launch a virtual machine to execute a distributed data processing application runtime is received, the request to launch the virtual machine including a bundle identifier that identifies the virtual machine bundle. The virtual machine is launched based at least in part on the virtual machine bundle. The virtual machine sends a message to register the virtual machine with a cluster of virtual machines.

Distributed data processing application service in a provider network

Techniques for distributed data processing application service networking in a cloud provider network are described. A first request to launch a virtual machine is received, the first request including a session network identifier to identify a network of a distributed data processing application cluster hosted by the provider network. The first virtual machine is launched. An internet protocol (IP) address of the first virtual machine is set, and the IP address of the first virtual machine is formed at least in part by combining an IP subnet assigned to the computer system with the session network identifier. A firewall controlling network traffic to and from the first virtual machine is configured to allow packets having a source or destination address that matches a portion of the IP address of the first virtual machine that includes the session network identifier.

Distributed data processing application service networking in a provider network

A database management system receives a request to process a database query on behalf of a security principal. The database management system determines that processing the database query requires access to an encrypted portion of a file containing data subject to access conditions. The database management system determines that the security principle is authorized to use a key that corresponds to the encrypted portion of the file. The database management system then completes processing of the query by using the key to access the encrypted portion of the file.

Column access control

A multi-framework blockchain service may be implemented with a common interface to manage different types of blockchain networks. Requests to create a blockchain network may be received via an interface for the control plane that triggers the creation of the blockchain network according to an identified workflow. Various operations to change the blockchain network, including membership changes, node additions, governance changes, analytics changes, and monitoring changes may be allowed or denied by the control plane according to a distributed governance policy in effect for the blockchain network.

Multi-framework managed blockchain service

Techniques for storage optimization in a distributed object store are described. A storage optimization service of a provider network monitors changes to data objects in a distributed object store that are part of a data lake and are referenced by a table index. The storage optimization service determines whether particular storage optimizations involving the data objects would be beneficial, prioritizes the ordering of these optimizations with a focus on performing impactful optimizations first, while intelligently scheduling the optimizations to avoid overutilization of available resources.

Storage optimization in a distributed object store

Techniques for indexing large scale datasets are described. A method for indexing large scale datasets can include receiving, by an indexing service, a request to generate an index for a dataset stored in an data storage service, the request including indexing information for the dataset, determining, by the indexing service, an index type based at least on the dataset, generating, by the indexing service, the index based at least on the indexing information and the index type, and receiving, by the indexing service, a request from a query service to identify a subset of the dataset using the index.

Indexing service for petabyte-scale datasets

Techniques are described for an interactive query service that enables users to query data stored at a federated collection of data sources. An interactive query service provides interfaces that enable users to configure the interactive query service to query any number of heterogeneous data sources pertinent to a user. In general, the configuration of a data source can include identification of: a data source type, access configurations related to accessing the data source, and in some cases metadata describing a structure of the data stored by the data source (for example, a data catalog describing schemas, tables, columns, partitions, datatypes, or other metadata associated with the stored data). Once configured, an interactive query service can receive and execute queries that involve data stored at any combination of a user's data sources, where the queries may be expressed using a standard query language such as the Structured Query Language (SQL).

Enabling federated query access to Heterogeneous data sources

Access across private networks may be enabled for blockchain networks in a managed blockchain service. Requests to enable access for a node hosted in a private network to one or more nodes hosted in a different private network that hosts the blockchain network as part of a managed blockchain service may be received. Routes to enable the access may be determined and added to networking devices so that communications between the node in the private network to specified nodes in the private network of the managed blockchain service may be correctly routed.

Enabling access across private networks for a managed blockchain service

Techniques for cooperative preemption of resources in a distributed multi-tenant resource pool are described. A query execution service manages the allocation of computing resources in a provider network by identifying computing nodes in the provider network to preempt. The service causes the computing nodes to be removed from further executing tasks as part of processing their queries in a first group of nodes in the provider network while other computing nodes in the first group continue to execute tasks as part of processing the queries. The service then adds the first computing node into a second group of nodes in the provider network to execute tasks as part of processing other queries in the provider network.

Cooperative preemption in a distributed multi-tenant resource pool

A shared resource service allows multiple clients to agree on rules for accessing a shared resource (e.g., a shared database or a shared service). The shared resource service also allows the clients to make changes to the rules (e.g., via consensus or majority vote). The clients use the shared resource service to enforce the rules, without having to trust each other to follow the rules when accessing the shared resource. First, the clients agree on a set of initial rules for accessing the shared resource and a set of initial rules for making changes to the rules. After the rules are initialized, then the clients can begin using the shared resource in accordance with the agreed upon rules. In response to a request for accessing the shared resource or a request for changing rules, the shared resource service enforce the applicable rules.

Agreement and enforcement of rules for a shared resource

Independent scheduling of resources may be performed for distributed data processing programs. When a request is received that causes the performance of a distributed data processing program, a plan to perform the distributed data processing program may be generated. Different resources may be obtained to perform different portions of the distributed data processing program according to different portions of the generated plan. A result of the request may be provided to a user. In some embodiments, resources obtained for different portions of the distributed data processing program may be returned for use by other programs.

Independent resource scheduling for distributed data processing programs

Techniques for providing an on-demand development environment are described. A service of a provider network receives a request to launch a development environment, such as a notebook, from an electronic device. The service obtains an identification of a computing cluster hosted by the provider network. The service obtains an identification of a compute instance hosted within the provider network, the compute instance executing a software environment to host one or more development environments. The service causes the compute instance to launch a development environment. The service sends a message to configure the launched development environment to execute a computer program written in the development environment using the computing cluster. The service generates a token to secure communications between the electronic device and the development environment and sends the token to an originator of the request.

On-demand development environment

Partitions of a database table may be predicted for processing database queries. When a database query to a database is received, partitions of the database table that may exist may be predicted according to a partitioning scheme. The predicted partitions may be accessed in order to perform the query. In some embodiments, the partitioning scheme may be received from a user of the database. In some embodiments, the partitioning scheme may be obtained by analyzing the database table.

Predicting partitions of a database table for processing a database query

Growing community of inventors

Hawthorne, NJ, United States of America

Anthony A Virtuoso