Amazon Technologies, Inc.

Mikhail Danilov

Marc John Brooker

Osman Surkatty

Holly Mesrobian

Timothy Allen Wagner

Anthony Nicholas Liguori

Philip Daniel Piwonka

Niall Mullen

Peter Martin McDonnell

Douglas Stewart Laurence

Tobias L Holgers

Andrew James Lusk

Derek Steven Manwaring

Stefan Schneider

David Nasi

Marcin Piotr Kowalski

Tao Chen

Dylan Owen Marriner

Peter Barry

Sean Tyler Myers

Deepthi Chelupati

Suganya Rajendran

Trishika Pattabiraman

Peter Martin Mcdonnell

Systems and methods are described for a multi-tenant mode of a serverless code execution system. For instance, a method may include maintaining a set of execution environments, wherein each execution environment is associated with a serverless function, wherein the serverless function is associated with a software as a service (SaaS) provider that is a tenant of a cloud services provider, wherein the SaaS provider provides services to sub-tenants, wherein the set of execution environments are partitioned based on sub-tenants of the SaaS provider; receiving a call to execute a serverless function, wherein the call includes a serverless function identifier and a sub-tenant identifier; identifying a sub-tenant-specific execution environment of the set of execution environments that is associated with the sub-tenant; and in response to identifying the tenant-specific execution environment, invoking the serverless function on the sub-tenant-specific execution environment.

Multi-tenant mode for serverless code execution

Systems and methods are described for enabling graceful recovery of network connections in a virtual machine instance that has been migrated or temporarily halted. To prevent the virtual machine instance from attempting to reuse open connections, which might fail due to migration or halting, a host device identifies open connections just prior to halting the virtual machine instance on the host, and transmits to the virtual machine instance termination signals for the open connections. The host device may further transmit termination signals to the other parties to such connections. Each termination signal may be formatted so as to appear to originate from the other party to the connection, causing both parties to synchronize their knowledge of the connection. On reactivation, the virtual machine instance can recovery the connections without errors associated with attempted utilization of broken connections.

Network connection recovery after virtual machine transition in an on-demand network code execution system

Systems and methods are described for providing updating of disk images supporting serverless code execution and controlled deployment of updated disk images. A disk image can be defined as a set of layers that represent a file system include code of a serverless function and other data used by the code. A function owner can designate one layer as containing software or other data subject to update. When a new version of the layer is obtained at a serverless compute system, the system can generate a new disk image containing the updated layer. The system can then gradually transition the function to the new disk image, by dividing calls to the function among two versions of the function—one using the prior disk image, and one using the new disk image. Performance data gained from the new version of the function can be used to control the gradual transition.

Controlled automatic updates to disk image layers with compatibility verification

Systems and methods are described for reducing latency to service requests to execute code on an on-demand code execution system by maintaining snapshots of virtual machine instances in a ready state to execute such code. A user may submit code to the on-demand code execution system, which code depends on other software, such as an operating system or runtime. The on-demand code execution system can generate a virtual machine instance provisioned with the other software, and initialize the instance into a state at which it is ready to execute the code. The on-demand code execution system can then generate a snapshot of the state of the instance, and halt the instance. When a request to execute the code is received, the snapshot can be used to quickly restore the instance. The code can then be executed within the instance, reducing the need to initialize the instance or maintain the instance in an executing state.

Reducing execution times in an on-demand network code execution system using saved machine states

Systems and methods are described for providing virtualized transaction retries in an on-demand network code execution system. A user may generate a task on the system by submitting code. The task may include a transaction that can succeed or fail depending on external factors, such as accessing a resource that other processes are also trying to access. The system may obtain success criteria for evaluating whether the transaction was successful, and may configure a virtual machine instance to execute the submitted code. The system may generate a 'snapshot' or otherwise capture information regarding the state of the virtual machine instance prior to executing the code, and may then execute the code or a portion of it to attempt the transaction. The system may then apply the success criteria to determine whether the transaction was successful, and if not may retry the transaction by restoring the snapshot and re-executing the code.

Virtualization-based transaction handling in an on-demand network code execution system

Systems and methods are provided for efficiently configuring an execution environment for an on-demand code execution system to handle a single request (or session) for a single user. Once the session or request is complete, the execution environment is reset, such as by having the hardware processor state, memory, and storage reset. In particular, prior to the execution of code, state of the execution environment of the host computing device is retrieved, such as hardware processor(s), memory, and/or storage state. Moreover, during execution of the code instructions, intermediate state can be gathered. Following the execution of the code, the execution environment is reset based on the saved state related to the hardware processor(s), memory, and/or storage. A subsequent code execution securely occurs in the execution environment and the execution environment is reset again, and so forth.

Single use execution environment for on-demand code execution

Systems and methods are described for providing secure storage of data sets while enabling efficient deduplication of data. Each data set can be divided into fixed-length blocks. The plaintext of each block can be convergently encrypted, such as by using a hash of the plaintext as an encryption key, to result in block-level ciphertext that can be stored. If two data sets share blocks, the resulting block-level ciphertext can be expected to overlap, and thus duplicative block-level ciphertexts need not be stored. A manifest can be created to facilitate re-creation of the data set, which manifest identifies the block-level ciphertexts of the data set and a key by which each block-level ciphertext was encrypted. By use of block-level encryption, nearly identical data sets can be largely deduplicated, even if they are not perfectly identical.

Efficient deduplication using block-based convergent encryption

Systems and methods are provided for scoped credentials within secure execution environments executing within virtual machines instances in an on-demand code execution system. In the on-demand code execution system, the execution environments are reset after every request or session. By resetting the single execution environment after each request or session, security issues are addressed, such as side-channel attacks and persistent malware. Additionally, the use of scoped credentials improves security by limiting the access rights for each code execution request or session to the smallest atomic level for the request or session. Following the request or session, the scoped credential is invalidated.

Single use execution environment with scoped credentials for on-demand code execution

Systems and methods are described for management of garbage collection processes in an on-demand code execution system. An on-demand code execution system may execute user-submitted code on virtual machine instances. As each virtual machine instance executes code, garbage (e.g., memory that is allocated and then not properly de-allocated) may accumulate and may reduce the computing resources that the virtual machine instance makes available for allocation to further code executions. A garbage collection schedule manager may be used to generate a system-wide garbage collection schedule and manage garbage collection processes in accordance with the schedule. The garbage collection schedule manager may schedule garbage collection processes at times when demand for computing resources is forecasted to be low, and may coordinate scheduling of garbage collection processes across multiple virtual machine instances and host computing devices to prevent garbage collection processes from impacting code execution performance.

Network-level garbage collection in an on-demand code execution system

Systems and methods are described for providing storage of encrypted data sets, deduplication of such data sets, and control of the redundancy of those data sets. A form of modified convergent encryption can be employed, whereby an encryption key for a data set is selected based on a combination of the plaintext of the data set and a salt value, with the salt value being selected from a number of permutations corresponding to a desired redundancy of the data set in a storage system. Accordingly, a given data set can result in a number of ciphertexts equal to the desired redundancy, and deduplication can occur by removing duplicative instances of individual ciphertexts. Salt values can be selected according to a variety of criteria, including user-based, time-based, and location-based criteria.

Redundancy controls in convergent encryption using dynamic salt values

Systems and methods are described for simulated data object storage on a data storage system. The system may allow clients to store computed data objects, which are generated from a source data object based on a user-defined transformation. For example, computed data objects may be thumbnail images generated based on a full resolution image. When a request to store a computed data object is received, the system can predict a timing of a next request for the data object. If expected resource consumption associated with storing the data object until a next request exceeds expected resource consumption associated with generating the data object in response to the next request, the system can acknowledge the request to store the data object, but not actually store the data object. Instead, the system may generate the data object in response to the next request.

Simulated data object storage using on-demand computation of data objects

Systems and methods are described for enabling cross-environment application of tracing information for code, such as code executed within an on-demand (or 'serverless') code execution system. Various optimizations exist that allow execution of code to proceed faster or more efficiently over time, by collecting tracing information regarding the execution and using that tracing information to guide compilation of the code. These optimizations are typically designed for long-lived environments. However, executions within an on-demand code execution system often occur in short-lived environments, reducing or eliminating any gains from these optimizations. To address this issue, optimizations made in a first environment based on tracing information can be passed to a subsequent environment, enabling those optimizations to persist across short-lived environments.

Cross-environment application of tracing information for improved code execution

Systems and methods are described for providing performance-based hardware emulation in an on-demand network code execution system. A user may generate a task on the system by submitting code. The system may determine, based on the code or its execution, that the code executes more efficiently if certain functionality is available, such as an extension to a processor's instruction set. The system may further determine that it can provide the needed functionality using various computing resources, which may include physical hardware, emulated hardware (e.g., a virtual machine), or combinations thereof. The system may then determine and provide a set of computing resources to use when executing the user-submitted code, which may be based on factors such as availability, cost, estimated performance, desired performance, or other criteria. The system may also migrate code from one set of computing resources to another, and may analyze demand and project future computing resource needs.

Growing community of inventors

Sammamish, WA, United States of America

Mikhail Danilov