Amazon Technologies, Inc.

Kevin Michael Beranek

Samuel Leonard Moniz

Keian Christopher

Kyle Bradley Peterson

Vijay Ravindra Kulkarni

Ajit Ashok Varangaonkar

Jun Xu

Christopher Keakini Kaulia

Andrew Evenson

Chris Hasz

Mechanisms for resource isolation allow tenants executing in a multi-tenant software container to be isolated in order to prevent resource starvation by one or more of the tenants. Mechanisms for dependency isolation may be utilized to prevent one tenant executing in a multi-tenant software container from using another tenant in the same container in a manner that requires co-tenancy. Mechanisms for security isolation may be utilized to prevent one tenant in a multi-tenant software container from accessing protected data or functionality of another tenant. Mechanisms for fault isolation may be utilized to prevent tenants in a multi-tenant software container from causing faults or other types of errors that affect other tenants executing in the same software container.

Isolating tenants executing in multi-tenant software containers

Technologies are described herein for managing the activation of software containers, for throttling of requests directed to tenants executing in a software container, and for priming the execution of software containers. The activation of software containers and tenants may be managed by maintaining an activation queue for storing requests to activate software containers or tenants on a host computer. Requests may be retrieved from the queue and utilized to determine whether a software container or a tenant is to be activated on a host. 'Bounce' or hot swap tenant activations may be performed. Tenant activation requests might also be throttled by denying the requests or by postponing the requests until a later time. A software container might also be configured to throttle incoming requests to tenants executing therein. The execution of a software container might also be primed by replaying previously recorded network traffic to the software container.

Software container activation and throttling

A classloader executing in an execution environment, such as a JAVA virtual machine or a software container, may be configured to generate class usage data describing the historical usage of classes by applications in the execution environment. Based upon the class usage data, one or more classes may be pre-loaded into a cache prior to receiving a request from an application to load the classes. If an application subsequently requests a class, the request may be satisfied using the class stored in the cache rather than by loading the class at the time the request is received. A probabilistic data structure, such as a Bloom filter, might also be utilized to determine whether a classloader can possibly load a requested class. Only if the classloader can possibly load the requested class will a search be made for the requested class in a classpath associated with the classloader.

Optimized class loading

Host computers in a host pool are configured to execute a tenant management service. The tenant management service receives a tenant configuration from a fleet management service that defines tenant services that are to be executed on each host computer. When tenant services are deployed, the tenant configuration may be utilized to determine whether deployed tenants are to be executed or not. The fleet management service might also provide commands at a later time to execute deployed but not executing tenant services or to terminate executing tenant services. The determination as to whether to execute or terminate tenant services may be made based, at least in part, on status information provided by the tenant management service to the fleet management service. The fleet management service might also provide a runtime configuration to tenant services in order to configure aspects of the runtime operation of the tenant services.

Deployment and management of tenant services

A routing service provides functionality for selecting an instance of a tenant service for processing a service request and for providing the network location of the selected instance to a service client. The service client may utilize the network location to make a service request to the selected instance of the tenant service. The routing service might also route the service request to the selected instance of a tenant service. The selection of a particular instance of a tenant service for processing a service request may be made based upon various factors including, but not limited to, attributes of the service client making the request and/or attributes of instances of the tenant service being called. The selection of an instance of a tenant service might also be made to implement segmentation, to implement A/B testing, to load balance service requests, and/or to gradually deploy new versions of a tenant service.

Routing to tenant services utilizing a service directory

The techniques described herein provide software testing of a candidate software system. In some examples, an interceptor intercepts at least one production request to a production software system. The duplicating proxy service causes the at least one intercepted request to be processed by the candidate software system being validated and the production software system being used to validate the candidate software system. The duplicating proxy service may then compare and/or analyze at least one candidate response to the intercepted request from the candidate software system and at least one production response to the intercepted request from the production software system.

Software testing environment that includes a duplicating proxy service

Technologies are described herein for isolating tenants executing in a multi-tenant software container. Mechanisms for resource isolation allow tenants executing in a multi-tenant software container to be isolated in order to prevent resource starvation by one or more of the tenants. Mechanisms for dependency isolation may be utilized to prevent one tenant executing in a multi-tenant software container from using another tenant in the same container in a manner that requires co-tenancy. Mechanisms for security isolation may be utilized to prevent one tenant in a multi-tenant software container from accessing protected data or functionality of another tenant. Mechanisms for fault isolation may be utilized to prevent tenants in a multi-tenant software container from causing faults or other types of errors that affect other tenants executing in the same software container.

Isolating resources for utilization by tenants executing in multi-tenant software containers

A multi-tenant service container receives a container health check request and responsively identifies a list of expected tenants. The list of expected tenants may include all of the tenants hosted by the multi-tenant service container, all of the tenants hosted by the multi-tenant service container that are associated with a particular process or a list of tenants defined by the container health check request. The multi-tenant service container issues a tenant health status request to the tenants in the expected tenant list and responsively receives a tenant health status from the tenants. The received tenant health status is either a tenant healthy status or a tenant unhealthy status. The multi-tenant service container issues a container health status based on the tenant health statuses received from the tenants on the expected tenant list.

Growing community of inventors

Seattle, WA, United States of America

Kevin Michael Beranek