Amazon Technologies, Inc.

Lucent Technologies Inc.

Nference, Inc.

Aravind Srinivasan

Mihir Sathe

Murali Aravamudan

Anshumali Shrivastava

Pranav Rao Perampalli Nekkar

Benjamin Ray Coleman

Rakesh Barve

Kajamalai Gopalaswamy Ramakrishnan

Krishnan Kumaran

Iraj Saniee

Venkataramanan Soundararajan

Markus Andreas Hofmann

Francis Xavier Zane, Jr

Arjun Puranik

Peter M Winkler

Akash Anand

Frederick Bruce Shepherd

Lisa Karen Fleischer

Matthew Andrews

Bruce Shepherd

Systems and methods are described for allocating requests to implement new workloads within a set of servers. Each server can have a given proportion of various resources, based on a hardware configuration of the server. Placing multiple workloads that do not match that proportion can result in stranded resources, which are unused but cannot be used due to a lack of other required resources. Embodiments of the present disclosure include load balancer that routes workload requests based on the proportion of resources expected to be used by the workload, by selecting a target server based on whether the target server hosts other workloads that are correlated or anti-correlated with the requested workload. To reduce maintained state information, the load balancer can characterize workloads in terms of clusters, rather than maintaining information as to individual workloads.

Efficient allocation of workloads based on correlated workload clusters

Systems and methods are described for reducing performance variance of code executions on a serverless code execution system. A serverless code execution system can operate to obtain requests to invoke code and handle such requests by generating an execution environment for the code on a host computing device and executing the code within the environment. In some cases, an execution environment is poorly placed, resulting in underperformance of code executions on that environment and variance in overall performance of the code executions. The present disclosure enables a serverless code execution system to identify underperforming execution environments and to replace such environments with new environments, reducing variation in performance across execution of the code. New environments may be placed on host computing devices asynchronously, using a placement algorithm that includes additional processing relative to an algorithm that operates synchronously to code invocation.

Reconfiguration of execution environment pools for serverless functions

A method and system is provided for mapping a term to a vector representation in a semantic space. Provided techniques allow for efficient and accurate determination of vector representations for query terms that are terms of emerging interest or are otherwise not included in a set of terms for which vector representations are pre-calculated.

Systems and methods for mapping a term to a vector representation in a semantic space

Systems and methods are described for allocating requests to implement new workloads within a dynamic set of servers. Existing load balancing techniques can result in 'focus firing' on new servers added to the set, since a load balancer may view a new server as underloaded. With sufficient intensity, focus firing can result in overshooting target load for the new server, and the new server in fact becoming overloaded. The present disclosure modifies selection of servers as potential targets for a workload by at least partly biasing against selection of young servers. The bias imposed can be scaled to avoid overloading new servers.

Allocation of workloads in dynamic worker fleet

Systems and methods are described for implementing an 'arc jump' technique in conjunction with bounded loads in consistent hashing. In general, bounded loads refers to limiting the ability of a single device within a distributed system to store data objects, such that when a request to store a new data object would otherwise be directed to that device, it is instead redirected to an alternative device. Redirecting all requests to a single alternative device can lead to cascading failures, as the alternative device must maintain its own load and that which has been redirected to it. Embodiments of the present disclosure address this by determining an alternative device on a per-object basis, such as by again hashing the object with an additional seed value. This distributes request from an overloaded device among all other devices of the distributed system, avoiding cascading failures.

Arc jump: per-key selection of an alternative server when implemented bounded loads

Systems and methods are described for load balancing requests in a distributed system using consistent hashing. Specifically, systems and methods are described for using 'the power of k choices' when placing new servers on a consistent hash ring used to load balance requests. Rather than placing each new server at a fixed point determined by a hashing algorithm, a load balancer can identify multiple potential points on the hash ring for the new server. The load balancer can then compare these points to determine a preferred location, and place the server at the preferred location. Techniques described herein can substantially improve placement of servers, which in turn results in better load balancing.

Consistent hashing using the power of k choices in server placement

Broadly, techniques for solving network routing within a predetermined error are disclosed. These techniques may be applied to networks supporting dedicated reserve capacity, where reserved capacity on links in the network is dedicated for a particular commodity (generally, a source and sink pair of computers), and shared recovery, where reserved capacity on links is shared amongst two or more commodities. These techniques use an iterative process to determine flows on each of the links in a network. Costs are set for each commodity, and primary and secondary (i.e., backup) flows are initialized. A commodity is selected and demand for the commodity is routed through the shortest path. Costs are updated for each potential failure mode. For each commodity, the flows and costs are updated. Once all flows and costs are updated, then it is determined if a function is less than a predetermined value. If the function is less than a predetermined value, then the commodity selection, and flow and cost adjustments are again performed. If the function is greater than the predetermined amount, then the network routing problem is solved to within a predetermined amount from an optimal network routing.

Fast and scalable approximation methods for finding minimum cost flows with shared recovery strategies, and system using same

A scalable system and method for locating a closest server in response to a client request via an interactive distribution network, such as the Internet, are provided. A closest content server is defined as having the least round trip time for responding to a client request. The system including a plurality of content servers; and a local server in communication with a plurality of clients, the local server acting as a proxy for communicating client requests from clients to a redirection server. Client network distance and load information is periodically collected at each content server in the network from clients communicating with each of the respective content servers. The redirection server periodically aggregates the network distance and load information from each content server to create client clusters from both current and previously aggregated network distance and load information. Each client cluster represents a division or partition of the total IP address space. Each client cluster is then mapped (paired) to one or more content servers in the network. The mapping or pairing is then utilized to respond to client DN requests received from any client in the network. Another aspect of the invention involves considering the respective capacities of the content servers in the network. A selection probability is assigned to each content server/domain index pair to prevent the repeated selection of the content server having lowest round trip time thereby overloading that server's service capacity. The selection probabilities assigned to each content server effect a load balancing to prevent overloading. Another aspect of the invention involves collecting the distance and load information without incurring any overhead cost by passively collecting TCP information as it is transmitted from clients in communication with content servers in the course of normal communications.

System and method for locating a closest server in response to a client domain name request

A method is disclosed for designing a signaling network of call coordinators (CCs) for internet telephony. The new method can be used to design a CC network of arbitrary size that satisfies, with high probability, limitations on the maximum number of sockets per CC and on the maximum number of hops between an arbitrary pair of switches in the network. According to the disclosed method, the network of CCs is treated initially as a collection of isolated points, one point for each CC. Then, links are added between pairs of CCs, excluding pairs lying within the same switch. The links are added randomly, but with a particular probability p, which may be different for different pairs. Thus, whether a given link is added depends, in effect, upon the outcome of a loaded coin toss in which the probability of a positive outcome is p.

Growing community of inventors

Ellicott City, MD, United States of America

Aravind Srinivasan