Amazon Technologies, Inc.

Venkatraghavan Srinivasan

James Christopher Sorenson

Douglas Stewart Laurence

Fan Zhang

Akshay Suhas Vaidya

Iii James Christopher Sorenson

James Christopher Iii Sorenson

James Christopher Sorenson Iii

A distributed load balancer in which a router receives packets from at least one client and routes packet flows to multiple load balancer (LB) nodes according to a per-flow hashed multipath routing technique. For a given packet flow, the LB nodes randomly select a server node as a target for the packet flow from among multiple server nodes and send a connection request to the server node. A load balancer module on the server node makes the decision on whether to accept or reject the connection based on one or more metrics indicating a respective server's current load. If the module accepts the connection request, a connection is established between the server and the client. Otherwise, the load balancer nodes may select another server node and try again. The connections established between clients and servers pass through the load balancer nodes, but are not terminated at the load balancer nodes.

Distributed load balancer

A connection publishing method for a distributed load balancer in which a router receives packets from at least one client and routes packet flows to multiple load balancer (LB) nodes, which in turn distribute the packet flows among multiple server nodes. A load balancer (LB) module on each server node determines and publishes a list of active connections between client(s) and the server (a connection publishing packet) back to the LB nodes. The LB module may randomly select a LB node to receive the packet. Upon receiving the packet, the LB node processes the connection information in the packet, distributing the connection information among appropriate LB nodes. Receiving the connection publishing information from the server nodes refreshes leases for the connections on the LB nodes.

Connection publishing in a distributed load balancer

A test system for a distributed load balancer in which a router receives packets from at least one client and routes packet flows to multiple load balancer (LB) nodes, which in turn distribute the packet flows among multiple server nodes. The test system includes message bus technology that enables the distributed load balancer to be run in a single process without requiring load balancing code to be deployed to multiple hosts in a production network. The message bus technology may be implemented in message bus layers of the test system to simulate network segments. The message bus functionality hooks into the IP tables at the kernel level, intercepts packets, and sends the packets up into the message bus process for routing.

Message bus for testing distributed load balancers

A distributed load balancer in which a router receives packets from at least one client and routes packet flows to multiple ingress servers. For unknown packet flows, an ingress server cooperates with primary and secondary flow trackers to establish connections to server nodes. For known packet flows, the ingress server sends the packets to target server nodes. The server nodes randomly select egress servers for outgoing packets of the packet flows. The ingress servers, flow trackers, and egress servers are implemented by multiple load balancer nodes in a load balancer node layer. The ingress and egress servers for a given packet flow may be on different load balancer nodes. The load balancer nodes may use a consistent hash function to compute a consistent hash ring for the nodes according to packet flow client/public endpoint pairs so that nodes associated with given packet flows can be located.

Asymmetric packet flow in a distributed load balancer

A distributed load balancer in which a router receives packets from at least one client and routes packet flows to multiple load balancer (LB) nodes, which in turn distribute the packet flows among multiple server nodes. Each LB node may serve in ingress, egress, and/or flow tracker roles. Each LB node may include a first network interface controller (NIC) that faces the router and a second NIC that faces the server nodes. Each LB node may implement a core packet processing architecture in which packets received at the NICs are distributed among non-blocking input queues of a set of worker cores by receiver cores for the NICs, processed from the input queues by the worker cores, and placed on non-blocking input queues of transmit cores for the NICs.

Growing community of inventors

Seattle, WA, United States of America

Venkatraghavan Srinivasan