US 12,413,523 B1
Low-latency stateful load-balanced connections using stateless load balancers
Ethan Joseph Torretta, Edmonds, WA (US); Schuyler David Thompson, Seattle, WA (US); Jose De Jesus Camacho Ruiz, Woodinville, WA (US); Aaron Graydon Bannert, Seattle, WA (US); and Gowtham Kumar Puligundla, Bothell, WA (US)
Assigned to Amazon Technologies, Inc., Seattle, WA (US)
Filed by Amazon Technologies, Inc., Seattle, WA (US)
Filed on Sep. 30, 2022, as Appl. No. 17/937,398.
Int. Cl. H04L 47/125 (2022.01); H04L 67/1031 (2022.01)
CPC H04L 47/125 (2013.01) [H04L 67/1031 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A system, comprising:
a plurality of computing devices comprising respective processors and memory and configured to implement a plurality of request handlers of a network-accessible service of a cloud provider network;
a first host external to the network-accessible service, wherein the first host comprises one or more respective processors and memory configured to implement a networking manager and a client program of the service; and
one or more processors and memory configured to implement a stateless load balancer;
wherein the stateless load balancer is configured to:
select, for processing a first message directed to the network-accessible service from the client program, a particular request handler from the plurality of request handlers, wherein the particular request handler is selected using an algorithm whose input includes (a) one or more properties of the first message and (b) configuration information of the network-accessible service, available at the stateless load balancer when the first message is received at the stateless load balancer, wherein the configuration information includes (a) the number of request handlers included in the plurality of request handlers and (b) respective network addresses, within an isolated virtual network configured for the network-accessible service, of individual request handlers of the plurality of request handlers, and wherein, with respect to a given message provided as input to the algorithm, a change in the number of request handlers results in a change in a probability of selection of individual ones of the plurality of request handlers for the given message;
cause a network connection to be established between the client program and the particular request handler, wherein flow state information pertaining to messages transmitted via the network connection is not stored at the stateless load balancer; and
wherein the networking manager is configured to:
store flow state information pertaining to messages transmitted via the network connection, wherein the flow state information includes identification information of the particular request handler;
cause, using the identification information, a second message directed to the network-accessible service from the client program to be received at the particular request handler without using the stateless load balancer as an intermediary, wherein the second message is received at the particular request handler during a first time interval in which the configuration information of the network-accessible service has not changed relative to the configuration information which was used to select the particular request handler via the algorithm; and
cause, using the identification information, a third message directed to the network-accessible service from the client program to be received at the particular request handler without using the stateless load balancer as an intermediary, wherein the third message is received at the particular request handler during a second time interval in which, as a result of at least a change in the number of request handlers, the configuration information of the network-accessible service has changed relative to the configuration information which was used to select the particular request handler via the algorithm.