US 11,838,357 B1
Incremental stream switching in distributed systems
Ankit Kumar, Bellevue, WA (US); Dan Popick, Seattle, WA (US); Sandesh Sanjeev, Seattle, WA (US); Meng Li, Seattle, WA (US); and David Michael Hulme, Seattle, WA (US)
Assigned to Amazon Technologies, Inc., Seattle, WA (US)
Filed by Amazon Technologies, Inc., Seattle, WA (US)
Filed on Jul. 29, 2021, as Appl. No. 17/388,660.
Int. Cl. H04L 67/1025 (2022.01); H04L 67/1014 (2022.01); H04L 67/1029 (2022.01)
CPC H04L 67/1014 (2013.01) [H04L 67/1025 (2013.01); H04L 67/1029 (2013.01)] 20 Claims
OG exemplary drawing
 
17. A computer-implemented method comprising:
obtaining, by a process executed by an electronic device, a plurality of events in a first event stream;
determining, by the process, that a first event of the plurality of events in the first event stream is a flip event indicating that the process is to switch from the first event stream to a second event stream that is different than the first event stream, the flip event including an identifier of a second event within the second event stream, wherein the flip event further includes an identifier of a third event within the first stream that is later in sequence than the first event;
after the determining that the first event is a flip event and prior to switching from the first event stream to the second event stream, processing an additional one or more events of the plurality of events of the first event stream until reaching or processing the third event, wherein the processing of the one or more events comprises updating, by the process, a data structure with entries including identifiers of other events of the second event stream corresponding to the one or more events of the first event stream;
switching, by the process, from the first event stream to the second event stream, the switching including obtaining a second plurality of events of the second event stream based at least in part on use of the identifier of the second event; and
after the switching from the first event stream to the second event stream, processing the second plurality of events, the processing including performing lookups into the data structure to determine which of the second plurality of events have already been processed, wherein processing the second plurality of events includes:
determining, by the process based on use of the data structure, that a lookup stopping condition has been satisfied; and
stopping the performing of lookups with the data structure during processing of additional events of the second event stream.