US 12,067,419 B1
Load balancing computing resources in an observability pipeline system
Dritan Bitincka, Edgewater, NJ (US); Ledion Bitincka, San Francisco, CA (US); Nicholas Robert Romito, Chicago, IL (US); and Clint Sharp, Oakland, CA (US)
Assigned to Cribl, Inc., San Francisco, CA (US)
Filed by Cribl, Inc., San Francisco, CA (US)
Filed on Jun. 27, 2023, as Appl. No. 18/342,427.
Application 18/342,427 is a continuation of application No. 17/346,881, filed on Jun. 14, 2021, granted, now 11,748,160.
This patent is subject to a terminal disclaimer.
Int. Cl. G06F 9/48 (2006.01); G06F 9/38 (2018.01); G06F 9/50 (2006.01)
CPC G06F 9/4887 (2013.01) [G06F 9/3842 (2013.01); G06F 9/505 (2013.01); G06F 9/5072 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A method of load balancing across computing resources in an observability pipeline system, the method comprising:
in an observability pipeline system deployed on one or more computer nodes operating in a leader role and a plurality of worker roles, identifying a plurality of computing jobs, each of the computing jobs comprising a plurality of computing tasks;
by operation of the leader role, dispatching the plurality of computing tasks to the plurality of worker roles, wherein dispatching the plurality of computing tasks comprises, iteratively:
identifying one of the plurality of worker roles as an available worker role;
identifying, from the plurality of computing jobs, one or more incomplete computing jobs comprising one or more computing tasks that have not been executed in the observability pipeline system;
selecting, from the one or more incomplete computing jobs, a computing job that has the least number of in-flight computing tasks currently being executed in the observability pipeline system;
identifying a next computing task from the selected computing job; and
dispatching the next computing task to the available worker role;
by operation of the worker roles in the observability pipeline system, executing the computing tasks dispatched to the respective worker roles.