US 11,689,466 B2
Distributed processing system throttling using a timestamp
Samuel Green McVeety, Seattle, WA (US); and Vyacheslav Alekseyevich Chernyak, Seattle, WA (US)
Assigned to Google Inc., Mountain View, CA (US)
Filed by Google LLC, Mountain View, CA (US)
Filed on Nov. 4, 2021, as Appl. No. 17/519,017.
Application 17/519,017 is a continuation of application No. 16/434,488, filed on Jun. 7, 2019, granted, now 11,171,872, issued on Nov. 9, 2021.
Application 16/434,488 is a continuation of application No. 15/443,518, filed on Feb. 27, 2017, granted, now 10,333,852.
Application 15/443,518 is a continuation of application No. 14/023,208, filed on Sep. 10, 2013, granted, now 9,621,636.
Prior Publication US 2022/0060420 A1, Feb. 24, 2022
This patent is subject to a terminal disclaimer.
Int. Cl. H04L 47/22 (2022.01); H04L 67/10 (2022.01); G06F 9/50 (2006.01); H04L 67/60 (2022.01); G06F 16/2458 (2019.01); G06F 16/951 (2019.01); H04L 43/16 (2022.01)
CPC H04L 47/22 (2013.01) [G06F 9/5083 (2013.01); G06F 16/2477 (2019.01); G06F 16/951 (2019.01); H04L 43/16 (2013.01); H04L 67/10 (2013.01); H04L 67/60 (2022.05)] 20 Claims
OG exemplary drawing
 
1. A method for data processing a set of records by a distributed processing system, the method comprising:
determining, by one or more processors, a first agent progress in processing the set of records, wherein the first agent progress is based on a first time difference between a first timestamp of an oldest record of the set of records processed by a first agent process and entered into a queue for a second agent process and a current system time, wherein the second agent process is a dependent agent process of the first agent process and processes records that have first been processed by the first agent process;
determining, by the one or more processors, a second agent progress in processing the set of records, wherein the second agent progress is based on a second time difference between a second timestamp of an oldest record of the set of records processed by the second agent process and the current system time, wherein the first timestamp of the oldest record of the set of records processed by the first agent process and the second timestamp of the oldest record of the set of records processed by the second agent process each correspond a time the respective oldest record was received by the distributed processing system;
determining, by the one or more processors, a difference between the first agent progress and the second agent progress; and
throttling, by the one or more processors, performance of the first agent process, based on the difference between the first agent progress and the second agent progress meeting a threshold measure.