US 11,726,893 B2
System for automatically evaluating a change in a large population of processing jobs
Mariana Sharp, Mountain View, CA (US); Craig D. Chambers, Seattle, WA (US); Jelena Pjesivac-Grbovic, Mountain View, CA (US); Kym Hines, Kenmore, WA (US); Sunghwan Ihm, Palo Alto, CA (US); Danijela Mijailovic, Belgrade (RS); and Ben Walters, Santa Clara, CA (US)
Assigned to Google LLC, Mountain View, CA (US)
Filed by Google LLC, Mountain View, CA (US)
Filed on Sep. 2, 2021, as Appl. No. 17/465,397.
Application 17/465,397 is a continuation of application No. 16/392,892, filed on Apr. 24, 2019, granted, now 11,138,087.
Prior Publication US 2021/0397534 A1, Dec. 23, 2021
This patent is subject to a terminal disclaimer.
Int. Cl. G06F 11/34 (2006.01); G06F 11/30 (2006.01); G06F 9/46 (2006.01); G06F 9/445 (2018.01)
CPC G06F 11/3409 (2013.01) [G06F 9/46 (2013.01); G06F 11/3006 (2013.01); G06F 9/44505 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A method for evaluating the rollout of a change to a data processing framework (DPF), the method comprising:
processing, by the one or more processors, a first set of jobs using the DPF with a proposed change to the DPF enabled to generate a first output set;
processing, by the one or more processors, a second set of jobs using the DPF with the proposed change to the DPF disabled to generate a second output set;
filtering, by the one or more processors, the first output set and the second output set based on one or more filtering criteria;
determining, by the one or more processors, a first performance cost value for processing the first set of jobs based on the filtered first output set;
determining, by the one or more processors, a second performance cost value for processing the second set of jobs based on the filtered second output set; and
determining, by the one or more processors, based on the first performance cost value and the second performance cost value, whether the proposed change is beneficial to the DPF.