US 12,032,464 B2
Optimization of parameters of a system, product, or process
Daniel Reuben Golovin, Pittsburgh, PA (US); Benjamin Solnik, Pittsburgh, PA (US); Subhodeep Moitra, Pittsburgh, PA (US); and David W. Sculley, II, Cambridge, MA (US)
Assigned to GOOGLE LLC, Mountain View, CA (US)
Appl. No. 16/495,295
Filed by Google LLC, Mountain View, CA (US)
PCT Filed Jun. 2, 2017, PCT No. PCT/US2017/035635
§ 371(c)(1), (2) Date Sep. 18, 2019,
PCT Pub. No. WO2018/222203, PCT Pub. Date Dec. 6, 2018.
Prior Publication US 2020/0111018 A1, Apr. 9, 2020
This patent is subject to a terminal disclaimer.
Int. Cl. G06F 11/30 (2006.01); G06F 11/34 (2006.01); G06N 5/01 (2023.01); G06N 7/01 (2023.01); G06N 20/00 (2019.01)
CPC G06F 11/3409 (2013.01) [G06F 11/3006 (2013.01); G06N 5/01 (2023.01); G06N 7/01 (2023.01); G06N 20/00 (2019.01)] 20 Claims
OG exemplary drawing
 
1. A computer-implemented method for use in optimization of parameters of a system, product, or process, the method comprising:
establishing, by one or more computing devices, an optimization procedure for a system, product, or process, wherein the system, product, or process has an evaluable performance that is dependent on values of one or more adjustable parameters;
receiving, by the one or more computing devices, one or more prior evaluations of performance of the system, product, or process, the one or more prior evaluations being respectively associated with one or more prior variants of the system, product, or process, the one or more prior variants each being defined by a set of values for the one or more adjustable parameters;
utilizing, by the one or more computing devices, an optimization algorithm to generate a suggested variant based at least in part on the one or more prior evaluations of performance and the associated set of values, the suggested variant being defined by a suggested set of values for the one or more adjustable parameters;
receiving, by the one or more computing devices, one or more intermediate evaluations of performance of the suggested variant, the intermediate evaluations having been obtained from on an ongoing evaluation of the suggested variant;
performing, by the one or more computing devices, non-parametric regression, based on the intermediate evaluations and the prior evaluations, to determine whether to perform early-stopping of the ongoing evaluation of the suggested variant, wherein early-stopping is to be performed when the ongoing evaluation is determined to fall below a performance threshold; and
in response to determining that early-stopping is to be performed, causing, by the one or more computing devices, early-stopping to be performed in respect of the ongoing evaluation or providing an indication that early-stopping should be performed.