US 12,223,407 B2
Efficient configuration selection for automated machine learning
Chi Wang, Redmond, WA (US); Silu Huang, Urbana, IL (US); Surajit Chaudhuri, Redmond, WA (US); and Bolin Ding, Redmond, WA (US)
Assigned to Microsoft Technology Licensing, LLC, Redmond, WA (US)
Filed by Microsoft Technology Licensing, LLC, Redmond, WA (US)
Filed on Aug. 23, 2018, as Appl. No. 16/110,419.
Prior Publication US 2020/0065712 A1, Feb. 27, 2020
Int. Cl. G06N 20/20 (2019.01); G06F 9/445 (2018.01); G06N 20/00 (2019.01)
CPC G06N 20/20 (2019.01) [G06F 9/44505 (2013.01); G06N 20/00 (2019.01)] 20 Claims
OG exemplary drawing
 
1. One or more non-transitory machine-readable media storing instructions for execution by one or more hardware processors, execution of the instructions causing the one or more hardware processors to determine an approximate best machine-learning configuration among a set of machine-learning configurations by performing operations comprising:
selecting a machine-learning configuration within the set of machine-learning configurations for training and determining an associated sample size for training;
causing a training dataset to be sampled in accordance with the determined associated sample size to obtain a sampled training dataset;
causing the selected machine-learning configuration to be trained on the sampled training dataset to optimize a training value of a quality metric;
causing the trained machine-learning configuration to be tested on at least a sample of a test dataset to determine a test value of the quality metric;
estimating, based at least in part on the training value of the quality metric and the test value of the quality metric, a confidence interval providing upper and lower bounds for the quality metric, the upper bound being computed from the training value but not the test value, and the lower bound being computed from the test value but not the training value; and
pruning the set of machine-learning configurations based on comparisons between the estimated confidence interval of the trained selected machine-learning configuration and estimated confidence intervals of other machine-learning configurations within the set of machine-learning configurations.