US 11,714,857 B2
Learning to select vocabularies for categorical features
Cong Li, Sunnyvale, CA (US); Jay Adams, San Carlos, CA (US); Manas Joglekar, Mountain View, CA (US); Pranav Khaitan, Mountain View, CA (US); Quoc V. Le, Sunnyvale, CA (US); and Mei Chen, Sunnyvale, CA (US)
Assigned to Google LLC, Mountain View, CA (US)
Filed by Google LLC, Mountain View, CA (US)
Filed on Dec. 7, 2022, as Appl. No. 18/76,662.
Application 18/076,662 is a continuation of application No. 16/878,912, filed on May 20, 2020, granted, now 11,537,664.
Claims priority of provisional application 62/852,200, filed on May 23, 2019.
Prior Publication US 2023/0146053 A1, May 11, 2023
Int. Cl. G06F 16/9035 (2019.01); G06F 40/242 (2020.01); G06F 11/34 (2006.01); G06N 20/00 (2019.01); G06N 3/08 (2023.01)
CPC G06F 16/9035 (2019.01) [G06F 11/3466 (2013.01); G06F 40/242 (2020.01); G06N 3/08 (2013.01); G06N 20/00 (2019.01)] 20 Claims
OG exemplary drawing
 
1. A method performed by one or more data processing apparatus for determining, for each of one or more categorical feature values, a respective embedding dimensionality of an embedding that should be generated for the categorical feature value during processing of inputs by a machine learning model having a plurality of machine learning model parameters, the method comprising:
generating, using a controller neural network having a plurality of controller parameters and in accordance with current values of the controller parameters, a batch of output sequences, each output sequence in the batch specifying, for each categorical feature value, a respective embedding dimensionality of an embedding that should be generated for the categorical feature value during processing of inputs by the machine learning model;
for each output sequence in the batch, determining a performance metric of the machine learning model on a machine learning task when the machine learning model generates embeddings of the categorical feature values with the embedding dimensionalities specified by the output sequence, wherein:
the machine learning model is configured to process an input that comprises one or more categorical feature values by performing operations comprising:
mapping each categorical feature value included in the input to an embedding having the corresponding embedding dimensionality specified by the output sequence; and
processing the embeddings to generate a machine learning model output for the input; and
using the performance metrics for the output sequences in the batch to adjust the current values of the controller parameters of the controller neural network.