CPC G06F 16/9035 (2019.01) [G06F 11/3466 (2013.01); G06F 40/242 (2020.01); G06N 3/08 (2013.01); G06N 20/00 (2019.01)] | 20 Claims |
1. A method performed by one or more data processing apparatus for determining, for each of one or more categorical feature values, a respective embedding dimensionality of an embedding that should be generated for the categorical feature value during processing of inputs by a machine learning model having a plurality of machine learning model parameters, the method comprising:
generating, using a controller neural network having a plurality of controller parameters and in accordance with current values of the controller parameters, a batch of output sequences, each output sequence in the batch specifying, for each categorical feature value, a respective embedding dimensionality of an embedding that should be generated for the categorical feature value during processing of inputs by the machine learning model;
for each output sequence in the batch, determining a performance metric of the machine learning model on a machine learning task when the machine learning model generates embeddings of the categorical feature values with the embedding dimensionalities specified by the output sequence, wherein:
the machine learning model is configured to process an input that comprises one or more categorical feature values by performing operations comprising:
mapping each categorical feature value included in the input to an embedding having the corresponding embedding dimensionality specified by the output sequence; and
processing the embeddings to generate a machine learning model output for the input; and
using the performance metrics for the output sequences in the batch to adjust the current values of the controller parameters of the controller neural network.
|