US 12,254,678 B2
Training a neural network using outputs of a corruption neural network
Dan-Andrei Calian, London (GB); Sven Adrian Gowal, Cambridge (GB); Timothy Arthur Mann, Harpenden (GB); and András György, London (GB)
Assigned to DeepMind Technologies Limited, London (GB)
Filed by DeepMind Technologies Limited, London (GB)
Filed on Apr. 1, 2022, as Appl. No. 17/711,951.
Prior Publication US 2023/0316729 A1, Oct. 5, 2023
Int. Cl. G06V 10/774 (2022.01); G06V 10/776 (2022.01); G06V 10/82 (2022.01)
CPC G06V 10/7747 (2022.01) [G06V 10/776 (2022.01); G06V 10/82 (2022.01)] 20 Claims
OG exemplary drawing
 
1. A method comprising:
receiving a network input; and
processing the network input using a first neural network to generate an output for a machine learning task, wherein the first neural network comprises a plurality of first neural network parameters and has been trained on training examples, the training comprising:
receiving the training examples for training the first neural network, wherein each training example comprises a training network input and a reference output;
for each training iteration in a set of training iterations:
generating, for each training example of a set of the training examples and using a corruption neural network, a respective corrupted network input for the training network input in the training example, wherein the corruption neural network has a plurality of corruption neural network parameters and a plurality of perturbation parameters;
updating, based on the respective corrupted network inputs, the plurality of perturbation parameters of the corruption neural network using a first objective function;
generating, for each training example of the set of training examples and using the corruption neural network with the updated perturbation parameters, a respective updated corrupted network input for the training network input in the training example; and
generating, by processing the respective updated corrupted network inputs using at least the first neural network, corresponding respective network outputs; and
for each training example in the set of training examples, based on (i) the corresponding respective network output for the respective updated corrupted network input and (ii) the reference output in the training example, updating the plurality of first neural network parameters using a second objective function.