	US 12,293,284 B2
	Meta cooperative training paradigms
Dingcheng Li, Sammamish, WA (US); Haiyan Yin, Singapore (SG); Xu Li, Beijing (CN); and Ping Li, Bellevue, WA (US)
Assigned to Baidu USA, LLC, Sunnyvale, CA (US)
Filed by Baidu USA, LLC, Sunnyvale, CA (US)
Filed on Dec. 29, 2020, as Appl. No. 17/136,054.
Claims priority of provisional application 62/970,638, filed on Feb. 5, 2020.
Prior Publication US 2021/0241099 A1, Aug. 5, 2021
Int. Cl. G06N 3/08 (2023.01); G06N 3/045 (2023.01)

CPC G06N 3/08 (2013.01) [G06N 3/045 (2023.01)]

21 Claims

1. A computer-implemented method for training a generator comprising:

responsive to a stop condition having not been reached, performing steps comprising:

sampling a set of data from a training data;

using a generator model, which comprises a set of generator parameter values, to generate a set of generated data;

computing an adversarial loss for the generator model using an adversarial training loss function;

determining a set of intermediate generator parameter values for the generator model using the adversarial loss and gradient descent;

using a set of data sampled from the training data as inputs:

into a second neural network model, which comprises a second neural network model set of parameter values, to obtain one or more output distributions from the second neural network model; and

into the generator model comprising the set of intermediate generator parameter values to obtain one or more output distributions from the generator model;

determining a meta gradient for a cooperate training loss that comprises comparing one or more output distributions from the second neural network model with one or more corresponding output distributions from the generator model;

updating a set of generator parameter values using an adversarial gradient, which is obtained using the adversarial loss for the generator model, and the meta gradient;

updating a set of discriminator parameter values for a discriminator model using an adversarial loss for the discriminator model; and

updating the second neural network model set of parameter values of the second neural network model using a cooperative training loss for the second neural network model; and

responsive to the stop condition having been reached, outputting the generator model, which comprises a final updated set of generator parameter values.