	US 11,699,074 B2
	Training sequence generation neural networks using quality scores
Mohammad Norouzi, Sunnyvale, CA (US); William Chan, Markham (CA); and Sara Sabour Rouh Aghdam, Toronto (CA)
Assigned to Google LLC, Mountain View, CA (US)
Filed by Google LLC, Mountain View, CA (US)
Filed on Jan. 17, 2020, as Appl. No. 16/746,654.
Application 16/746,654 is a continuation of application No. 16/421,406, filed on May 23, 2019, granted, now 10,540,585.
Claims priority of provisional application 62/675,733, filed on May 23, 2018.
Prior Publication US 2020/0151567 A1, May 14, 2020
This patent is subject to a terminal disclaimer.
Int. Cl. G06N 3/08 (2006.01)

CPC G06N 3/08 (2013.01)

20 Claims

1. A method of training a neural network having a plurality of network parameters and being configured to map a system input to an output sequence comprising a plurality of system outputs, wherein the method comprises:

obtaining a batch of training examples, each training example comprising a training network input and, for each training network input, a ground truth output sequence;

for each of the training examples:

processing the training network input in the training example using the neural network and in accordance with current values of the network parameters to map the training network input to an output sequence comprising a respective system output from a vocabulary of possible system outputs at each of a plurality of output positions, comprising, for each of the plurality of output positions, generating a likelihood distribution over the possible system outputs in the vocabulary and selecting the system output at the output position using the likelihood distribution;

for each particular output position of the plurality of output positions in the output sequence:

identifying a prefix that includes the system outputs at positions before the particular output position in the output sequence,

for each possible system output in the vocabulary, determining a highest quality score that can be assigned to any candidate output sequence that includes the prefix followed by the possible system output and followed by any suffix of one or more system outputs, wherein the quality score measures a quality of a candidate output sequence relative to the ground truth output sequence,

generating a target likelihood distribution for the particular output position from the highest quality sores for the possible system outputs, and

determining an update to the current values of the network parameters based on a gradient with respect to the network parameters of an objective function that depends on a divergence between the target likelihood distribution for the particular output position and the likelihood distribution generated by the neural network for the particular output position; and

determining updated values of the network parameters from the updates for the particular output positions in the output sequences generated by the neural network for the batch of training examples.