| CPC G06F 16/24573 (2019.01) [G06F 40/284 (2020.01)] | 20 Claims |
|
1. A method performed by one or more computers and for generating an output sequence that comprises a plurality of vocabulary of tokens that are each selected from a vocabulary of tokens that comprises a set of vocabulary tokens and an end of sequence token, the method comprising:
obtaining a network input;
initializing beam data specifying a set of k candidate output sequences and a respective total score for each of the candidate output sequences, wherein k is an integer greater than or equal to one;
updating the beam data at each of a plurality of decoding steps, the updating comprising, at each decoding step:
processing each candidate output sequence in the beam data as of the decoding step using an auto-regressive neural network that is conditioned on the network input to generate a score distribution that comprises a respective score for each token in the vocabulary;
identifying a plurality of expanded sequences, wherein each expanded sequence corresponds to a respective candidate output sequence and includes the tokens from the corresponding candidate output sequence followed by a respective additional token from the vocabulary;
generating, for each expanded sequence, a respective backwards-looking score based on the respective score for the respective additional token in the score distribution generated for the corresponding candidate output sequence by the auto-regressive neural network;
generating, for each expanded sequence and using the auto-regressive neural network, a respective forward-looking score that estimates a score for a highest-scoring partial output sequence that has the expanded sequence as a prefix;
computing, for each expanded sequence, the respective total score from the respective forward-looking score for the expanded sequence and the respective backwards-looking score for the expanded sequence; and
updating the set of k candidate output sequences using the respective total scores for the expanded sequences.
|