CPC G06F 40/58 (2020.01) [G06F 40/47 (2020.01); G06N 3/02 (2013.01); G06N 3/04 (2013.01)] | 20 Claims |
1. A computer-implemented method for neural machine translation comprising:
encoding, using an encoder, a source expression into a sequence of encoder hidden states, the source expression comprising one or more words;
generating, using a decoder, decoder hidden states conditioned at least on the sequence of encoder hidden states, the decoder uses beam search comprising multiple search steps for target words of a target model, the beam search has a variable beam width and a variable search scope within a target word vocabulary during the multiple search steps, in which the variable search scope within the target word vocabulary is constrained for at least one or more of search steps of the multiple search steps to exclude target words that cannot appear at a current position in a target expression for a current search step based upon the target model or to include target words that can appear at the current position in the target expression for the current search step based upon the target model;
producing, from an output prediction layer, a probability distribution among target words searched for each of the multiple search steps; and
generating, using the probability distribution for each of the multiple search steps, one or more target expressions corresponding to the source expression after the multiple search steps, each of one or more the target expressions comprising one or more target words.
|