US 11,669,699 B2
Systems and methods for composed variational natural language generation
Congying Xia, Chicago, IL (US); and Caiming Xiong, Menlo Park, CA (US)
Assigned to saleforce.com, inc., San Francisco, CA (US)
Filed by salesforce.com, inc., San Francisco, CA (US)
Filed on Sep. 2, 2020, as Appl. No. 17/10,465.
Claims priority of provisional application 63/032,673, filed on May 31, 2020.
Prior Publication US 2021/0374603 A1, Dec. 2, 2021
This patent is subject to a terminal disclaimer.
Int. Cl. G06F 40/56 (2020.01); G06F 16/9032 (2019.01); G06F 40/284 (2020.01); G06N 20/00 (2019.01); G06F 40/30 (2020.01); G06N 7/01 (2023.01)
CPC G06F 40/56 (2020.01) [G06F 16/90332 (2019.01); G06F 40/284 (2020.01); G06F 40/30 (2020.01); G06N 7/01 (2023.01); G06N 20/00 (2019.01)] 20 Claims
OG exemplary drawing
 
1. A system for training a composed variational natural language generator implemented on a neural network, the system comprising:
an input interface configured to receive a training sequence of tokens including a first intent token and a second intent token;
a memory storing the composed variational natural language generator including an encoder and a decoder, and a plurality of processor-executable instructions; and
one or more processors executing the plurality-executable instructions to:
encode, by the encoder, the training sequence of tokens into an encoded sequence including a first latent variable corresponding to the first intent token and a second latent variable corresponding to the second intent token, wherein the first intent token and the second intent token are prevented from attending to each other during encoding;
compute an encoder loss based on a first conditional distribution of the first latent variable conditioned on the first intent token and a second conditional distribution of the second latent variable conditioned on the second intent token;
generate, by the decoder and from the encoded sequence, a reconstructed sequence of tokens;
compute a reconstruction loss based on a third conditional distribution of the reconstructed sequence of tokens conditioned on the first intent token, the second intent token, the first latent variable and the second latent variable;
train the composed variational natural language generator by updating parameters of the encoder and the decoder based at least in part on the encoder loss and the reconstruction loss; and
generate, by the trained composed variational natural language generator, a response output in response to an input of a testing utterance.