US 12,321,707 B2
Multi-turn dialogue response generation via mutual information maximization
Oluwatobi Olabiyi, Arlington, VA (US); Zachary Kulis, Washington, DC (US); and Erik T. Mueller, Chevy Chase, MD (US)
Assigned to Capital One Services, LLC, McLean, VA (US)
Filed by Capital One Services, LLC, McLean, VA (US)
Filed on Sep. 22, 2022, as Appl. No. 17/950,732.
Application 17/950,732 is a continuation of application No. 16/935,717, filed on Jul. 22, 2020, granted, now 11,487,954.
Claims priority of provisional application 62/877,076, filed on Jul. 22, 2019.
Prior Publication US 2023/0021852 A1, Jan. 26, 2023
This patent is subject to a terminal disclaimer.
Int. Cl. G06F 40/30 (2020.01); G06F 18/21 (2023.01); G06F 18/214 (2023.01); G06F 40/284 (2020.01); G06F 40/35 (2020.01); G06F 40/56 (2020.01); G06N 3/049 (2023.01); G06N 20/00 (2019.01); G10L 15/06 (2013.01); G10L 15/16 (2006.01); G10L 15/22 (2006.01)
CPC G06F 40/30 (2020.01) [G06F 18/2148 (2023.01); G06F 18/217 (2023.01); G06F 40/284 (2020.01); G06F 40/35 (2020.01); G06F 40/56 (2020.01); G06N 3/049 (2013.01); G06N 20/00 (2019.01); G10L 15/063 (2013.01); G10L 15/16 (2013.01); G10L 15/22 (2013.01); G10L 2015/0631 (2013.01); G10L 2015/228 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A computer-implemented method comprising:
receiving, from a user device, user input data comprising multi-turn dialogs;
training, by a computing device and using a plurality of training sequences, a model having a sequence to sequence network architecture and comprising an encoder and a decoder, wherein each training sequence comprises an encoder sequence and a decoder sequence, and wherein training the model comprises:
generating, for each training sequence of the plurality of training sequences, a bidirectional encoding comprising a forward encoding based on the encoder sequence of the training sequence and a backward encoding based on the decoder sequence of the training sequence; and
for each bidirectional encoding:
padding the encoder sequence of the bidirectional encoding with an informative padding comprising a random sampling of encoded tokens from the plurality of training sequences, and wherein the informative padding further comprises contextual information added to the encoder sequence to reduce syntactic redundancy across conversation turns;
prepending a start of sequence token to the decoder sequence of the bidirectional encoding;
appending an end of sequence token to the decoder sequence of the bidirectional encoding;
training the encoder based on inputting, to the encoder, the encoder sequence of the bidirectional encoding; and
training the decoder based on inputting, to the decoder, the decoder sequence of the bidirectional encoding;
generating, by the computing device, based on input data and using the trained model, an output sequence; and
outputting, to the user device and based on the output sequence, an automated response to the user input data.