US 11,790,894 B2
Machine learning based models for automatic conversations in online systems
Yixin Mao, San Francisco, CA (US); Zachary Alexander, Berkeley, CA (US); Victor Winslow Yee, Berkeley, CA (US); Joseph R. Zeimen, San Francisco, CA (US); Na Cheng, Bellevue, WA (US); Chien-Sheng Wu, Mountain View, CA (US); Wenhao Liu, Redwood City, CA (US); and Caiming Xiong, Menlo Park, CA (US)
Assigned to Salesforce, Inc., San Francisco, CA (US)
Filed by salesforce.com, inc., San Francisco, CA (US)
Filed on Mar. 15, 2021, as Appl. No. 17/202,077.
Prior Publication US 2022/0293094 A1, Sep. 15, 2022
Int. Cl. G10L 15/16 (2006.01); H04L 51/02 (2022.01); G10L 15/06 (2013.01); G10L 15/08 (2006.01); G10L 15/22 (2006.01); G06F 16/33 (2019.01); G06F 40/56 (2020.01)
CPC G10L 15/16 (2013.01) [G10L 15/063 (2013.01); G10L 15/08 (2013.01); G10L 15/22 (2013.01); H04L 51/02 (2013.01); G06F 16/3344 (2019.01); G06F 40/56 (2020.01)] 20 Claims
OG exemplary drawing
 
1. A computer-implemented method, comprising:
receiving a sequence of utterances representing a context in an online conversation, the sequence comprising one or more utterances;
generating a context vector by:
generating one or more utterance vectors by providing the one or more utterances to an utterance vector generator comprising a multilingual neural network;
generating one or more encoded utterance vectors comprising, for each of the one or more utterance vector: adding a user embedding of the user performing the conversation to the utterance vector to generate a combined vector and providing the combined vector as input to a transformer encoder; and
aggregating the one or more encoded utterance vectors to obtain a context vector;
accessing one or more candidate responses for the online conversation;
determining distance scores for the candidate responses by:
generating a response vector for the candidate response using the utterance vector generator; and
combining the response vector with the context vector to obtain the distance score;
selecting a response based on the distance scores of the candidate responses; and
providing the selected response to the user via the online conversation.