US 12,236,321 B2
Batching techniques for handling unbalanced training data for a chatbot
Thanh Long Duong, Seabrook (AU); Mark Edward Johnson, Castle Cove (AU); Vishal Vishnoi, Redwood City, CA (US); Balakota Srinivas Vinnakota, Sunnyvale, CA (US); Yu-Heng Hong, Carlton (AU); and Elias Luqman Jalaluddin, Seattle, WA (US)
Assigned to Oracle International Corporation, Redwood Shores, CA (US)
Filed by Oracle International Corporation, Redwood Shores, CA (US)
Filed on Mar. 30, 2021, as Appl. No. 17/217,623.
Claims priority of provisional application 63/002,151, filed on Mar. 30, 2020.
Prior Publication US 2021/0304075 A1, Sep. 30, 2021
Int. Cl. G06N 20/00 (2019.01); G06F 16/906 (2019.01); G06F 18/22 (2023.01); G06F 18/2413 (2023.01); G06F 40/30 (2020.01); G10L 15/06 (2013.01); G10L 15/18 (2013.01); G10L 15/197 (2013.01); G10L 15/22 (2006.01)
CPC G06N 20/00 (2019.01) [G06F 16/906 (2019.01); G06F 18/22 (2023.01); G06F 18/24147 (2023.01); G06F 40/30 (2020.01); G10L 15/063 (2013.01); G10L 15/1815 (2013.01); G10L 15/197 (2013.01); G10L 15/22 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A method comprising:
storing, in a repository, a prediction model constructed as an intent classifier for one or more skills, the prediction model comprising a plurality of model parameters learned by a use of an objective function;
obtaining, by a processor, a plurality of utterances, wherein each utterance of the plurality of utterances corresponds to one of a plurality of intents, which are different from each other and are included in a set of intents, and wherein each intent of the plurality of intents corresponds to a skill of the one or more skills;
determining, by the processor, an intent distribution among the plurality of utterances as a plurality of intent proportions, each intent proportion of the plurality of intent proportions respectively corresponding to each intent, the determining comprising determining the plurality of intent proportions based on an overall number of intents in the set of intents and an individual number of each intent in the plurality of utterances;
generating, by the processor, a batch, the batch comprising a plurality of subsets of batch utterances selected from the plurality of utterances based on the plurality of intent proportions, wherein each subset of the plurality of subsets of batch utterances is selected for each intent and a number of utterances in the subset corresponds to the determined intent proportion for the intent;
accessing, by the processor, the prediction model stored in the repository;
training, by the processor, the prediction model on sets of training data from the batch, the training comprising:
iteratively inputting the sets of training data from the batch into the prediction model;
training the prediction model using the sets of training data by minimizing or maximizing the objective function, which measures a difference between predicted intents and ground truth intents, to cause the prediction model to learn relationships within the sets of training data that are used by the prediction model to generate the predicted intents, and
updating the plurality of model parameters;
and
generating, by the processor, a trained prediction model comprising the plurality of model parameters associated with the learned relationships,
wherein the training the prediction model using the sets of training data from the batch causes the trained prediction model to exhibit a minimized bias toward any particular intent, and
wherein the trained prediction model is configured to, based on an utterance provided as an input, predict an intent for the one or more skills, and
wherein the predicting the intent causes the processor to launch an interactive application corresponding to a skill among the one or more skills.