US 12,235,790 B2
Deep neural network-based decision network
Alexander Rosenberg Johansen, San Francisco, CA (US); Bryan McCann, San Francisco, CA (US); James Bradbury, San Francisco, CA (US); and Richard Socher, Menlo Park, CA (US)
Assigned to Salesforce, Inc., San Francisco, CA (US)
Filed by salesforce.com, inc., San Francisco, CA (US)
Filed on Feb. 11, 2022, as Appl. No. 17/670,368.
Application 17/670,368 is a continuation of application No. 15/853,570, filed on Dec. 22, 2017, granted, now 11,250,311.
Claims priority of provisional application 62/471,934, filed on Mar. 15, 2017.
Prior Publication US 2022/0164635 A1, May 26, 2022
Int. Cl. G06N 20/00 (2019.01); G06F 15/76 (2006.01); G06F 18/21 (2023.01); G06F 18/24 (2023.01); G06F 18/241 (2023.01); G06F 18/2413 (2023.01); G06F 40/169 (2020.01); G06F 40/30 (2020.01); G06N 3/04 (2023.01); G06N 3/044 (2023.01); G06N 3/045 (2023.01); G06N 3/047 (2023.01); G06N 3/048 (2023.01); G06N 3/08 (2023.01); G06N 3/084 (2023.01); G06N 5/04 (2023.01); G06V 10/764 (2022.01); G06V 10/776 (2022.01); G06V 10/82 (2022.01)
CPC G06F 15/76 (2013.01) [G06F 18/217 (2023.01); G06F 18/24 (2023.01); G06F 18/241 (2023.01); G06F 18/24133 (2023.01); G06F 40/169 (2020.01); G06F 40/30 (2020.01); G06N 3/044 (2023.01); G06N 3/045 (2023.01); G06N 3/047 (2023.01); G06N 3/048 (2023.01); G06N 3/084 (2013.01); G06N 5/04 (2013.01); G06N 20/00 (2019.01); G06V 10/764 (2022.01); G06V 10/776 (2022.01); G06V 10/82 (2022.01)] 18 Claims
OG exemplary drawing
 
1. A system for performing a selection task using a decision neural network-based classifier, comprising: a memory storing a plurality of processor-executable instructions for training and operating the decision neural network-based classifier; and a processor that reads the plurality of processor-executable instructions from the memory to perform operations comprising: constructing a decision training set for the decision neural network-based classifier by: identifying, based on a confusion matrix generated from performance of a non-recurrent network-based classifier and a recurrent neural network-based classifier, a first subset of inputs inferred by the recurrent neural network-based classifier and a second subset of inputs comprising inputs not in the first subset, and including, in the decision training set, the first subset of inputs labeled with a first model class label identifying the recurrent neural network-based classifier and the second subset of inputs labeled with a second model class label identifying the non-recurrent neural network-based classifier; generating, by the decision neural network based classifier, a plurality of selection outputs in response to inputs from the decision training set; computing a training objective by comparing the plurality of selection outputs with labels from the first subset and the second subset; and updating the decision neural network based classifier based on the training objective wherein the operations further comprise: training the non-recurrent and recurrent neural network-based classifiers to perform the machine classification task using a training set, the training set comprising training inputs annotated with task class labels defined for the machine classification task; using the trained non-recurrent and recurrent neural network-based classifiers to perform the machine classification task on a validation set, the validation set comprising validation inputs annotated with the task class labels; and
training the decision neural network-based classifier using the decision training set to output probabilities for the first and second model class labels on an input-by-input basis, the output probabilities specifying respective likelihoods of selecting the trained recurrent neural network-based classifier and the trained non-recurrent neural network-based classifier.