US 12,223,404 B2
Iterative attention-based neural network training and processing
Steven Dennis Flinn, Sugar Land, TX (US); and Naomi Felina Moneypenny, Bellevue, WA (US)
Filed by Steven D. Flinn, Sugar Land, TX (US)
Filed on Jan. 26, 2023, as Appl. No. 18/101,612.
Application 18/101,612 is a continuation of application No. 16/660,908, filed on Oct. 23, 2019, granted, now 11,593,708.
Application 16/660,908 is a continuation of application No. 15/000,011, filed on Jan. 18, 2016, granted, now 10,510,018, issued on Dec. 17, 2019.
Application 15/000,011 is a continuation in part of application No. 14/816,439, filed on Aug. 3, 2015, abandoned.
Application 14/816,439 is a continuation in part of application No. 14/497,645, filed on Sep. 26, 2014, abandoned.
Claims priority of provisional application 61/929,432, filed on Jan. 20, 2014.
Claims priority of provisional application 61/884,224, filed on Sep. 30, 2013.
Prior Publication US 2024/0005205 A1, Jan. 4, 2024
Int. Cl. G06N 20/00 (2019.01); G06F 40/211 (2020.01); G06F 40/216 (2020.01); G06F 40/30 (2020.01); G06N 3/045 (2023.01); G06N 5/048 (2023.01); G06N 3/02 (2006.01)
CPC G06N 20/00 (2019.01) [G06F 40/211 (2020.01); G06F 40/216 (2020.01); G06F 40/30 (2020.01); G06N 3/045 (2023.01); G06N 5/048 (2013.01); G06N 3/02 (2013.01)] 375 Claims
OG exemplary drawing
 
10. A system, comprising:
one or more processors;
one or more memories in communication with the one or more processors; and
one or more programs, wherein the one or more programs are stored in the one or more memories to be executed by the one or more processors, the one or more programs including instructions for causing:
access to a computer-implemented neural network that is trained, the trained computer-implemented neural network being of a type other than a recurrent neural network;
access to information including a first plurality of syntactical elements;
generation, by application of the trained computer-implemented neural network, of a first plurality of probabilities that is associated with the first plurality of syntactical elements;
a first attention to be directed to a representation of a first subset of the first plurality of syntactical elements based on the first plurality of probabilities, for use by the trained computer-implemented neural network;
generation, by application of the trained computer-implemented neural network, of a second plurality of probabilities based on the first attention and the first plurality of probabilities generated by the application of the trained computer-implemented neural network;
a second attention to be directed to a representation of a second subset of the first plurality of syntactical elements based on the second plurality of probabilities, for use by the trained computer-implemented neural network;
generation, by application of the trained computer-implemented neural network, of a third plurality of probabilities based on the second attention and the second plurality of probabilities generated by the application of the trained computer-implemented neural network;
generation, by application of the trained computer-implemented neural network, of a second plurality of syntactical elements based on the third plurality of probabilities generated by the application of the trained computer-implemented neural network; and
a communication to be sent to a user.