US 12,293,270 B1
Iterative attention-based neural network training and processing
Steven Dennis Flinn, Sugar Land, TX (US); and Naomi Felina Moneypenny, Bellevue, WA (US)
Assigned to Steven D. Flinn, Sugar Land, TX (US)
Filed by Steven D Flinn, Brenham, TX (US)
Filed on Aug. 20, 2024, as Appl. No. 18/810,464.
Application 18/810,464 is a continuation of application No. 18/101,612, filed on Jan. 26, 2023.
Application 18/101,612 is a continuation of application No. 16/660,908, filed on Oct. 23, 2019, granted, now 11,593,708, issued on Feb. 8, 2023.
Application 16/660,908 is a continuation of application No. 15/000,011, filed on Jan. 18, 2016, granted, now 10,510,018, issued on Dec. 17, 2019.
Application 15/000,011 is a continuation in part of application No. 14/816,439, filed on Aug. 3, 2015, abandoned.
This patent is subject to a terminal disclaimer.
Int. Cl. G06N 20/00 (2019.01); G06F 40/211 (2020.01); G06F 40/216 (2020.01); G06F 40/30 (2020.01); G06N 3/02 (2006.01); G06N 3/045 (2023.01); G06N 5/048 (2023.01)
CPC G06N 20/00 (2019.01) [G06F 40/211 (2020.01); G06F 40/216 (2020.01); G06F 40/30 (2020.01); G06N 3/045 (2023.01); G06N 5/048 (2013.01); G06N 3/02 (2013.01)] 224 Claims
OG exemplary drawing
 
1. A system, comprising:
one or more processors;
one or more memories in communication with the one or more processors; and
one or more programs, wherein the one or more programs are stored in the one or more memories and configured to be executed by the one or more processors, the one or more programs including instructions for causing:
training of a computer-implemented non-recurrent neural network on a training set including a first plurality of syntactical elements, by utilizing computing hardware that processes the first plurality of syntactical elements to direct a plurality of attentions to representations of different subsets of the first plurality of syntactical elements;
access to a second plurality of syntactical elements;
a first attention of the trained computer-implemented non-recurrent neural network to be directed to a representation of a first subset of the second plurality of syntactical elements;
a second attention of the trained computer-implemented non-recurrent neural network to be directed to a representation of a second subset of the second plurality of syntactical elements;
generation, by application of the trained computer-implemented non-recurrent neural network, of a plurality of probabilities that are each associated with a corresponding subset of a third plurality of syntactical elements, based on the second attention of the trained computer-implemented non-recurrent neural network;
a selection of one or more of the third plurality of syntactical elements based on the plurality of probabilities; and
the selected one or more of the third plurality of syntactical elements to be sent to a user.