US 12,321,845 B1
Iterative attention-based neural network training and processing
Steven Dennis Flinn, Sugar Land, TX (US); and Naomi Felina Moneypenny, Bellevue, WA (US)
Assigned to Steven D. Flinn, Sugar Land, TX (US)
Filed by Steven D. Flinn, Sugar Land, TX (US)
Filed on Nov. 27, 2024, as Appl. No. 18/963,462.
Application 18/963,462 is a division of application No. 18/810,460, filed on Aug. 20, 2024.
Application 18/810,460 is a continuation of application No. 18/101,612, filed on Jan. 26, 2023, granted, now 12,223,404.
Application 18/101,612 is a continuation of application No. 16/660,908, filed on Oct. 23, 2019, granted, now 11,593,708, issued on Feb. 8, 2023.
Application 16/660,908 is a continuation of application No. 15/000,011, filed on Jan. 18, 2016, granted, now 10,510,018, issued on Dec. 17, 2019.
Application 15/000,011 is a continuation in part of application No. 14/816,439, filed on Aug. 3, 2015, abandoned.
This patent is subject to a terminal disclaimer.
Int. Cl. G06N 3/043 (2023.01); G06F 40/211 (2020.01); G06F 40/216 (2020.01); G06F 40/30 (2020.01); G06N 3/045 (2023.01); G06N 5/048 (2023.01)
CPC G06N 3/043 (2023.01) [G06F 40/211 (2020.01); G06F 40/216 (2020.01); G06F 40/30 (2020.01); G06N 3/045 (2023.01); G06N 5/048 (2013.01)] 30 Claims
OG exemplary drawing
 
1. A computer-implemented method, comprising:
at a system including one or more processors and one or more memories that is in communication with the one or more processors and that has one or more programs stored therein to be executed by the one or more processors:
causing access to a computer-implemented neural network trained on a training set including: images associated with corresponding syntactical elements to infer one or more syntactical elements corresponding to pixel patterns within the images, and training syntactical elements, including training natural language syntactical elements and training computer-executable code syntactical elements, that are utilized by computing hardware that processes the training syntactical elements to direct training attentions to representations of different subsets of the training syntactical elements;
causing access to a first plurality of syntactical elements that includes an entirety of a syntactical element portion of a prompt;
causing generation, based on at least one of: a position of each of the first plurality of syntactical elements of the entirety of the syntactical element portion of the prompt or a relationship other than position of at least one of the first plurality of syntactical elements with at least one other of the first plurality of syntactical elements, of a single matrix that represents the entirety of the syntactical element portion of the prompt;
utilizing first weights that reflect a relative importance or relevance of a plurality of relationships among the first plurality of syntactical elements, causing performance of a first prioritization, in connection with the single matrix that represents the entirety of the syntactical element portion of the prompt, of a first plurality of attentions each corresponding to different subsets of the first plurality of syntactical elements;
causing generation, based on at least a portion of the first prioritization and by application of the trained computer-implemented neural network, of a first plurality of probabilities, such that the trained computer-implemented neural network utilizes one or more hardware processors for the generation of the first plurality of probabilities;
utilizing second weights that reflect the relative importance or relevance of the plurality of relationships as updated based on the first plurality of probabilities, causing performance of a second prioritization of a second plurality of attentions;
causing generation, based on at least a portion of the second prioritization and by application of the trained computer-implemented neural network, of a second plurality of probabilities that are each associated with a corresponding subset of a plurality of items of content, such that the trained computer-implemented neural network utilizes the one or more hardware processors for the generation of the second plurality of probabilities;
causing a selection of one or more of the plurality of items of content, based on the second plurality of probabilities, where the selected one or more of the plurality of items of content includes at least one of: one or more images, one or more natural language syntactical elements, or one or more computer-executable code syntactical elements, based on the first plurality of syntactical elements; and
causing the selected one or more of the plurality of items of content to be sent to a user.