US 11,750,552 B2
Systems and methods for real-time machine learning model training
Kyle Habermehl, Niwot, CO (US); Karen Lochbaum, Broomfield, CO (US); Robert Sanders, Marion, IA (US); Walter Denny Way, Pinehurst, NC (US); and Ryan Calme, Boulder, CO (US)
Assigned to PEARSON EDUCATION, INC., New York, NY (US)
Filed by Pearson Education, Inc., New York, NY (US)
Filed on Jun. 21, 2017, as Appl. No. 15/629,422.
Claims priority of provisional application 62/352,890, filed on Jun. 21, 2016.
Prior Publication US 2017/0364832 A1, Dec. 21, 2017
Int. Cl. G06N 20/00 (2019.01); H04L 51/226 (2022.01); G06Q 10/00 (2023.01); G06Q 10/101 (2023.01); G06Q 30/0201 (2023.01); H04L 51/02 (2022.01)
CPC H04L 51/226 (2022.05) [G06Q 10/00 (2013.01); G06Q 10/101 (2013.01); G06Q 30/0201 (2013.01); G06N 20/00 (2019.01); H04L 51/02 (2013.01)] 13 Claims
OG exemplary drawing
 
1. A system for training of an artificial intelligence (AI) model, the system comprising:
a first user device coupled to a network and configured to:
receive, from a user, via a user interface, a response to a question transmitted by at least one server coupled to the network; and
generate an evaluation for the response to the question, the evaluation identifying at least one feature corresponding to the response; and
a database coupled to the network and storing:
a plurality of AI models, at least one of the plurality of AI models associated with the received response to the question; and
a performance threshold associated with each of the plurality of AI models; and
a model status identifier;
the at least one server comprising a computing device coupled to the network and being configured to:
receive an evaluated response communication comprising the response and the evaluation from the first user device;
identify the AI model of the plurality of AI models corresponding to the response based on information in the evaluated response communication;
train the AI model in real-time with the received evaluated response communication, wherein the training comprises:
upon receipt of the evaluated response communication, applying the response to the AI model to identify at least one feature of the response and correlating the at least one feature identified by the AI model to the at least one feature in the received evaluation of the response;
evaluating the AI model against a performance threshold to determine whether the AI model has converged; and
responsive to a determination that the AI model exceeds the performance threshold, updating the training model status identifier associated with the AI model indicating that training of the AI model is completed, wherein the AI model is trained without first identifying a threshold number of training data sets to train the model.