| CPC G09B 7/06 (2013.01) [G06F 16/212 (2019.01); G06N 20/20 (2019.01)] | 18 Claims |
|
1. A computer-implemented method comprising:
using an application server computer that is communicatively coupled to a database via a telecommunication network, the database comprising a digitally stored relational table schema storing a plurality of term sets, each of the term sets comprising a plurality of terms, the application server computer hosting an application program programmed with a plurality of distractor selection algorithms;
using the application server computer, obtaining labeled answer logging data from a plurality of answer signals received from a plurality of client computers based on answers to prior questions electronically presented by the plurality of client computers;
using the application server computer, executing each of the distractor algorithms to access a particular term from among the plurality of the terms as an input and to generate a plurality of corresponding first interim sets of candidate distractor terms, for the particular term, as an output;
using the application server computer, executing a training stage of a within-set machine learning model over a first training dataset of the labeled answer logging data;
using the application server computer, executing a first inference stage of the within-set machine learning model over the same particular term as input to generate classification output comprising a second interim set of candidate distractor terms, for the particular term;
using the application server computer, executing a second inference stage of a cross-algorithm ranking machine learning model over the plurality of corresponding first interim sets and the second interim set to generate a top-N set of best candidate distractors;
using the application server computer, digitally storing the top-N set of best candidate distractors in a table of the relational table schema in association with the particular term;
using the application server computer, formatting the top-N set of best candidate distractors with the particular term as a multiple-choice question set and transmitting, to an additional client computer, presentation instructions which when rendered using the additional client computer cause presenting a user interface comprising the particular term as a prompt and the top-N set of best candidate distractors as available answers.
|