| CPC G10L 15/22 (2013.01) [G06F 40/205 (2020.01); G06N 7/01 (2023.01); G10L 25/27 (2013.01); G10L 2015/225 (2013.01)] | 20 Claims |
|
1. A computer-implemented method comprising:
receiving data comprising a passage of text comprising a transcription of a spoken response by a language learner to a test question relating to a stimulus material, wherein the spoken response was produced after listening to the stimulus material or reading the stimulus material;
detecting, by at least one trained machine learning model, presences and absences of key points within the passage and location spans of the key points detected as being present within the passage, wherein:
each key point comprises a predefined piece of content from the stimulus material,
the location spans of the key points detected as being present within the passage tend to comprise one or more sentences, wherein each detected location span comprises at least five consecutive words, and
the at least one machine learning model is configurable to be trained using a corpus comprising of a plurality of passages annotated with at least a quality score or a location span for each of a plurality of key points, and
jointly optimize both location span detection and quality score prediction through automatic adjustment of relative weights associated with the location span detection and the quality score prediction, wherein each key point quality score is based on the relative weights associated with the location span detection and the quality score prediction, the relative weights having been jointly optimized during training;
scoring, by the at least one trained machine learning model, a quality of each of the key points detected as being present within the passage to result in a corresponding key point quality score;
determining, based on the detecting and the scoring, diagnostic feedback targeting content development skills of the language learner; and
providing data characterizing the determined diagnostic feedback.
|