US 12,394,332 B2
Computer-automated systems and methods for using language models to generate text based on reading errors
Scott Sosso, Gibsonia, PA (US); Siyu Chen, Pittsburgh, PA (US); Ciara Figliuolo, Williamsburg, PA (US); Jack Mostow, Glennview, IL (US); and Marlies Goes, Pittsburgh, PA (US)
Assigned to SkyHigh Ventures, LLC, Gibsonia, PA (US); and Government of the United States, as Represented by the Secretary of the Air Force, Wright-Patterson AFB, OH (US)
Filed by SkyHigh Ventures, LLC, Gibsonia, PA (US)
Filed on May 9, 2024, as Appl. No. 18/659,230.
Claims priority of provisional application 63/465,842, filed on May 11, 2023.
Prior Publication US 2024/0379022 A1, Nov. 14, 2024
Int. Cl. G09B 17/00 (2006.01); G10L 13/08 (2013.01)
CPC G09B 17/003 (2013.01) [G10L 13/08 (2013.01)] 16 Claims
OG exemplary drawing
 
1. A method, performed by at least one computer processor executing computer program instructions stored on at least one non-transitory computer-readable medium, for generating personalized story text for a reader, the method comprising:
(A) generating, based on the reader's reading of text other than the personalized story text, a set of statistics representing performance of the reader at the word level and a set of statistics representing performance of the reader at the grapheme-phoneme level, the generating comprising:
performing phoneme-level automatic speech recognition on audio of the reader's speech to generate a phonetic transcript;
using a grapheme-phoneme dictionary to align graphemes from the text with phonemes from the phonetic transcript;
generating, based on the alignment of the graphemes with the phonemes:
the set of statistics representing the performance of the reader at the grapheme-phoneme level; and
the set of statistics representing the performance of the reader at the word level by determining which words in the text were read correctly by the reader;
wherein the set of statistics representing the performance of the reader at the grapheme-phoneme level includes, for each of a plurality of grapheme-phoneme pairs, data indicating whether the reader substituted a phoneme, inserted a phoneme, deleted a phoneme, or correctly uttered a phoneme when reading a corresponding grapheme;
(B) training a transformer-based neural network-based language model based on at least one of an age of the reader and a reading level of the reader, wherein the transformer-based neural network-based language model includes at least 1 billion parameters;
(C) generating, using the transformer-based neural network-based language model after training the transformer-based neural network-based language model, the personalized story text based on the set of statistics representing the performance of the reader at the word level and the set of statistics representing the performance of the reader at the grapheme-phoneme level, the generating comprising:
(C)(1) identifying a set of target words based on the set of statistics representing the performance of the reader at the word level and the set of statistics representing the performance of the reader at the grapheme-phoneme level;
(C)(2) generating a story text creation prompt based on the set of target words, wherein the story text creation prompt includes the set of target words and a topic of interest to the reader; and
(C)(3) providing the story text creation prompt to the transformer-based neural network-based language model to generate the personalized story text based on the story text creation prompt, wherein the personalized story text includes at least some of the target words.