US 12,073,187 B2
Automatic evaluation of natural language text generated based on structured data
Markus Freitag, Sunnyvale, CA (US); and Howard Scott Roy, Palo Alto, CA (US)
Assigned to GOOGLE LLC, Mountain View, CA (US)
Appl. No. 17/608,616
Filed by GOOGLE LLC, Mountain View, CA (US)
PCT Filed Aug. 22, 2019, PCT No. PCT/US2019/047719
§ 371(c)(1), (2) Date Nov. 3, 2021,
PCT Pub. No. WO2020/231453, PCT Pub. Date Nov. 19, 2020.
Claims priority of provisional application 62/847,078, filed on May 13, 2019.
Prior Publication US 2022/0215184 A1, Jul. 7, 2022
Int. Cl. G06F 40/56 (2020.01); G06F 40/226 (2020.01); G06F 40/30 (2020.01)
CPC G06F 40/56 (2020.01) [G06F 40/226 (2020.01); G06F 40/30 (2020.01)] 20 Claims
OG exemplary drawing
 
14. A method implemented by one or more processors, the method comprising:
receiving a plurality of instances of natural language text based on a set of structured data, wherein each instance of natural language text is generated by processing the set of structured data using at least one natural language generation model;
processing the plurality of instances of natural language text using an alignments and language model (ALM) to automatically generate a plurality of corresponding ALM scores, wherein processing each instance of natural language text using the ALM comprises:
processing the instance of natural language text using a fluency model portion of the ALM to generate a fluency score, wherein the fluency score is an evaluation of the fluency and the grammar of the instance of natural language text;
processing the instance of natural language text and the set of structured data using a semantics model portion of the ALM to generate a semantics score, wherein the semantics score evaluates the content of the instance of natural language text based on the corresponding set of structured data;
determining the corresponding ALM score based on the fluency score and the semantics score;
selecting one or more instances of natural language text from the plurality of instances of natural language text based on the corresponding ALM scores;
for each of the one or more selected instances of natural language text, determining a corresponding audio waveform based on the instance of natural language text; and
causing a client device to render output based on each of the corresponding audio waveforms corresponding to the one or more selected instances of natural language text.