US 12,462,798 B1
Evaluation of speech processing components
Shubham Srivastav, Natick, MA (US); James J Logan, Southborough, MA (US); Siyong Liang, Menlo Park, CA (US); Arjun R Athreya, Fairfield, CT (US); Parag Kurlawala, Bedford, MA (US); and Michael K Leung, Stoneham, MA (US)
Assigned to Amazon Technologies, Inc., Seattle, WA (US)
Filed by Amazon Technologies, Inc., Seattle, WA (US)
Filed on Sep. 27, 2021, as Appl. No. 17/485,897.
Int. Cl. G10L 15/22 (2006.01); G06F 40/295 (2020.01); G10L 13/02 (2013.01)
CPC G10L 15/22 (2013.01) [G06F 40/295 (2020.01); G10L 13/02 (2013.01); G10L 2015/225 (2013.01)] 17 Claims
OG exemplary drawing
 
1. A computer-implemented method comprising:
receiving, from a device, a task request to evaluate processing performed by speech processing components with respect to a first entity name and an intent, the first entity name being associated with a first entity identifier;
determining first text data representing a first machine-generated input including the first entity name and corresponding with the intent;
determining, using the first text data, audio data corresponding to synthesized speech representing the first machine-generated input;
determining automatic speech recognition (ASR) results data corresponding to the audio data;
determining first natural language understanding (NLU) results data corresponding to the ASR results data;
sending, to an entity resolution (ER) task component, a first command to perform ER processing with respect to the first NLU results data;
receiving, from the ER task component, first ER results data corresponding to the first NLU results data, the first ER results data including a second entity identifier;
determining, using the entity data and the first ER results data, that the first entity identifier is different than the second entity identifier; and
sending, to the device, first data representing processing of the first machine-generated input results in identification of an improper entity identifier.