US 11,996,101 B2
Streaming action fulfillment based on partial hypotheses
Matthew Sharifi, Kilchberg (CH); and Victor Carbune, Zürich (CH)
Assigned to Google LLC, Mountain View, CA (US)
Filed by Google LLC, Mountain View, CA (US)
Filed on Jan. 27, 2023, as Appl. No. 18/160,342.
Application 18/160,342 is a continuation of application No. 17/247,334, filed on Dec. 8, 2020, granted, now 11,587,568.
Prior Publication US 2023/0169976 A1, Jun. 1, 2023
Int. Cl. G10L 15/22 (2006.01); G06F 3/16 (2006.01); G10L 15/07 (2013.01); G10L 15/08 (2006.01)
CPC G10L 15/22 (2013.01) [G06F 3/167 (2013.01); G10L 15/075 (2013.01); G10L 15/083 (2013.01); G10L 2015/223 (2013.01)] 18 Claims
OG exemplary drawing
 
1. A computer-implemented method executed on data processing hardware that causes the data processing hardware to perform operations comprising:
receiving audio data corresponding to at least a portion of an utterance spoken by a user, the utterance comprising a query to perform an action;
processing, using a speech recognizer, the audio data to generate a sequence of intermediate automated speech recognition (ASR) results;
based on the sequence of intermediate ASR results, launching both a first application and a second application to execute in parallel on a user device, the first application and the second application both associated with a particular application type;
displaying, in a graphical user interface of the user device, a first graphical element and a second graphical element simultaneously, the first graphical element representing the first application executing on the user device and the second graphical element representing the second application executing on the user device; and
in response to detecting an end of speech condition, fulfilling performance of the action using one of the first application or the second application.