US 12,353,798 B2
Condensed spoken utterances for automated assistant control of an intricate application GUI
Jacek Szmigiel, Zurich (CH); and Joseph Lange, Zurich (CH)
Assigned to GOOGLE LLC, Mountain View, CA (US)
Filed by GOOGLE LLC, Mountain View, CA (US)
Filed on Apr. 30, 2024, as Appl. No. 18/650,938.
Application 18/650,938 is a continuation of application No. 17/947,359, filed on Sep. 19, 2022, granted, now 11,995,379.
Application 17/947,359 is a continuation of application No. 16/972,987, granted, now 11,449,308, issued on Sep. 20, 2022, previously published as PCT/US2019/046145, filed on Aug. 12, 2019.
Claims priority of provisional application 62/876,323, filed on Jul. 19, 2019.
Prior Publication US 2024/0281205 A1, Aug. 22, 2024
This patent is subject to a terminal disclaimer.
Int. Cl. G06F 3/16 (2006.01); G06F 3/0482 (2013.01); G06F 3/04847 (2022.01); G06F 3/04883 (2022.01); G10L 15/22 (2006.01)
CPC G06F 3/167 (2013.01) [G06F 3/0482 (2013.01); G06F 3/04847 (2013.01); G06F 3/04883 (2013.01); G10L 15/22 (2013.01); G10L 2015/223 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A method implemented by one or more processors, the method comprising:
receiving, via a computing device, a spoken utterance intended for controlling a GUI element; and
in response to receiving the spoken utterance:
accessing, by an automated assistant accessible via the computing device, content description data that characterizes GUI elements displayed via a display of the computing device, wherein the content description data is not displayed via the display,
comparing, by the automated assistant, one or more terms in the spoken utterance to the content description data that characterizes the GUI elements displayed at the display, to identify whether the one or more terms in the spoken utterance are associated with any GUI element from the GUI elements displayed at the display, and
in response to identifying that at least a portion of the one or more terms in the spoken utterance is associated with a particular GUI element from the GUI elements displayed at the display:
controlling, by the automated assistant, the particular GUI element in response to the spoken utterance.