US 12,079,543 B2
Rendering visual components on applications in response to voice commands
Anuj Thakkar, Mountain View, CA (US)
Assigned to GOOGLE LLC, Mountain View, CA (US)
Filed by GOOGLE LLC, Mountain View, CA (US)
Filed on May 8, 2023, as Appl. No. 18/144,740.
Application 18/144,740 is a continuation of application No. 17/837,896, filed on Jun. 10, 2022, granted, now 11,675,566.
Application 17/837,896 is a continuation of application No. 16/466,254, granted, now 11,360,738, issued on Jun. 14, 2022, previously published as PCT/US2019/030930, filed on May 6, 2019.
Prior Publication US 2023/0280974 A1, Sep. 7, 2023
This patent is subject to a terminal disclaimer.
Int. Cl. G10L 15/22 (2006.01); G06F 3/16 (2006.01); G06F 9/451 (2018.01); G10L 15/18 (2013.01)
CPC G06F 3/167 (2013.01) [G06F 9/451 (2018.02); G10L 15/1822 (2013.01); G10L 15/22 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A system to render visual components on applications, comprising:
a natural language processor executed on a data processing system having one or more processors, the natural language processor to:
receive a data packet comprising an input audio signal detected by a sensor of a client device, the client device displaying a graphical user interface of a first application that is running in a foreground process on the client device; and
parse the input audio signal of the data packet to identify a request;
an action handler executed on the data processing system to:
select an action-inventory that executes an action corresponding to the request by a second application, the second application installed on the client device and not in the foreground process; and
generate an action data structure in accordance with the action-inventory; and
an agent interface executed on the data processing system to:
provide the action data structure to the second application to cause the second application to parse the action data structure and execute the action to generate an output;
determine, based on an authorization policy, that the output of the second application from execution of the action is not authorized to be presented with the graphical user interface of the first application; and
in response to determining that the output of the second application from execution of the action is not authorized to be presented with the graphical user interface of the first application:
remove the first application from the foreground process on the client device and setting the first application to a non-foreground process on the client device;
set the second application to run in the foreground process on the client device; and
display a graphical user interface of the second application, including the output generated by the second application, on the client device.