US 11,675,566 B2
Rendering visual components on applications in response to voice commands
Anuj Thakkar, Mountain View, CA (US)
Assigned to GOOGLE LLC, Mountain View, CA (US)
Filed by GOOGLE LLC, Mountain View, CA (US)
Filed on Jun. 10, 2022, as Appl. No. 17/837,896.
Application 17/837,896 is a continuation of application No. 16/466,254, granted, now 11,360,738, previously published as PCT/US2019/030930, filed on May 6, 2019.
Prior Publication US 2022/0308830 A1, Sep. 29, 2022
Int. Cl. G10L 15/22 (2006.01); G06F 3/16 (2006.01); G06F 9/451 (2018.01); G10L 15/18 (2013.01)
CPC G06F 3/167 (2013.01) [G06F 9/451 (2018.02); G10L 15/1822 (2013.01); G10L 15/22 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A system to render visual components on applications, comprising:
a natural language processor executed on a data processing system having one or more processors, the natural language processor to:
receive a data packet comprising an input audio signal detected by a sensor of a client device, the client device displaying a graphical user interface of a first application in a foreground process on the client device; and
parse the input audio signal of the data packet to identify a request;
an action handler executed on the data processing system to:
select an action-inventory that executes an action corresponding to the request by a second application, the second application installed on the client device and not in the foreground process; and
generate an action data structure in accordance with the action-inventory; and
an agent interface executed on the data processing system to:
provide the action data structure to the second application to cause the second application to parse the action data structure and execute the action to generate an output;
determine that the output of the second application from execution of the action is authorized to be presented with the graphical user interface of the first application based on an authorization policy of the second application;
identify the user interface component of the second application for the action-inventory, responsive to the determination the output of the second application is authorized to be presented with the first application; and
display, on the client device, the user interface component including the output from the second application with the graphical user interface of the first application authorized to be presented with the second application.