US 12,405,705 B2
System and method for providing an artificial intelligence control surface for a user of a computing device
David Matthew Jones, London (GB); Robert Marchant, Richmond (GB); Benjamin Guy Alexander Pawle, London (GB); George Joseph Rickerby, London (GB); and Michael Colville, London (GB)
Assigned to GOOGLE LLC, Mountain View, CA (US)
Appl. No. 17/626,265
Filed by Google LLC, Mountain View, CA (US)
PCT Filed Jul. 11, 2019, PCT No. PCT/US2019/041343
§ 371(c)(1), (2) Date Jan. 11, 2022,
PCT Pub. No. WO2021/006906, PCT Pub. Date Jan. 14, 2021.
Prior Publication US 2022/0291789 A1, Sep. 15, 2022
Int. Cl. G06F 3/0482 (2013.01); G06F 3/04812 (2022.01); G06F 3/0488 (2022.01); G06F 3/04886 (2022.01)
CPC G06F 3/0482 (2013.01) [G06F 3/04812 (2013.01); G06F 3/0488 (2013.01); G06F 3/04886 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A computing system comprising:
one or more processors;
an artificial intelligence system that comprises one or more machine-learned models, the artificial intelligence system configured to receive context data and, based on receipt of the context data, output data descriptive of a plurality of available control actions available for performance by the artificial intelligence system on behalf of a user of the computing system;
one or more non-transitory computer-readable media that collectively store instructions that, when executed by the one or more processors, cause the computing system to perform operations, the operations comprising:
obtaining the context data;
inputting the context data into the artificial intelligence system;
receiving, as an output of the artificial intelligence system, the data descriptive of the plurality of available control actions available for performance by the artificial intelligence system;
providing a user interface for display to a user, wherein the user interface includes a first portion configured to display received messages and sent messages, a second portion configured to display a data entry field for composing messages to potentially be sent, and a third portion configured to display an artificial intelligence control surface that comprises a plurality of user-selectable tiles, the plurality of user-selectable tiles respectively corresponding to the plurality of available control actions, wherein a first user-selectable tile of the plurality of user-selectable tiles includes content automatically generated by the artificial intelligence system based on a received message displayed in the first portion;
based on detecting a first type of a user touch action selecting the first user-selectable tile of the plurality of user-selectable tiles within the third portion of the user interface, generating at least one additional user-selectable tile for display within the third portion of the user interface, the at least one additional user-selectable tile being displayed together with the first user-selectable tile within the third portion of the user interface and at a location proximate to the first user-selectable tile; and
based on detecting a second type of a user touch action selecting the first user-selectable tile, inserting the content from the first user-selectable tile into the data entry field provided within the second portion of the user interface to form part of a message to potentially be sent,
wherein generating the at least one additional user-selectable tile comprises inputting at least one characteristic of the first user-selectable tile into the artificial intelligence system as further context data, and receiving, as a further output of the artificial intelligence system, further data descriptive of at least one further available control action available for performance by the artificial intelligence system, the at least one additional user-selectable tile corresponding to the at least one further available control action.