US 12,346,712 B1
Artificial intelligence application assistant
Sven Eberhardt, Seattle, WA (US); Brian Westphal, Livermore, CA (US); Yangyong Zhang, San Francisco, CA (US); Allen Lu, Issaquah, WA (US); Nathan Hurst, Seattle, WA (US); Harry Lu, San Mateo, CA (US); Evan Welbourne, Seattle, WA (US); John Bicket, San Francisco, CA (US); and Sanjit Biswas, San Francisco, CA (US)
Assigned to Samsara Inc., San Francisco, CA (US)
Filed by Samsara Inc., San Francisco, CA (US)
Filed on Apr. 2, 2024, as Appl. No. 18/624,774.
Int. Cl. G06F 9/451 (2018.01); G01C 21/34 (2006.01); G06F 40/20 (2020.01); G08G 1/00 (2006.01)
CPC G06F 9/453 (2018.02) [G01C 21/34 (2013.01); G06F 40/20 (2020.01); G08G 1/20 (2013.01)] 18 Claims
OG exemplary drawing
 
1. A method performed by an application assistant computing system, the method including:
receiving, from a user device, an input from a user indicating a query or task;
determining at least two components associated with a user interface displayed on the user device, each of the at least two components providing different user interface functionality;
accessing component metadata of the determined components;
accessing context information associated with the user or user device;
generating a natural language text prompt by combining at least some of the user input, text indicative of at least some of the component metadata, first text indicative of at least some of the context information, and second text indicative of one or more available response elements;
providing the prompt to a large language model (LLM);
receiving an output from the LLM indicating one or more response elements selected by the LLM, wherein a first of the one or more response elements indicates a function call;
generate a second prompt requesting executable code that interacts with an external service to perform the function call, the LLM prompt including one or more examples of outputs;
providing the second prompt to the LLM;
receiving a second output from the LLM including customized executable code that interacts with the external service in accordance with the query or task received from the user;
executing the customized executable code received from the LLM;
receiving, from the external service, additional response elements associated with the query or task received from the user;
generating, based on the one or more response elements and additional response elements, updated user interface code indicating one or more updates to the user interface displayed on the user device; and
initiating update of the user interface displayed on the user device according to the updated user interface code.