US 11,734,581 B1
Providing contextual actions for mobile onscreen content
Ibrahim Badr, Zurich (CH); Mauricio Zuluaga, Adliswil (CH); Aneto Okonkwo, Zurich (CH); and Gökhan Bakir, Zurich (CH)
Assigned to GOOGLE LLC, Mountain View, CA (US)
Filed by Google LLC, Mountain View, CA (US)
Filed on May 19, 2021, as Appl. No. 17/324,249.
Application 17/324,249 is a continuation of application No. 16/702,248, filed on Dec. 3, 2019, granted, now 11,017,299.
Application 16/702,248 is a continuation of application No. 15/386,771, filed on Dec. 21, 2016, granted, now 10,535,005, issued on Jan. 14, 2020.
Claims priority of provisional application 62/413,174, filed on Oct. 26, 2016.
This patent is subject to a terminal disclaimer.
Int. Cl. G06F 9/54 (2006.01); G06N 5/022 (2023.01); H04L 9/40 (2022.01); G06F 3/0482 (2013.01); G06F 40/134 (2020.01); G06F 3/04817 (2022.01); G06F 8/61 (2018.01); H04W 88/02 (2009.01)
CPC G06N 5/022 (2013.01) [G06F 3/0482 (2013.01); G06F 3/04817 (2013.01); G06F 9/54 (2013.01); G06F 40/134 (2020.01); H04L 63/08 (2013.01); G06F 8/61 (2013.01); H04W 88/02 (2013.01)] 13 Claims
OG exemplary drawing
 
7. A system comprising:
at least one processor; and
memory storing instructions that, when executed, cause the at least one processor to:
receive, via an application programming interface and from a third-party application that is administered by a third-party, and during a current session in which a user is interacting with the third-party application via a client device:
context information that reflects user interactions with the third-party application, wherein the context information is not visible on a current screen of the client device during the current session, and wherein the user interactions with the third-party application include one or more of: current user interactions with the third-party application during the current session, or historical user interactions with the third-party application prior to the current session;
predict, based on the context information, an action to be performed using an additional third-party application that is administered by an additional third-party, wherein the additional third-party that administers the additional third-party application differs from the third-party that administers the third-party application;
cause an indication of the predicted action to be performed using the additional third-party application to be provided for presentation to the user; and
in response to causing the indication of the predicted action to be performed using the additional third-party application to be provided for presentation to the user:
receive spoken input that is directed to the predicted action to be performed using the additional third-party application; and
cause the additional third-party application to initiate performance of the predicted action, wherein the instructions to cause the additional third-party application to initiate performance of the predicted action comprise instructions to:
cause the current session to transfer focus from the third-party application to the additional third-party application; and
cause data derived from the user interactions with the third-party application to be transferred to the additional third-party application,
 wherein the data derived from the user interactions with the third-party application to be transferred to the additional third-party application includes: (i) an intent that is indicative of the action to be performed using the additional third-party application, and (ii) one or more values for one or more corresponding parameters associated with the intent that is indicative of the action to be performed using the additional third-party application.