	US 12,014,739 B2
	Using corrections, of automated assistant functions, for training of on-device machine learning models
Françoise Beaufays, Mountain View, CA (US); Rajiv Mathews, Sunnyvale, CA (US); Dragan Zivkovic, Sunnyvale, CA (US); Kurt Partridge, San Francisco, CA (US); and Andrew Hard, Menlo Park, CA (US)
Assigned to GOOGLE LLC, Mountain View, CA (US)
Filed by GOOGLE LLC, Mountain View, CA (US)
Filed on Jul. 6, 2023, as Appl. No. 18/218,818.
Application 18/218,818 is a continuation of application No. 16/973,572, granted, now 11,741,953, previously published as PCT/US2019/060537, filed on Nov. 8, 2019.
Prior Publication US 2023/0352019 A1, Nov. 2, 2023
Int. Cl. G10L 15/22 (2006.01); G10L 15/065 (2013.01); G10L 15/10 (2006.01); G10L 15/30 (2013.01)

CPC G10L 15/22 (2013.01) [G10L 15/065 (2013.01); G10L 15/10 (2013.01); G10L 15/30 (2013.01)]

20 Claims

20. A client device comprising:

one or more hardware processors; and

memory storing instructions that, when executed by the one or more hardware processors, cause the one or more hardware processors to:

receive, via one or more microphones of the client device, audio data that captures a spoken utterance of a user;

process the audio data using a machine learning model, stored locally at the client device, to generate a predicted output;

make a decision, based on the predicted output failing to satisfy a threshold, to refrain from initiating one or more currently dormant automated assistant functions;

subsequent to making the decision to refrain from initiating the one or more currently dormant automated assistant functions:

receive, via the client device, further user interface input to initiate one or more of the currently dormant automated assistant functions and that contradicts the decision to refrain from initiating one or more of the currently dormant automated assistant functions; and

determine, based on a duration of time between receiving the audio data that was processed in making the decision and receiving the further user interface input that contradicts the decision, that the decision to refrain from initiating one or more of the one or more currently dormant automated assistant functions was incorrect; and

in response to determining that the decision was incorrect:

generate a gradient based on comparing the predicted output to ground truth output that satisfies the threshold, and

transmit, over a network and to a remote system, the generated gradient without transmitting any of: the audio data and the further user interface input,

wherein the remote system utilizes the generated gradient, and additional gradients from additional client devices, to update global weights of a global machine learning model that is a global counterpart of the machine learning model that is stored locally at the client device.