US 12,443,890 B2
Partially local federated learning
Karan Singhal, Roslyn, NY (US); Hakim Sidahmed, Jr., Redmond, WA (US); Zachary A. Garrett, Seattle, WA (US); Shanshan Wu, Seattle, WA (US); John Keith Rush, Seattle, WA (US); and Sushant Prakash, Marblehead, MA (US)
Assigned to Google LLC, Mountain View, CA (US)
Filed by Google LLC, Mountain View, CA (US)
Filed on May 27, 2021, as Appl. No. 17/332,893.
Prior Publication US 2022/0398500 A1, Dec. 15, 2022
Int. Cl. G06N 20/20 (2019.01)
CPC G06N 20/20 (2019.01) 25 Claims
OG exemplary drawing
 
1. A method performed by a client computing device that is in data communication with a server computing device over a data communication network, the method comprising:
maintaining local data and data defining a set of local parameters of a machine learning model, wherein the local data is a proper subset of a plurality of proper subsets of data that are used to train the machine learning model, each proper subset of data maintained on a separate client computing device, and is used to train the machine learning model only on the client computing device on which the proper subset is maintained;
receiving, from the server computing device, data defining current values of a set of global parameters of the machine learning model over the data communication network;
after receiving the data defining current values of the set of global parameters, but prior to training the set of global parameters of the machine learning model, determining, based on i) a support dataset that is a subset of the local data maintained at the client computing device that is excluded from being used for training the set of global parameters of the machine learning model, ii) the set of local parameters, and iii) the current values of the set of global parameters, current values of the set of local parameters of the machine learning model, the current values of the set local parameters being different from the current values of the global parameters;
after determining the current values of the set of local parameters of the machine learning model, training the set of global parameters of the machine learning model based on i) a query dataset that is another subset of the local data maintained at the client computing device that is used for training the set of global parameters of the machine learning model, ii) the current values of the set of local parameters, and iii) the current values of the set of global parameters to determine updated values of the set of global parameters of the machine learning model, the query dataset being different from the support dataset;
generating, based on a difference between the current values of the set of global parameters and the updated values of the set of global parameters, parameter update data that does not contain updates to the set of local parameters of the machine learning model and thus precludes recovery of the local data maintained at the client computing device from the parameter update data, and wherein the parameter update data defines an update to the set of global parameters of the machine learning model; and
transmitting, to the server computing device, the parameter update data over the data communication network.