US 12,450,489 B2
Method, system and apparatus for federated learning
Stefanos Laskaridis, Middlesex (GB); Samuel Horvath, Middlesex (GB); Mario Almeida, Middlesex (GB); Ilias Leontiadis, Middlesex (GB); and Stylianos I. Venieris, Middlesex (GB)
Assigned to Samsung Electronics Co., Ltd., Suwon-si (KR)
Filed by Samsung Electronics Co., Ltd., Suwon-si (KR)
Filed on Jan. 27, 2022, as Appl. No. 17/586,178.
Application 17/586,178 is a continuation of application No. PCT/KR2022/000938, filed on Jan. 18, 2022.
Claims priority of application No. 20210100067 (GR), filed on Feb. 2, 2021; and application No. 21195060 (EP), filed on Sep. 6, 2021.
Prior Publication US 2022/0245459 A1, Aug. 4, 2022
Int. Cl. G06N 3/082 (2023.01); G06N 3/045 (2023.01)
CPC G06N 3/082 (2013.01) [G06N 3/045 (2023.01)] 19 Claims
OG exemplary drawing
 
1. A method performed by a central computing device for training a machine learning (ML) model comprising a neural network using federated learning performed by a plurality of client devices, the method comprising:
determining a computation capability of each client device of the plurality of client devices;
associating the each client device with a dropout rate defining how much of each neural network layer of the ML model is to be included in a target submodel to be trained by the each client device, based on the determined computation capability;
generating different sized submodels of the ML model that are nested within the ML model such a largest submodel of the different sized submodels is included in the ML model and each of an increasingly smaller in size submodel of the different sized submodels is included in a next larger in size submodel of the different sized submodels, the different sized submodels being generated by using the dropout rate associated with the each client device to perform pruning of at least one neuron of at least one neural network layer of the ML model according to a predefined ordering such that the at least one neuron that is pruned is not used by any submodel of the different sized submodels for which the at least one neuron is pruned; and
distributing, during each federated learning training round, at least two submodels of the different sized submodels to at least one client device of the plurality of client devices based on the dropout rate associated with the at least one client device, the at least two submodels including, a submodel and at least one submodel nested within the submodel, from among the different sized submodels,
wherein the generating of the different sized submodels of the ML model that are nested within the ML model comprises:
using the dropout rate associated with the each client device to perform the ordered pruning of one neural network layer of the ML model; and
using at least one further dropout rate to perform the pruning of at least one further neural network layer of the ML model.