CPC G06N 20/00 (2019.01) [H04L 47/2441 (2013.01); H04L 47/32 (2013.01); H04L 49/3027 (2013.01); H04L 67/10 (2013.01); H04L 49/252 (2013.01)] | 20 Claims |
1. A network switching apparatus, comprising:
a plurality of communication interfaces configured to connect to specific computing devices in a network, including compute devices of a distributed learning system;
packet-switching logic configured to:
receive data units via the communication interfaces; and
forward first data units of the data units to destination devices identified for the first data units over the communication interfaces;
machine learning logic configured to:
recognize, in the data units, second data units carrying gradients of parameters in a machine learning model being trained against a training data set, each of the second data units carrying at least a portion of one of the gradients;
based on the second data units, aggregate sets of the gradients by performing one or more reduction operations on data sets within the gradients, the one or more reduction operations selected based on an attribute of the data sets; and
return the aggregated gradients to the compute devices.
|