US 11,785,073 B2
Systems and methods for communication efficient distributed mean estimation
Ananda Theertha Suresh, New York, NY (US); Sanjiv Kumar, Jericho, NY (US); Hugh Brendan McMahan, Seattle, WA (US); and Xinnan Yu, Forest Hills, NY (US)
Assigned to GOOGLE LLC, Mountain View, CA (US)
Filed by Google LLC, Mountain View, CA (US)
Filed on Oct. 15, 2021, as Appl. No. 17/502,794.
Application 17/502,794 is a continuation of application No. 15/708,793, filed on Sep. 19, 2017, granted, now 11,196,800.
Application 15/708,793 is a continuation in part of application No. 15/676,076, filed on Aug. 14, 2017, abandoned.
Claims priority of provisional application 62/400,019, filed on Sep. 26, 2016.
Prior Publication US 2022/0046082 A1, Feb. 10, 2022
Int. Cl. H04L 67/10 (2022.01); G06F 17/16 (2006.01); H03M 7/30 (2006.01); H03M 7/40 (2006.01); G06F 17/12 (2006.01); G06N 20/00 (2019.01); G06F 17/18 (2006.01); G06N 7/01 (2023.01); H04L 67/01 (2022.01)
CPC H04L 67/10 (2013.01) [G06F 17/12 (2013.01); G06F 17/16 (2013.01); G06F 17/18 (2013.01); G06N 7/01 (2023.01); G06N 20/00 (2019.01); H03M 7/3059 (2013.01); H03M 7/3082 (2013.01); H03M 7/40 (2013.01); H04L 67/01 (2022.05)] 20 Claims
OG exemplary drawing
 
1. A computing system that performs stochastic quantization to facilitate efficient transmission of machine-learned model updates from client devices to a centralized server computing device, the computing system comprising:
one or more client computing devices, wherein each client computing device comprises one or more processors and one or more non-transitory computer-readable media that store instructions that, when executed by the one or more processors cause the client computing device to perform operations, the operations comprising:
determining an update to a machine-learned model based at least in part on a local dataset stored at the client computing device, wherein the update comprises a plurality of coordinate values;
performing stochastic quantization of the update to obtain a quantized update, wherein performing stochastic quantization of the update comprises:
establishing a plurality of quantization levels;
identifying a minimum value among the plurality of coordinate values of the update;
setting a first quantization level of the plurality of quantization levels equal to the minimum value among the plurality of coordinate values; and
respectively quantizing each coordinate value of the plurality of coordinate value into one of the plurality of quantization levels according to a respective probability determined based on the coordinate value; and
transmitting the quantized update to the centralized server computing device.