US 12,219,004 B2
Systems and methods for communication efficient distributed mean estimation
Ananda Theertha Suresh, New York, NY (US); Sanjiv Kumar, Jericho, NY (US); Hugh Brendan McMahan, Seattle, WA (US); and Xinnan Yu, Forest Hills, NY (US)
Assigned to GOOGLE LLC, Mountain View, CA (US)
Filed by Google LLC, Mountain View, CA (US)
Filed on Aug. 31, 2023, as Appl. No. 18/240,799.
Application 18/240,799 is a continuation of application No. 17/502,794, filed on Oct. 15, 2021, granted, now 11,785,073.
Application 17/502,794 is a continuation of application No. 15/708,793, filed on Sep. 19, 2017, granted, now 11,196,800, issued on Dec. 7, 2021.
Application 15/708,793 is a continuation in part of application No. 15/676,076, filed on Aug. 14, 2017, abandoned.
Claims priority of provisional application 62/400,019, filed on Sep. 26, 2016.
Prior Publication US 2024/0098138 A1, Mar. 21, 2024
This patent is subject to a terminal disclaimer.
Int. Cl. G06N 20/00 (2019.01); G06F 17/12 (2006.01); G06F 17/16 (2006.01); G06F 17/18 (2006.01); G06N 7/01 (2023.01); H03M 7/30 (2006.01); H03M 7/40 (2006.01); H04L 67/10 (2022.01); H04L 67/01 (2022.01)
CPC H04L 67/10 (2013.01) [G06F 17/12 (2013.01); G06F 17/16 (2013.01); G06F 17/18 (2013.01); G06N 7/01 (2023.01); G06N 20/00 (2019.01); H03M 7/3059 (2013.01); H03M 7/3082 (2013.01); H03M 7/40 (2013.01); H04L 67/01 (2022.05)] 20 Claims
OG exemplary drawing
 
1. A computing system that performs stochastic quantization to facilitate efficient transmission of machine-learned model updates from client devices to a centralized server computing device, the computing system comprising:
one or more client computing devices, wherein each client computing device comprises one or more processors and one or more non-transitory computer-readable media that store instructions that, when executed by the one or more processors cause the client computing device to perform operations, the operations comprising:
determining an update to a machine-learned model based at least in part on a local dataset stored at the client computing device, wherein the update comprises a plurality of coordinate values;
performing quantization of the update to obtain a quantized update, wherein performing quantization of the update comprises:
establishing a plurality of quantization levels;
respectively quantizing each coordinate value of the plurality of coordinate value into one of the plurality of quantization levels according to a respective probability determined based on the coordinate value; and
encoding the quantized update according to a variable length coding scheme; and
transmitting the quantized update to the centralized server computing device.