US 11,843,586 B2
Systems and methods for providing a modified loss function in federated-split learning
Gharib Gharibi, Lenexa, KS (US); Ravi Patel, Kansas City, MO (US); Babak Poorebrahim Gilkalaye, Kansas City (CA); Praneeth Vepakomma, Weymouth, MA (US); Greg Storm, Parkville, MO (US); and Riddhiman Das, Parkville, MO (US)
Assigned to TRIPLEBLIND, INC., Kansas City, MO (US)
Filed by TripleBlind, Inc., Kansas City, MO (US)
Filed on Aug. 29, 2022, as Appl. No. 17/897,884.
Application 17/897,884 is a continuation of application No. 17/499,153, filed on Oct. 12, 2021, granted, now 11,431,688.
Application 17/499,153 is a continuation of application No. 17/499,153, filed on Oct. 12, 2021, granted, now 11,431,688.
Application 17/499,153 is a continuation in part of application No. 16/828,085, filed on Mar. 24, 2020, granted, now 11,582,203.
Application 16/828,085 is a continuation of application No. 17/499,153, filed on Oct. 12, 2021, granted, now 11,431,688.
Application 17/499,153 is a continuation in part of application No. 16/828,216, filed on Mar. 24, 2020.
Application 16/828,216 is a continuation of application No. 17/499,153, filed on Oct. 12, 2021, granted, now 11,431,688.
Application 17/499,153 is a continuation in part of application No. 17/176,530, filed on Feb. 16, 2021.
Application 17/176,530 is a continuation of application No. 16/828,354, filed on Mar. 24, 2020, granted, now 10,924,460, issued on Feb. 16, 2021.
Application 16/828,354 is a continuation of application No. 17/499,153, filed on Oct. 12, 2021, granted, now 11,431,688.
Application 17/499,153 is a continuation in part of application No. 16/828,420, filed on Mar. 24, 2020, granted, now 11,363,002, issued on Jun. 14, 2022.
Application 16/828,420 is a continuation of application No. 17/499,153, filed on Oct. 12, 2021, granted, now 11,431,688.
Claims priority of provisional application 63/090,904, filed on Oct. 13, 2020.
Claims priority of provisional application 62/948,105, filed on Dec. 13, 2019.
Claims priority of provisional application 63/226,135, filed on Jul. 27, 2021.
Prior Publication US 2022/0417225 A1, Dec. 29, 2022
This patent is subject to a terminal disclaimer.
Int. Cl. H04L 9/40 (2022.01); G06F 17/16 (2006.01); H04L 9/00 (2022.01); H04L 9/06 (2006.01); G06N 3/04 (2023.01); G06N 3/082 (2023.01); G06Q 20/40 (2012.01); G06Q 30/0601 (2023.01); G06F 18/24 (2023.01); G06F 18/2113 (2023.01)
CPC H04L 63/0428 (2013.01) [G06F 17/16 (2013.01); G06F 18/2113 (2023.01); G06F 18/24 (2023.01); G06N 3/04 (2013.01); G06N 3/082 (2013.01); G06Q 20/401 (2013.01); G06Q 30/0623 (2013.01); H04L 9/008 (2013.01); H04L 9/0625 (2013.01); G06Q 2220/00 (2013.01); H04L 2209/46 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A method comprising:
training, at a client system of a plurality of client systems, a part of a deep learning network up to a split layer of the client system;
based on an output of the split layer of the client system, completing, at a server system, training of the deep learning network;
calculating a weighted loss function for the client system at a last layer of the server system to yield a calculated loss function for the client system;
after each respective client system of the plurality of client systems has a respective loss function to yield a plurality of respective weighted client loss functions, averaging, at the server system, the plurality of respective weighted client loss functions to yield an average loss value;
back propagating gradients based on the average loss value from the last layer of the server system to a split layer of the server system to yield server system split layer gradients;
transmitting just the server system split layer gradients to the plurality of client systems, wherein no weights are shared across different client systems of the plurality of client systems;
back propagating, at the client system and from the split layer of the client system to an input layer of the client system, the server system split layer gradients to complete a training epoch of the deep learning network to yield a first model of a first client system of the plurality of client systems and a second model of a second client system of the plurality of client systems;
selecting, at the server system, a generator number g and a prime number p for a protocol and sending the generator number g and the prime number p to each client system i of the plurality of client systems, wherein a value of i identifies a respective client system of the plurality of client systems;
generating, at each client system i of the plurality of client systems, a random number ri and computing a key ki and sending, from each client system i of the plurality of client systems, the key ki to the server system to yield received keys ki;
sending all the received keys ki to each client system i of the plurality of client systems;
computing, at each client system i of the plurality of client systems with another client system j, a key Kij with all other clients, wherein a value of j identifies a different client system of the plurality of client systems from each client system i;
creating n shares of data for each client system i using a ShareGeneration (d) function: [d]i1, . . . , [d]in=ShareGeneration (d) and masking a client system j share associated with the other client system j using the key kij, where n represent number of shares created;
computing, for each client system i of the plurality of client systems, the client system j share associated with the other client system j;
sending, from the client system i of the plurality of client systems, the client system j share to the server system where [d′]ij is an encrypted share of d shared between client systems i and j;
sending the n shares to each corresponding client system of each client system i of the plurality of client systems;
unmasking, at each client system i of the plurality of client systems, a received share with the key kij;
adding, at each client system i of the plurality of client systems, all respective local shares and sending the respective local shares to the server system;
adding, at the server system, all received respective local shares from each client system i of the plurality of client systems to yield a result, and dividing the result by n to yield the average model, wherein n is a numeric value; and
distributing the average model to both the first client system and the second client system.