US 12,254,412 B2
Optimizer based prunner for neural networks
Enxu Yan, Los Altos, CA (US); Sergey Tulyakov, Marina del Rey, CA (US); Aleksei Podkin, Santa Monica, CA (US); and Aleksei Stoliar, Marina del Rey, CA (US)
Assigned to Snap Inc., Santa Monica, CA (US)
Filed by Snap Inc., Santa Monica, CA (US)
Filed on Jan. 12, 2023, as Appl. No. 18/096,338.
Application 18/096,338 is a continuation of application No. 16/586,635, filed on Sep. 27, 2019, granted, now 11,580,400.
Prior Publication US 2023/0146865 A1, May 11, 2023
Int. Cl. G06N 3/082 (2023.01); G06F 17/16 (2006.01); G06N 3/04 (2023.01); G06T 7/10 (2017.01)
CPC G06N 3/082 (2013.01) [G06F 17/16 (2013.01); G06N 3/04 (2013.01); G06T 7/10 (2017.01); G06T 2207/20081 (2013.01); G06T 2207/20084 (2013.01)] 14 Claims
OG exemplary drawing
 
1. A method comprising:
training, at a server, a full generative neural network model;
generating a sparsely pruned generative neural network model from the full generative neural network model by accessing only a parameter vector of the full generative neural network model and a gradient vector of a loss function with respect to parameters of the full generative neural network model, applying a sparse pruning optimizer that is agnostic to the full generative neural network model and individually pruning parameters of the full generative neural network model by pruning non-zero parameters in the parameter vector of the full generative neural network model based on gradient data in the gradient vector of the loss function with respect to the parameters of the full generative neural network model, and iteratively updating the parameters based on the gradient vector while encouraging as many non-parameters to be zeros as possible based on a cumulative penalty; and
communicating the sparsely pruned generative neural network model to a client device, the client device configured to: generate an image with a camera of the client device, generate image output data by applying the sparsely pruned generative neural network model to the image, and generate a modified image based on the image output data.