US 10,891,723 B1
Realistic neural network based image style transfer
Jaewook Chung, Mountain View, CA (US); Christopher Yale Crutchfield, San Diego, CA (US); and Emre Yamangil, San Francisco, CA (US)
Assigned to Snap Inc., Santa Monica, CA (US)
Filed by Snap Inc., Santa Monica, CA (US)
Filed on Sep. 29, 2018, as Appl. No. 16/147,705.
Claims priority of provisional application 62/566,072, filed on Sep. 29, 2017.
Int. Cl. G06T 5/40 (2006.01); G06T 7/90 (2017.01); G06T 5/00 (2006.01); G06N 3/08 (2006.01); H04L 29/08 (2006.01); G06K 9/00 (2006.01); G06N 20/00 (2019.01); G06Q 50/00 (2012.01)
CPC G06T 5/40 (2013.01) [G06K 9/00228 (2013.01); G06N 3/08 (2013.01); G06N 20/00 (2019.01); G06T 5/001 (2013.01); G06T 5/009 (2013.01); G06T 7/90 (2017.01); H04L 67/04 (2013.01); G06Q 50/01 (2013.01); G06T 2207/10024 (2013.01); G06T 2207/20132 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A method comprising:
generating, using an image sensor on a client device, an initial image, the initial image depicting a user face having a first appearance;
receiving an indication of an instruction to modify the initial image, the instruction specifying changing the user face from having the first appearance to a second appearance;
detecting the user face in the initial image;
generating a cropped portion comprising the user face and a non-cropped portion;
generating a modified image from the cropped portion using a convolutional neural network, the convolutional neural network trained on a set of images including images of user faces exhibiting the first appearance and images of user faces exhibiting the second appearance, the modified image displaying the user face having the second appearance;
generating a result image by blending the modified image with the initial image;
applying adjustments to a non-cropped portion of the result image, the non-cropped portion of the result image corresponding to the non-cropped portion of the initial image, and wherein the adjustments are effects selected based on the convolutional neural network used to generate the modified image; and
storing the result image.