US 11,775,158 B2
Device-based image modification of depicted objects
Theresa Barton, San Mateo, CA (US); Yanping Chen, San Jose, CA (US); Jaewook Chung, Mountain View, CA (US); Christopher Yale Crutchfield, San Diego, CA (US); Aymeric Damien, London (GB); Sergei Kotcur, Sochi (RU); Igor Kudriashov, Saratov (RU); Sergey Tulyakov, Marina del Rey, CA (US); Andrew Wan, Marina del Rey, CA (US); and Emre Yamangil, San Francisco, CA (US)
Assigned to Snap Inc., Santa Monica, CA (US)
Filed by Snap Inc., Santa Monica, CA (US)
Filed on Jun. 22, 2021, as Appl. No. 17/354,520.
Application 17/354,520 is a continuation of application No. 16/265,355, filed on Feb. 1, 2019, granted, now 11,068,141.
Claims priority of provisional application 62/625,861, filed on Feb. 2, 2018.
Prior Publication US 2021/0311618 A1, Oct. 7, 2021
Int. Cl. G06T 7/11 (2017.01); G06F 3/04845 (2022.01); G06F 3/0482 (2013.01); H04N 5/265 (2006.01); H04N 5/262 (2006.01); G06V 20/40 (2022.01); G06V 40/16 (2022.01); G06F 3/04817 (2022.01)
CPC G06F 3/04845 (2013.01) [G06F 3/0482 (2013.01); G06T 7/11 (2017.01); G06V 20/40 (2022.01); G06V 40/161 (2022.01); H04N 5/265 (2013.01); H04N 5/2628 (2013.01); G06F 3/04817 (2013.01); G06T 2207/10016 (2013.01); G06T 2207/20081 (2013.01); G06T 2207/20084 (2013.01); G06T 2207/20132 (2013.01); G06T 2207/30201 (2013.01)] 17 Claims
OG exemplary drawing
 
1. A method comprising:
accessing a frame;
detecting within the frame a first face having a first facial configuration and a second face having a second facial configuration;
segmenting the frame to generate a first face segment and a second face segment;
selecting, based on the first facial configuration, a first convolutional neural network trained to modify the first facial configuration to have a third facial configuration;
selecting, based on the second facial configuration, a second convolutional neural network trained to modify the second facial configuration to have the third facial configuration, wherein the first convolutional neural network is a first style transfer neural network trained to transfer images from the first facial configuration to the third facial configuration and the second convolutional neural network is a second style transfer neural network trained to transfer images from the second facial configuration to the third facial configuration; and
modifying, using the first convolutional neural network and the second convolutional neural network, the first face segment and the second face segment, respectively, to generate a first modified face segment and a second modified face segment.