US 12,277,640 B2
Photorealistic real-time portrait animation
Eugene Krokhalev, London (GB); Aleksandr Mashrabov, Los Angeles, CA (US); and Pavel Savchenkov, London (GB)
Assigned to Snap Inc., Santa Monica, CA (US)
Filed by Snap Inc., Santa Monica, CA (US)
Filed on Apr. 22, 2024, as Appl. No. 18/641,472.
Application 18/641,472 is a continuation of application No. 18/080,779, filed on Dec. 14, 2022, granted, now 11,995,758.
Application 18/080,779 is a continuation of application No. 17/751,796, filed on May 24, 2022, granted, now 11,568,589, issued on Jan. 31, 2023.
Application 17/751,796 is a continuation of application No. 17/325,225, filed on May 20, 2021, granted, now 11,393,152, issued on Jul. 19, 2022.
Application 17/325,225 is a continuation of application No. 16/251,472, filed on Jan. 18, 2019, granted, now 11,049,310, issued on Jun. 29, 2021.
Prior Publication US 2024/0296614 A1, Sep. 5, 2024
Int. Cl. G06T 13/80 (2011.01); G06T 7/174 (2017.01); G06V 40/16 (2022.01)
CPC G06T 13/80 (2013.01) [G06T 7/174 (2017.01); G06V 40/167 (2022.01); G06T 2207/20084 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A method for portrait animation, the method comprising:
receiving, by a computing device, scenario data including information concerning movements of a first head;
receiving, by the computing device, a target image including a second head and a background;
determining, by the computing device and based on the target image and the information concerning the movements of the first head, two-dimensional (2D) deformations of the second head in the target image;
applying, by the computing device, the 2D deformations to the target image to obtain at least one output frame of an output video, the at least one output frame including the second head displaced according to the movements of the first head; and
using a background prediction neural network, filling, by the computing device, a portion of the background in gaps between the displaced second head and the background.