US 12,322,020 B1
System apparatus and method for providing facial expression to avatars
Sergei Sherman, Givatayim (IL); and Dmitrii Ulianov, Ramat Gan (IL)
Assigned to Goodsize, Inc., Wilmington, DE (US)
Filed by Goodsize, Inc., Wilmington, DE (US)
Filed on Aug. 7, 2024, as Appl. No. 18/796,304.
Claims priority of provisional application 63/603,700, filed on Nov. 29, 2023.
Int. Cl. G06T 13/40 (2011.01); G06V 40/16 (2022.01)
CPC G06T 13/40 (2013.01) [G06V 40/174 (2022.01)] 20 Claims
OG exemplary drawing
 
11. A method for providing a facial expression to one or more video frames of a virtual avatar processed by a neural network system comprising:
training a neural network system to replace a face of the virtual avatar with a source face and to provide a facial expression of the source face to the face of the avatar in a real-time at a training phase using a neural network training system; and
providing by an inference system configured to use the trained neural network system to one or more facial expressions of the source face to the face of the avatar in real-time to cause the one or more facial expressions of the avatar to imitate the one or more facial expressions of a source face media which is represented by the virtual avatar by reconstructing a three dimensional (3D) model of a face of the virtual avatar, animating the face of the virtual avatar by reconstructing geometry of the 3D model, rendering the animated face of the virtual avatar and apply to the rendered virtual avatar the trained neural network system in a post-processing phase.