US 12,387,411 B2
Puppeteering a remote avatar by facial expressions
Tarek Hefny, Redmond, WA (US); Nicholas Reiter, Mountain View, CA (US); Brandon Young, Mountain View, CA (US); Arun Kandoor, Santa Clara, CA (US); and Dillon Cower, Mountain View, CA (US)
Assigned to Google LLC, Mountain View, CA (US)
Filed by Google LLC, Mountain View, CA (US)
Filed on Dec. 21, 2023, as Appl. No. 18/391,767.
Application 18/391,767 is a continuation of application No. 18/058,621, filed on Nov. 23, 2022, granted, now 11,887,235.
Application 18/058,621 is a continuation of application No. 17/052,161, granted, now 11,538,211, issued on Dec. 27, 2022, previously published as PCT/US2019/030218, filed on May 1, 2019.
Claims priority of provisional application 62/667,767, filed on May 7, 2018.
Prior Publication US 2024/0127523 A1, Apr. 18, 2024
Int. Cl. G06T 13/40 (2011.01); G06T 7/13 (2017.01); G06T 7/73 (2017.01); G06T 17/20 (2006.01); G06T 19/20 (2011.01); H04L 67/10 (2022.01)
CPC G06T 13/40 (2013.01) [G06T 7/13 (2017.01); G06T 7/73 (2017.01); G06T 17/20 (2013.01); G06T 19/20 (2013.01); G06T 2207/10024 (2013.01); G06T 2207/30201 (2013.01); H04L 67/10 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A computer-implemented method when executed by data processing hardware causes the data processing hardware to perform operations comprising:
receiving a first facial framework and a first captured image of a face of a user with a neutral facial expression, the first facial framework corresponding to the face of the user at a first frame;
determining a first facial texture based on the first facial framework and the first captured image, the first facial texture corresponding to the neutral facial expression;
receiving a second facial framework and a second captured image of the face of the user with a non-neutral facial expression, the second facial framework corresponding to the face of the user at a second frame;
determining a second facial texture based on the second facial framework, the second facial texture corresponding to the non-neutral facial expression;
blending the neutral facial expression corresponding to the first facial texture and the non-neutral facial expression corresponding to the second facial texture together to generate a third facial texture; and
displaying the third facial texture as a three-dimensional avatar, the three-dimensional avatar corresponding to a virtual representation of the face of the user.