US 11,941,736 B2
Systems and methods for motion-controlled animation
Shan Huang, Shanghai (CN)
Assigned to GOOGLE LLC, Mountain View, CA (US)
Filed by Google LLC, Mountain View, CA (US)
Filed on Sep. 30, 2020, as Appl. No. 17/039,538.
Application 17/039,538 is a continuation of application No. PCT/CN2020/119279, filed on Sep. 30, 2020.
Prior Publication US 2022/0101587 A1, Mar. 31, 2022
Int. Cl. G06T 13/00 (2011.01); G06N 20/00 (2019.01); G06T 7/246 (2017.01); G06T 7/73 (2017.01); G06T 13/40 (2011.01); G06T 15/20 (2011.01); H04N 7/15 (2006.01)
CPC G06T 13/40 (2013.01) [G06N 20/00 (2019.01); G06T 7/251 (2017.01); G06T 7/73 (2017.01); G06T 15/205 (2013.01); H04N 7/157 (2013.01); G06T 2207/20101 (2013.01); G06T 2207/30196 (2013.01); G06T 2215/16 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A computer-implemented method for motion-controlled animation via vector-based graphics, the method comprising:
obtaining, by a computing system comprising one or more computing devices, a vector-based surface illustration and a rigging skeleton at an initial pose, wherein the rigging skeleton comprises a plurality of rigging vectors that define a plurality of rigging vertices, and wherein correspondence data describes one or more correspondences between one or more rigging vectors or rigging vertices and one or more portions of the vector-based surface illustration;
obtaining, by the computing system, a plurality of image frames that depict an entity in motion; and
for each image frame of the plurality of image frames:
processing the image frame with one or more machine-learned models to obtain landmark positions within the image frame for a plurality of facial or body landmarks of the entity;
updating one or more respective positions of the plurality of rigging vertices of the rigging skeleton based on the landmark positions for the plurality of facial or body landmarks within the image frame; and
performing a skinning algorithm to transform and render the vector-based surface illustration based on the respective positions of the plurality of rigging vertices of the rigging skeleton and the correspondence data.