US 11,887,233 B2
Machine learning acceleration of complex deformations such as muscle, skin and clothing simulation
David Sebastian Minor, Vancouver (CA)
Assigned to Digital Domain Virtual Human (US), Inc., Los Angeles, CA (US)
Filed by David Sebastian Minor, Vancouver (CA)
Filed on Feb. 18, 2022, as Appl. No. 17/676,087.
Prior Publication US 2023/0267666 A1, Aug. 24, 2023
Int. Cl. G06T 13/40 (2011.01); G06T 19/00 (2011.01); G06N 3/08 (2023.01); G06T 17/20 (2006.01)
CPC G06T 13/40 (2013.01) [G06N 3/08 (2013.01); G06T 17/20 (2013.01); G06T 19/00 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A method for preparing training data for training a neural network to simulate deformations of a surface of a CG character, the method comprising:
obtaining a distribution of joint angles and/or bone positions of a CG character over a set of animation data comprising a plurality of frames;
randomly generating a plurality of random poses according to the distribution of joint angles and/or bone positions;
generating a high-fidelity deformation of the surface of the CG character for each of the plurality of random poses;
transforming each of the high-fidelity deformations from a respective pose coordinate system to a rest pose coordinate system to obtain a plurality of warped rest poses, each warped rest pose corresponding to one of the high-fidelity deformations and one of the random poses and each warped rest pose parameterized at least in part by a three-dimensional (3D) surface mesh comprising a plurality of vertices;
parsing the warped rest poses into a plurality of regions to obtain, for each warped rest pose, a corresponding plurality of warped rest regions, each warped rest region parameterized at least in part by a regional three-dimensional (3D) surface mesh comprising a regional plurality of vertices from among the plurality of vertices of the vertices of the warped rest pose;
determining an approximation weight for each regional vertex of each of the plurality of warped rest regions;
for each region:
decomposing the warped rest region over the plurality of warped rest poses to obtain, for the region: a regional decomposition neutral vector, a regional set of decomposition basis (blendshape) vectors and, for each warped rest pose, a regional set of decomposition weights;
wherein, for each warped rest pose, the corresponding regional set of decomposition weights together with the regional decomposition neutral vector and the regional set of decomposition basis (blendshape) vectors can be used to at least approximately reconstruct the warped rest region of the warped rest pose;
wherein decomposing the warped rest region over the plurality of warped rest poses is based at least in part on the approximation weights; and
determining the training data to comprise the plurality of random poses and, for each random pose and for each region, the corresponding set of decomposition weights.