US 11,915,365 B2
3D body model generation
Riza Alp Guler, London (GB); Haoyang Wang, London (GB); Iason Kokkinos, London (GB); and Stefanos Zafeiriou, London (GB)
Assigned to Snap Inc., Santa Monica, CA (US)
Filed by Snap Inc., Santa Monica, CA (US)
Filed on Nov. 13, 2020, as Appl. No. 16/949,781.
Claims priority of provisional application 62/936,272, filed on Nov. 15, 2019.
Prior Publication US 2021/0150806 A1, May 20, 2021
Int. Cl. G06T 17/00 (2006.01); G06V 20/64 (2022.01)
CPC G06T 17/00 (2013.01) [G06V 20/64 (2022.01)] 20 Claims
OG exemplary drawing
 
1. A computer-implemented method of generating a 3D body model, the method comprising:
receiving a plurality of bone scale coefficients each corresponding to respective bones of a skeleton model;
receiving a plurality of joint angle coefficients that collectively define a pose for the skeleton model;
obtaining a mean skeleton and a body template that represents an average population;
adjusting each individual bone length of the mean skeleton based on the plurality of bone scale coefficients to generate a scaled skeleton model in parallel with correcting the body template based on the plurality of bone scale coefficients, wherein some components of the mean skeleton are reduced in size to match the respective bones of the skeleton model while other components of the mean skeleton remain unchanged in size;
after adjusting each individual bone length of the mean skeleton model to generate the scaled skeleton model, generating a posed skeleton model based on the scaled skeleton model and the received joint angle coefficients;
generating an identity surface by deformation of a base surface comprising the corrected body template;
generating the 3D body model by mapping the identity surface onto the posed skeleton model; and
training a 3D model generator to generate the 3D body model by performing training operations comprising:
obtaining a collection of 3D scans that depict various persons in different poses and shapes, each 3D scan of the collection of 3D scans being associated with a set of anatomical landmarks that have been localized in 3D;
fitting an individual template to the set of landmarks by gradient descent on joint angles and bone scales to minimize 3D distances between landmark positions and respective template vertices to generate a collection of bone scales and parameters;
refining the fitting of the individual template by registering a prediction of each of the parameters to each 3D scan; and
minimizing a function based on a difference between a ground truth template and the refined fitted individual template.