US 12,394,154 B2
Body mesh reconstruction from RGB image
Riza Alp Guler, London (GB); Frank Lu, London (GB); Georgios Papandreou, London (GB); and Haoyang Wang, London (GB)
Assigned to SNAP INC., Santa Monica, CA (US)
Filed by Snap Inc., Santa Monica, CA (US)
Filed on Jun. 22, 2023, as Appl. No. 18/339,780.
Claims priority of application No. 20230100328 (GR), filed on Apr. 13, 2023.
Prior Publication US 2024/0346763 A1, Oct. 17, 2024
Int. Cl. G06T 17/20 (2006.01); G06T 7/73 (2017.01); G06T 13/40 (2011.01); G06T 19/00 (2011.01)
CPC G06T 17/20 (2013.01) [G06T 7/73 (2017.01); G06T 13/40 (2013.01); G06T 19/006 (2013.01); G06T 2207/20081 (2013.01); G06T 2207/20132 (2013.01); G06T 2207/30201 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A method comprising:
accessing a monocular image depicting an object;
predicting both a volumetric reconstruction tensor of the monocular image and a pose of the object by applying a first machine learning model to the monocular image;
identifying a portion of the pose of the object that corresponds to a point in a canonical space associated with a set of position encoding information;
obtaining a point of the volumetric reconstruction tensor corresponding to the identified portion of the pose;
classifying the obtained point as being inside or outside of a canonical volume by applying a second machine learning model to the obtained point of the volumetric reconstruction tensor together with the set of position encoding information; and
generating a three-dimensional (3D) mesh representing the object in the canonical space in response to classifying the obtained point as being inside or outside of the canonical volume.