US 11,989,846 B2
Mixture of volumetric primitives for efficient neural rendering
Stephen Anthony Lombardi, Pittsburgh, PA (US); Tomas Simon Kreuz, Pittsburgh, PA (US); Jason Saragih, Pittsburgh, PA (US); Gabriel Bailowitz Schwartz, Seattle, WA (US); Michael Zollhoefer, Pittsburgh, PA (US); and Yaser Sheikh, Pittsburgh, PA (US)
Assigned to Meta Platforms Technologies, LLC, Menlo Park, CA (US)
Filed by Meta Platforms Technologies, LLC, Menlo Park, CA (US)
Filed on Dec. 17, 2021, as Appl. No. 17/554,992.
Claims priority of provisional application 63/141,862, filed on Jan. 26, 2021.
Prior Publication US 2022/0245910 A1, Aug. 4, 2022
Int. Cl. G06T 19/20 (2011.01); G06T 15/06 (2011.01); G06T 17/20 (2006.01)
CPC G06T 19/20 (2013.01) [G06T 15/06 (2013.01); G06T 17/20 (2013.01); G06T 2219/2012 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A computer-implemented method, comprising:
collecting multiple images of a subject, the images of the subject including one or more different angles of view of the subject;
selecting a plurality of vertex positions in a guide mesh, indicative of multiple vertices of a one or more volumetric primitives enveloping the subject;
determining a geometric attribute for each of the one or more volumetric primitives, the geometric attribute including a position, a rotation, and a scale factor of the one or more volumetric primitives;
determining a payload attribute for each of the one or more volumetric primitives, the payload attribute including a color value and an opacity value for each voxel in a voxel grid defining the one or more volumetric primitives;
determining a loss factor for each point in the one or more volumetric primitives based on the geometric attribute, the payload attribute and a ground truth value; and
updating a three-dimensional model for the subject according to the loss factor, the three-dimensional model including the one or more volumetric primitives.