US 11,669,986 B2
Generating enhanced three-dimensional object reconstruction models from sparse set of object images
Sai Bi, La Jolla, CA (US); Zexiang Xu, San Jose, CA (US); Kalyan Krishna Sunkavalli, San Jose, CA (US); David Jay Kriegman, San Diego, CA (US); and Ravi Ramamoorthi, Carlsbad, CA (US)
Assigned to ADOBE INC., San Jose, CA (US); and THE REGENTS OF THE UNIVERSITY OF CALIFORNIA, Oakland, CA (US)
Filed by ADOBE INC., San Jose, CA (US)
Filed on Apr. 16, 2021, as Appl. No. 17/233,122.
Prior Publication US 2022/0343522 A1, Oct. 27, 2022
Int. Cl. G06T 7/514 (2017.01); G06T 17/20 (2006.01); H04N 13/111 (2018.01); H04N 13/282 (2018.01); H04N 13/128 (2018.01)
CPC G06T 7/514 (2017.01) [G06T 17/20 (2013.01); H04N 13/111 (2018.05); H04N 13/128 (2018.05); H04N 13/282 (2018.05); G06T 2207/10012 (2013.01); G06T 2207/10028 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A method for a reconstruction of a three-dimensional (3D) physical object, the method comprising:
receiving a set of images, wherein each image of the set of images depicts the physical object from a separate viewpoint of a set of viewpoints;
generating, based on the set of image, a set of per-view (PV) reflectance maps, a set of composite feature maps, and a coarse geometry model encoding a coarse representation of a geometry of a surface of the physical object, wherein each PV reflectance map of the set of PV reflectance maps corresponds to one of the viewpoints of the set of viewpoints and each composite feature map of the set of composite feature maps corresponds to one of the viewpoints of the set of viewpoints and is based a composite of the set of images warped to the corresponding viewpoint;
aggregating the set of PV reflectance maps across the set of viewpoints to generate a coarse reflectance model that includes a set of estimated reflectance parameters for each vertex of the coarse geometry model and for each viewpoint of the set of viewpoints; and
jointly refining, based on the set of composite feature maps, the coarse geometry model and the coarse reflectance model to generate a refined geometry model and a refined reflectance model, wherein the refined geometry model encodes a refined representation of the geometry and the refined reflectance models encodes a refined representation of the reflectance property, wherein the coarse geometry model and the coarse reflectance model are refined by fusing the set of estimated reflectance parameters for each vertex to reconstruct geometry and reflectance associated with the physical object.