CPC G06T 17/20 (2013.01) [G06T 5/20 (2013.01); G06T 5/70 (2024.01); G06T 15/506 (2013.01); G06T 2207/20021 (2013.01); G06T 2210/44 (2013.01); H04N 13/239 (2018.05)] | 18 Claims |
1. A method comprising:
receiving a plurality of object images of an object, each object image obtained under ambient lighting conditions using a handheld device and corresponding to a different view direction, wherein the plurality of object images comprises first and second object images corresponding to first and second directions;
determining a global coordinate space based on plurality of object images, and mapping out the relative positions and viewpoints corresponding to each object image;
determining a mesh corresponding to the target region of the object surface based on a first subset of the plurality of object images which comprises two or more object images of the plurality of object images;
determining diffuse and specular maps corresponding to the target region of the object surface based on processing a second subset of the object images using a deep learning neural network model trained to estimate diffuse and specular albedo components based on an input image, wherein the second subset comprises at least the first and second object images;
determining a tangent normal map corresponding to the target region of the object surface based on high-pass filtering each object image of the second subset;
storing and/or outputting the mesh, the diffuse map, the specular map and the tangent normal map;
wherein determining diffuse and specular maps corresponding to the target region of the object surface comprises:
for each of the second subset of the object images:
providing that object image as input to the deep learning neural network model and obtaining a corresponding camera-space diffuse map and a corresponding camera-space specular map as output.
|