US 11,989,897 B2
Depth map generation from sparse depth samples in an augmented reality environment
Bing Zhou, Rye, NY (US); and Sinem Guven Kaya, New York, NY (US)
Assigned to INTERNATIONAL BUSINESS MACHINES CORPORATION, Armonk, NY (US)
Filed by International Business Machines Corporation, Armonk, NY (US)
Filed on Mar. 16, 2021, as Appl. No. 17/202,839.
Prior Publication US 2022/0301205 A1, Sep. 22, 2022
Int. Cl. G06T 7/55 (2017.01); G06T 17/20 (2006.01); G06T 19/00 (2011.01)
CPC G06T 7/55 (2017.01) [G06T 17/20 (2013.01); G06T 19/006 (2013.01); G06T 2207/10028 (2013.01); G06T 2207/20081 (2013.01); G06T 2207/20084 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A system, comprising:
a processor that executes computer executable components stored in memory, the computer executable components comprising:
a depth completion component that:
generates, using a model, a first depth map from an image and sparse depth samples, wherein the model is trained using a semantic edge-weighted loss function, and wherein generating the first depth map comprises:
generating, using a trained semantic segmentation network, respective semantic segmentation mask images of objects in the image,
generating, using an edge detection process, respective edge images of the objects from the respective semantic segmentation mask images, and
assigning, using the semantic edge-weighted loss function, respective weights to pixels along edges of the objects in the edge images based on an exponential density function of respective losses of the pixels; and
a semantic mesh deformation component that performs a semantic mesh deformation process on the first depth map, using the sparse depth samples, to generate a second depth map comprising a defined image accuracy.