US 12,444,029 B2
Method and device for depth image completion
Zhaohui Lv, Xi'an (CN); Xiaolei Zhang, Xi'an (CN); and Mingming Fan, Xi'an (CN)
Assigned to SAMSUNG ELECTRONICS CO., LTD., Suwon-si (KR)
Filed by SAMSUNG ELECTRONICS CO., LTD., Suwon-si (KR)
Filed on Jan. 30, 2023, as Appl. No. 18/103,281.
Claims priority of application No. 202210112535.2 (CN), filed on Jan. 29, 2022; and application No. 10-2022-0178638 (KR), filed on Dec. 19, 2022.
Prior Publication US 2023/0245282 A1, Aug. 3, 2023
Int. Cl. G06T 7/55 (2017.01); G06T 5/00 (2024.01); G06T 5/50 (2006.01); G06T 5/77 (2024.01)
CPC G06T 5/77 (2024.01) [G06T 5/50 (2013.01); G06T 7/55 (2017.01); G06T 2207/10024 (2013.01); G06T 2207/10028 (2013.01); G06T 2207/20081 (2013.01); G06T 2207/20084 (2013.01); G06T 2207/20221 (2013.01)] 18 Claims
OG exemplary drawing
 
1. A method for depth image completion using a depth image completion model comprising a first neural network (NN) and a second NN, the method comprising:
obtaining a first depth image using the first NN based on an original color image corresponding to an original depth image;
obtaining a second depth image using the second NN based on the original depth image; and
generating a final depth image by fusing the first depth image and the second depth image,
wherein the first NN comprises a first encoder network comprising cascaded first N-layer residual blocks, and a first decoder network comprising cascaded second N-layer residual blocks, and wherein the second NN comprises a second encoder network comprising cascaded third N-layer residual block, and a second decoder network comprising cascaded fourth N-layer residual blocks, wherein
the depth image completion model further comprises an attention processing unit, and
wherein the generating of the final depth image by fusing the first depth image and the second depth image comprises:
obtaining a first pixel weight map of the first depth image and a second pixel weight map of the second depth image using the attention processing unit; and
obtaining the final depth image by weighting and summing the first depth image and the second depth image based on the first pixel weight map and the second pixel weight map.