US 12,190,440 B2
Method of generating multi-layer representation of scene and computing device implementing the same
Gleb Mikhaylovich Sterkin, Moscow (RU); Pavel Ilyich Solovev, Moscow (RU); Denis Mikhaylovich Korzhenkov, Moscow (RU); Victor Sergeevich Lempitsky, Yerevan (AM); and Taras Andreevich Khakhulin, Moscow (RU)
Assigned to SAMSUNG ELECTRONICS CO., LTD., Suwon-si (KR)
Filed by SAMSUNG ELECTRONICS CO., LTD., Suwon-si (KR)
Filed on Dec. 16, 2022, as Appl. No. 18/083,354.
Application 18/083,354 is a continuation of application No. PCT/KR2022/003655, filed on Mar. 16, 2022.
Claims priority of application No. RU2021106881 (RU), filed on Mar. 16, 2021.
Prior Publication US 2023/0123532 A1, Apr. 20, 2023
Int. Cl. G06T 15/50 (2011.01); G06T 7/50 (2017.01); G06T 7/90 (2017.01); G06T 15/08 (2011.01); G06T 15/10 (2011.01); G06T 17/20 (2006.01)
CPC G06T 15/50 (2013.01) [G06T 7/50 (2017.01); G06T 7/90 (2017.01); G06T 15/08 (2013.01); G06T 15/10 (2013.01); G06T 17/20 (2013.01); G06T 2207/10012 (2013.01); G06T 2207/10016 (2013.01); G06T 2207/10024 (2013.01); G06T 2207/20081 (2013.01); G06T 2207/20084 (2013.01); G06T 2207/20212 (2013.01); G06T 2207/30244 (2013.01)] 15 Claims
OG exemplary drawing
 
1. A method of generating a multi-layer representation of a scene, the method comprising:
obtaining a pair of images of the scene, the pair of the images comprising a reference image and a source image;
performing a reprojection operation on the pair of images to generate a plane-sweep volume;
predicting, using a geometry network, a layered structure of the scene based on the plane-sweep volume; and
estimating, using a coloring network, color values and opacity values for the predicted layered structure of the scene to obtain the multi-layer representation of the scene;
wherein the geometry network and the coloring network are trained in end-to-end manner.