US 12,260,493 B2
Method and device for multi-camera hole filling
Bertrand Nepveu, Montreal (CA); Vincent Chapdelaine-Couture, Carignan (CA); and Emmanuel Piuze-Phaneuf, Los Gatos, CA (US)
Assigned to APPLE INC., Cupertino, CA (US)
Filed by Apple Inc., Cupertino, CA (US)
Filed on Dec. 28, 2022, as Appl. No. 18/089,664.
Application 18/089,664 is a continuation of application No. PCT/US2021/036321, filed on Jun. 8, 2021.
Claims priority of provisional application 63/045,394, filed on Jun. 29, 2020.
Prior Publication US 2023/0377249 A1, Nov. 23, 2023
Int. Cl. G06T 15/20 (2011.01); G06T 3/18 (2024.01); G06T 5/70 (2024.01); G06T 7/50 (2017.01); G06T 7/70 (2017.01)
CPC G06T 15/205 (2013.01) [G06T 3/18 (2024.01); G06T 5/70 (2024.01); G06T 7/50 (2017.01); G06T 7/70 (2017.01); G06T 2207/10024 (2013.01); G06T 2207/10028 (2013.01)] 26 Claims
OG exemplary drawing
 
1. A method comprising:
obtaining a first image of an environment from a first image sensor communicatively coupled to a computing system with non-transitory memory and one or more processors, wherein the first image sensor is associated with first intrinsic parameters;
performing a warping operation on the first image according to perspective offset values to generate a warped first image in order to account for perspective differences between the first image sensor and a user of the computing system;
determining an occlusion mask based on the warped first image that includes a plurality of holes;
obtaining a second image of the environment from a second image sensor communicatively coupled to the computing system, wherein the second image sensor is associated with second intrinsic parameters;
normalizing the second image based on a difference between the first and second intrinsic parameters to produce a normalized second image; and
filling a first set of one or more holes of the occlusion mask based on the normalized second image to produce a modified first image.