US 12,112,457 B2
Selective extended depth-of-field correction for image reconstruction
Mikko Ollila, Tampere (FI); and Mikko Strandborg, Hangonkylä (FI)
Assigned to Varjo Technologies Oy, Helsinki (FI)
Filed by Varjo Technologies Oy, Helsinki (FI)
Filed on Nov. 21, 2022, as Appl. No. 17/991,343.
Prior Publication US 2024/0169492 A1, May 23, 2024
Int. Cl. G06T 5/73 (2024.01); G06F 3/01 (2006.01); G06T 5/50 (2006.01); G06T 7/55 (2017.01); G06T 15/20 (2011.01); G06T 17/00 (2006.01)
CPC G06T 5/73 (2024.01) [G06F 3/013 (2013.01); G06T 5/50 (2013.01); G06T 7/55 (2017.01); G06T 15/20 (2013.01); G06T 17/00 (2013.01); G06T 2207/20081 (2013.01); G06T 2207/20084 (2013.01)] 18 Claims
OG exemplary drawing
 
1. A system comprising at least one server and a data repository communicably coupled to the at least one server, wherein the at least one server is configured to:
receive images of a real-world environment that are captured using at least one camera, corresponding depth maps, and at least one of:
pose information indicative of corresponding poses of the at least one camera from which the images and the depth maps are captured,
relative pose information indicative of a relative pose of a given camera with respect to a pose of at least one other given camera during capturing of the images, wherein the at least one camera comprises the given camera and the at least one other given camera;
process the images to generate a three-dimensional model of the real-world environment, based on the corresponding depth maps and the at least one of: the pose information, the relative pose information;
store the three-dimensional model of the real-world environment at the data repository;
utilise the three-dimensional model of the real-world environment to generate an output image from a perspective of a new pose, the output image representing at least one given object present in the real-world environment;
determine whether an extended depth-of-field correction is required to be applied to any one of: at least one of the images captured by the at least one camera representing the at least one given object, the three-dimensional model of the real-world environment, the output image, based on whether an optical focus of the at least one camera was adjusted for capturing the at least one of the images according to an optical depth of the at least one given object from a given pose of the at least one camera from which the at least one of the images was captured; and
when it is determined that an extended depth-of-field correction is required to be applied, apply the extended depth-of-field correction to at least a portion of any one of: the at least one of the images captured by the at least one camera, the three-dimensional model of the real-world environment, the output image, wherein said portion to which the extended depth-of-field correction is applied represents the at least one given object or at least a part of the at least one given object.