US 12,342,085 B2
Temporal reconstruction using raw image data and previous images
Mikko Strandborg, Hangonkylä (FI); and Mikko Ollila, Tampere (FI)
Assigned to Varjo Technologies Oy, Helsinki (FI)
Filed by Varjo Technologies Oy, Helsinki (FI)
Filed on Nov. 23, 2023, as Appl. No. 18/518,572.
Prior Publication US 2025/0175711 A1, May 29, 2025
Int. Cl. H04N 23/84 (2023.01); G06T 5/70 (2024.01); G06T 7/70 (2017.01); H04N 23/68 (2023.01)
CPC H04N 23/843 (2023.01) [G06T 5/70 (2024.01); G06T 7/70 (2017.01); H04N 23/683 (2023.01); G06T 2207/10016 (2013.01); G06T 2207/10024 (2013.01); G06T 2207/30244 (2013.01)] 15 Claims
OG exemplary drawing
 
1. An imaging system comprising:
an image sensor;
pose-tracking means; and
at least one processor configured to:
capture, via the image sensor, a sequence of images of a real-world environment, whilst processing pose-tracking data, collected by the pose-tracking means, to determine corresponding poses of the image sensor with respect to which said images of said sequence are captured, wherein when capturing a given image of said sequence, the at least one processor is configured to perform demosaicing on image data read out by the image sensor;
for at least one image of said sequence, identify at least one static region in the at least one image that represents at least one static object in the real-world environment;
obtain new image data read out by the image sensor, whilst processing the pose-tracking data, collected by the pose-tracking means, to determine a new pose of the image sensor with respect to which the new image data is read out by the image sensor;
reproject the at least one image from a corresponding pose of the image sensor with respect to which the at least one image is captured to the new pose of the image sensor, wherein the at least one static region in the at least one image is reprojected to generate at least one reprojected static region; and
generate a new image corresponding to the new image data, by performing temporal reconstruction of at least one static region in the new image that corresponds to the at least one static region of the at least one image, based on the at least one reprojected static region of the at least one reprojected image and a part of the new image data that corresponds to the at least one reprojected static region.