US 12,260,522 B2
Low light and thermal image normalization for advanced fusion
Raymond Kirk Price, Redmond, WA (US); Michael Bleyer, Seattle, WA (US); and Christopher Douglas Edmonds, Carnation, WA (US)
Assigned to Microsoft Technology Licensing, LLC, Redmond, WA (US)
Filed by Microsoft Technology Licensing, LLC, Redmond, WA (US)
Filed on Oct. 2, 2023, as Appl. No. 18/375,879.
Application 18/375,879 is a division of application No. 17/306,681, filed on May 3, 2021, granted, now 11,810,278.
Prior Publication US 2024/0029206 A1, Jan. 25, 2024
Int. Cl. G06T 5/50 (2006.01); G06T 3/40 (2024.01); G06T 5/70 (2024.01); G06T 7/13 (2017.01); G06T 7/33 (2017.01); G06T 7/40 (2017.01); H04N 5/265 (2006.01); H04N 23/90 (2023.01)
CPC G06T 5/50 (2013.01) [G06T 3/40 (2013.01); G06T 5/70 (2024.01); G06T 7/13 (2017.01); G06T 7/33 (2017.01); G06T 7/40 (2013.01); H04N 5/265 (2013.01); H04N 23/90 (2023.01); G06T 2207/10024 (2013.01); G06T 2207/10048 (2013.01); G06T 2207/20221 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A computer system that mitigates effects of noise when fusing multiple images together to generate an enhanced image, said computer system comprising:
a first camera of a first modality;
a second camera of a second modality;
one or more processors; and
one or more computer-readable hardware storage devices that store instructions that are executable by the one or more processors to cause the computer system to:
trigger generation of a first image of an environment using the first camera;
trigger generation of a second image of the environment using the second camera;
identify pixels that are common between the first image and the second image;
determine a first set of textures for the common pixels in the first image;
determine a second set of textures for the common pixels in the second image;
identify a gain setting of the first camera;
based on the identified gain setting, apply a scaling factor to the first set of textures to generate a scaled set of textures, wherein applying the scaling factor operates to mitigate an effect of noise that is potentially present in the first image;
use the scaled set of textures to determine a first saliency of the first image, wherein the first saliency reflects an amount of texture variation that is present in the scaled set of textures;
determine a second saliency of the second image, wherein the second saliency reflects an amount of texture variation that is present in the second set of textures;
generate an alpha map that reflects edge detection weights that have been computed for each one of the common pixels, wherein the alpha map is based on the first saliency and the second saliency; and
based on the alpha map, merge textures from the common pixels included in the first image and the second image to generate a fused enhanced image.