US 11,756,177 B2
Temporal filtering weight computation
Raymond Kirk Price, Redmond, WA (US); Christopher Douglas Edmonds, Carnation, WA (US); Michael Bleyer, Seattle, WA (US); and Christian Markus Maekelae, Redmond, WA (US)
Assigned to Microsoft Technology Licensing, LLC
Filed by Microsoft Technology Licensing, LLC, Redmond, WA (US)
Filed on Nov. 11, 2021, as Appl. No. 17/524,274.
Prior Publication US 2023/0145672 A1, May 11, 2023
Int. Cl. G06T 5/50 (2006.01); G06T 3/40 (2006.01); G06T 5/00 (2006.01); H04N 23/68 (2023.01)
CPC G06T 5/50 (2013.01) [G06T 3/40 (2013.01); G06T 5/002 (2013.01); H04N 23/682 (2023.01); H04N 23/6812 (2023.01); G06T 2207/20216 (2013.01); G06T 2207/20224 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A method for performing temporal filtering, said method comprising a filtering weight computation that includes:
acquiring a first image having a first timestamp, the first image being generated by a camera;
acquiring a second image having a second timestamp, the second image also being generated by the camera, wherein the first timestamp is before the second timestamp such that the first image is a previously acquired image relative to the second image;
performing a motion compensation (MC) operation on the first image to account for motion that might have occurred during a time between the first timestamp and the second timestamp such that an MC image is generated;
generating a difference image using the MC image and the second image, the difference image reflecting differences in intensities that exist between corresponding pixels in both the MC image and the second image;
generating a local weight map based on the differences in intensities in the difference image;
generating a global weight map based on inertial measurement unit (IMU) data that was generated by an IMU, the IMU data reflecting both an angular movement of the camera and an acceleration of the camera during a period between the first timestamp and the second timestamp;
generating a final weight map based on a combination of the local weight map and the global weight map;
using the final weight map to generate a filtered image; and
during a subsequent iteration of the filtering weight computation, causing the filtered image to be used as the previously acquired image.