US 11,711,628 B2
Systems and methods for obtaining color imagery using single photon avalanche diodes
Raymond Kirk Price, Redmond, WA (US); Michael Bleyer, Seattle, WA (US); and Christopher Douglas Edmonds, Carnation, WA (US)
Assigned to Microsoft Technology Licensing, LLC, Redmond, WA (US)
Filed by Microsoft Technology Licensing, LLC, Redmond, WA (US)
Filed on May 28, 2021, as Appl. No. 17/334,635.
Prior Publication US 2022/0385843 A1, Dec. 1, 2022
Int. Cl. H04N 25/702 (2023.01); G06T 3/40 (2006.01); G06T 5/20 (2006.01); H04N 9/64 (2023.01); H04N 23/84 (2023.01)
CPC H04N 25/702 (2023.01) [G06T 3/4015 (2013.01); G06T 5/20 (2013.01); H04N 9/646 (2013.01); H04N 23/843 (2023.01); G06T 2207/20028 (2013.01); H04N 2209/042 (2013.01)] 13 Claims
OG exemplary drawing
 
1. A system for obtaining color imagery using single photon avalanche diodes (SPADs), the system comprising:
a SPAD array comprising a plurality of SPAD pixels, each of the plurality of SPAD pixels comprising a respective color filter positioned thereover;
one or more processors; and
one or more hardware storage devices storing instructions that are executable by the one or more processors to configure the system to:
capture an image frame using the SPAD array;
generate a filtered image by performing a temporal filtering operation using the image frame and at least one preceding image frame, the at least one preceding image frame being captured by the SPAD array at a timepoint that temporally precedes a timepoint associated with the image frame wherein performing the temporal filtering operation comprises:
generating a plurality of demultiplexed image frames by demultiplexing the image frame, each of the plurality of demultiplexed image frames being associated with a respective color channel;
accessing a plurality of demultiplexed preceding image frames, the plurality of demultiplexed preceding image frames being generated by demultiplexing the at least one preceding image frame, the plurality of demultiplexed preceding image frames comprising, for each particular demultiplexed image frame of the plurality of demultiplexed image frames, a corresponding demultiplexed preceding image frame associated with a same color channel as the particular demultiplexed image frame; and
generating a plurality of temporally filtered demultiplexed image frames by, for each particular demultiplexed image frame of the plurality of demultiplexed image frames:
generating a corresponding aligned demultiplexed preceding image frame by using motion data associated with the SPAD array to align the corresponding demultiplexed preceding image frame with the particular demultiplexed image frame; and
compositing the particular demultiplexed image frame with the corresponding aligned demultiplexed preceding image frame; and
after performing the temporal filtering operation, generate a color image by demosaicing the filtered image.