US 12,488,481 B2
Video reconstruction from videos with ultra-low frame-per-second
Xiaoyu Xiang, Santa Clara, CA (US); Ingrid Anda Cotoros, Hillsborough, CA (US); Rakesh Ranjan, Sunnyvale, CA (US); and Vikas Chandra, Fremont, CA (US)
Assigned to Meta Platforms, Inc., Menlo Park, CA (US)
Filed by Meta Platforms, Inc., Menlo Park, CA (US)
Filed on Aug. 12, 2022, as Appl. No. 17/887,034.
Prior Publication US 2024/0054664 A1, Feb. 15, 2024
Int. Cl. G06T 7/38 (2017.01); G06T 7/33 (2017.01); G06V 10/80 (2022.01)
CPC G06T 7/38 (2017.01) [G06T 7/33 (2017.01); G06V 10/806 (2022.01); G06T 2207/10016 (2013.01)] 19 Claims
OG exemplary drawing
 
1. A method comprising:
accessing a video captured by one or more cameras, wherein the accessed video is associated with a first framerate lower than a threshold framerate;
for any two adjacent frames of the accessed video:
generating respective warped frames for each frame of the any two adjacent frames based on a respective optical flow associated with each frame of the any two adjacent frames;
fusing the respective warped frames to generate an intermediate frame;
determining respective alignments associated with each frame of the any two adjacent frames based on aligning each of the any two adjacent frames to the intermediate frame;
fusing the respective determined alignments to generate residual information associated with the intermediate frame; and
generating a reconstructed frame based on adding the intermediate frame with the residual information; and
reconstructing the accessed video based on the any two adjacent frames and their respective reconstructed frame, wherein the reconstructed accessed video is associated with a second framerate higher than the threshold framerate.