US 11,869,274 B2
Face-based frame rate upsampling for video calls
Dillon Cower, Woodinville, WA (US)
Assigned to Google LLC, Mountain View, CA (US)
Filed by Google LLC, Mountain View, CA (US)
Filed on Mar. 29, 2022, as Appl. No. 17/707,661.
Application 17/707,661 is a continuation of application No. 16/968,115, granted, now 11,321,555, previously published as PCT/US2019/060700, filed on Nov. 11, 2019.
Claims priority of provisional application 62/884,135, filed on Aug. 7, 2019.
Prior Publication US 2022/0222968 A1, Jul. 14, 2022
This patent is subject to a terminal disclaimer.
Int. Cl. G06V 40/16 (2022.01); G06T 7/246 (2017.01); G06T 5/00 (2006.01); G06T 5/50 (2006.01); G06V 20/40 (2022.01); G06T 3/40 (2006.01)
CPC G06V 40/172 (2022.01) [G06T 3/4007 (2013.01); G06T 5/002 (2013.01); G06T 5/50 (2013.01); G06T 7/248 (2017.01); G06V 20/40 (2022.01); G06T 2207/10016 (2013.01); G06T 2207/20212 (2013.01); G06T 2207/30201 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A computer-implemented method comprising:
receiving a set of video frames that correspond to a video, the set of video frames including a first video frame and a second video frame that each include a face, wherein the second video frame is subsequent to the first video frame;
performing face tracking on the first video frame to identify a first face resampling keyframe;
performing face tracking on the second video frame to identify a second face resampling keyframe;
deriving an interpolation amount;
interpolating a first background of the first face resampling keyframe and a second background of the second face resampling keyframe based on the interpolation amount;
rendering an interpolated first face and an interpolated second face; and
displaying a final frame that is based on the interpolated first background, the interpolated second background, the interpolated first face, and the interpolated second face.