US 11,889,227 B2
Occlusion processing for frame rate conversion using deep learning
Petr Pohl, Lobnya (RU); Sergei Dmitrievich Ilichev, Moscow (RU); Igor Mironovich Kovliga, Moscow (RU); and Kristina Olegovna Rakova, Omsk (RU)
Assigned to SAMSUNG ELECTRONICS CO., LTD., Suwon-si (KR)
Filed by SAMSUNG ELECTRONICS CO., LTD., Suwon-si (KR)
Filed on Nov. 15, 2021, as Appl. No. 17/526,639.
Application 17/526,639 is a continuation of application No. PCT/KR2021/013597, filed on Oct. 5, 2021.
Claims priority of application No. 2020132721 (RU), filed on Oct. 5, 2020.
Prior Publication US 2022/0109807 A1, Apr. 7, 2022
Int. Cl. H04N 7/01 (2006.01); G06N 3/08 (2023.01); G06N 3/04 (2023.01)
CPC H04N 7/014 (2013.01) [G06N 3/04 (2013.01); G06N 3/08 (2013.01); H04N 7/0127 (2013.01); H04N 7/0137 (2013.01)] 15 Claims
OG exemplary drawing
 
1. A method for converting a frame rate of an input video, the method comprising:
performing a motion estimation by generating at least one motion field for at least one pair of frames of the input video, wherein the at least one motion field comprises a set of motion vectors for each block of a first reference frame of the input video, the set of motion vectors indicating to a second reference frame of the input video;
preparing first data for a predetermined interpolation phase for each block of an interpolated frame, wherein the first data comprise at least one parameter for a pre-trained occlusion processing (OcC) convolutional neural network (CNN), the at least one parameter being obtained based on the at least one motion field;
performing occlusion correction with the OcC CNN for each block of the interpolated frame by predicting weights associated with the set of motion vectors based on the prepared first data; and
performing motion compensation for each block or for each pixel of the interpolated frame by processing data from the OcC CNN.