US 11,973,973 B2
Prediction refinement based on optical flow
Kai Zhang, San Diego, CA (US); Li Zhang, San Diego, CA (US); Hongbin Liu, Beijing (CN); Jizheng Xu, San Diego, CA (US); and Yue Wang, Beijing (CN)
Assigned to BEIJING BYTEDANCE NETWORK TECHNOLOGY CO., LTD, Beijing (CN); and BYTEDANCE INC., Los Angeles, CA (US)
Filed by Beijing Bytedance Network Technology Co., Ltd., Beijing (CN); and Bytedance Inc., Los Angeles, CA (US)
Filed on Sep. 16, 2021, as Appl. No. 17/476,637.
Application 17/476,637 is a continuation of application No. PCT/CN2020/079662, filed on Mar. 17, 2020.
Claims priority of application No. PCT/CN2019/078411 (WO), filed on Mar. 17, 2019; application No. PCT/CN2019/078501 (WO), filed on Mar. 18, 2019; application No. PCT/CN2019/078719 (WO), filed on Mar. 19, 2019; and application No. PCT/CN2019/079961 (WO), filed on Mar. 27, 2019.
Prior Publication US 2022/0007050 A1, Jan. 6, 2022
Int. Cl. H04N 19/52 (2014.01); H04N 19/107 (2014.01); H04N 19/132 (2014.01); H04N 19/139 (2014.01); H04N 19/176 (2014.01); H04N 19/184 (2014.01); H04N 19/186 (2014.01); H04N 19/577 (2014.01)
CPC H04N 19/52 (2014.11) [H04N 19/107 (2014.11); H04N 19/132 (2014.11); H04N 19/139 (2014.11); H04N 19/176 (2014.11); H04N 19/184 (2014.11); H04N 19/186 (2014.11); H04N 19/577 (2014.11)] 20 Claims
OG exemplary drawing
 
1. A method of processing video data, comprising:
determining a refined prediction sample P′(x,y) at a position (x,y) in a video block by modifying a prediction sample P(x,y) at the position (x,y) with a first gradient component Gx(x, y) in a first direction estimated at the position (x,y) and a second gradient component Gy(x, y) in a second direction estimated at the position (x,y) and a first motion displacement Vx(x,y) estimated for the position (x,y) and a second motion displacement Vy(x,y) estimated for the position (x,y), wherein x and y are integer numbers;
determining the first motion displacement Vx(x,y) at the position (x,y) and the second motion displacement Vy(x,y) at the position (x,y) in the video block based on information from at least a spatial neighboring video block of the video block or from at least a temporal neighboring video block located in time relative to that of the video block, wherein (Vx(x,y), Vy(x,y)) is equal to f(MVMix, MVc), wherein f is a function, and wherein MVc is a motion vector of the video block; and
performing a conversion between the video block and a bitstream of the video block using a reconstructed sample value Rec(x,y) at the position (x,y) that is obtained based on the refined prediction sample P′(x,y) and a residue sample value Res(x,y).