US 12,407,829 B2
Transform coding based on matrix-based intra prediction
Zhipin Deng, Beijing (CN); Kai Zhang, San Diego, CA (US); Li Zhang, San Diego, CA (US); Hongbin Liu, Beijing (CN); and Jizheng Xu, San Diego, CA (US)
Assigned to BEIJING BYTEDANCE NETWORK TECHNOLOGY CO., LTD., Beijing (CN); and BYTEDANCE INC., Los Angeles, CA (US)
Filed by Beijing Bytedance Network Technology Co., Ltd., Beijing (CN); and Bytedance Inc., Los Angeles, CA (US)
Filed on Jun. 9, 2021, as Appl. No. 17/343,086.
Application 17/343,086 is a continuation of application No. PCT/CN2020/084472, filed on Apr. 13, 2020.
Claims priority of application No. PCT/CN2019/082424 (WO), filed on Apr. 12, 2019.
Prior Publication US 2021/0297672 A1, Sep. 23, 2021
Int. Cl. H04N 19/132 (2014.01); H04N 19/105 (2014.01); H04N 19/117 (2014.01); H04N 19/12 (2014.01); H04N 19/159 (2014.01); H04N 19/176 (2014.01); H04N 19/186 (2014.01); H04N 19/60 (2014.01); H04N 19/70 (2014.01); H04N 19/82 (2014.01)
CPC H04N 19/132 (2014.11) [H04N 19/105 (2014.11); H04N 19/117 (2014.11); H04N 19/12 (2014.11); H04N 19/159 (2014.11); H04N 19/176 (2014.11); H04N 19/186 (2014.11); H04N 19/60 (2014.11); H04N 19/70 (2014.11); H04N 19/82 (2014.11)] 20 Claims
OG exemplary drawing
 
1. A video processing method, comprising:
determining, for a conversion between a video block of a video and a bitstream of the video, that a first intra mode is applied on the video block of the video,
performing, in the first intra mode, a boundary downsampling operation on reference samples of the video block based on a size of the video block, followed by a matrix vector multiplication operation, and selectively followed by an upsampling operation to generate prediction samples for the video block, and
performing the conversion based on the prediction samples and residual samples of the video block,
wherein a secondary transform tool is applied on the video block, and a secondary transform matrix is selected to generate the residual samples for the video block, and
wherein the secondary transform tool includes applying, during encoding, a forward secondary transform to an output of a forward primary transform before applying a quantization process, or applying, during decoding, an inverse secondary transform to an output of a dequantization process before applying an inverse primary transform,
wherein in response to a width-height ratio of the video block being greater than 2, a context with an increment value of 3 is used for a first bin of a second syntax element,
wherein in response to the width-height ratio of the video block being smaller than or equal to 2, a single context selected from contexts with increment values of 0, 1 or 2 is used for the first bin of the second syntax element,
wherein the boundary downsampling operation includes deriving, according to a rule, boundary samples by applying a left bit shift operation or a right bit shift operation on a sum of at least one reference boundary sample, wherein the rule determines whether to apply the left bit shift operation or the right bit shift operation, wherein the rule defines that a right bit shift operation is applied in response to a number of shifted bits being greater than zero, and wherein the boundary samples redBdryS[x] are calculated using one of following equations:

OG Complex Work Unit Math
if bDwn>1, or redBdryS[x ]=refS[x ] if bDwn=1,
wherein bDwn is equal to a function of the size of the video block and a boundary size,
wherein refS [x] indicates a number of reference samples x,
wherein >> indicates the right bit shift operation, and
wherein << indicates the left bit shift operation,
wherein the boundary size is predefined based on the size of the video block, and
wherein bDwn is calculated as bDwn=nTbs/boundarySize, wherein nTbs and boundarySize represent the size of the video block and the boundary size, respectively.