	US 12,283,029 B2
	Image inpainting method and electronic device
Yan Sun, Beijing (CN); Yunhe Tong, Beijing (CN); Tiankuo Shi, Beijing (CN); Xue Chen, Beijing (CN); Yanhui Xi, Beijing (CN); Yuxin Bi, Beijing (CN); Anjie Wang, Beijing (CN); Songchao Tan, Beijing (CN); and Xiaomang Zhang, Beijing (CN)
Assigned to BOE TECHNOLOGY GROUP CO., LTD., Beijing (CN); and PEKING UNIVERSITY, Beijing (CN)
Appl. No. 17/907,849
Filed by BOE TECHNOLOGY GROUP CO., LTD., Beijing (CN); and Peking University, Beijing (CN)
PCT Filed Mar. 12, 2021, PCT No. PCT/CN2021/080464 § 371(c)(1), (2) Date Aug. 29, 2022, PCT Pub. No. WO2021/180204, PCT Pub. Date Sep. 16, 2021.
Claims priority of application No. 202010169599.7 (CN), filed on Mar. 12, 2020.
Prior Publication US 2023/0125649 A1, Apr. 27, 2023
Int. Cl. G06T 5/77 (2024.01); G06T 7/11 (2017.01); G06T 7/40 (2017.01); G06T 7/50 (2017.01); G06V 10/74 (2022.01)

CPC G06T 5/77 (2024.01) [G06T 7/11 (2017.01); G06T 7/40 (2013.01); G06T 7/50 (2017.01); G06V 10/761 (2022.01)]

13 Claims

1. An image inpainting method, comprising:

acquiring an image to be inpainted based on both depth information and texture information of a reference image, the image to be inpainted including at least one region to be inpainted; wherein each region to be inpainted includes at least one pixel point to be inpainted with a pixel value of zero;

determining at least one reference block matching the at least one region to be inpainted respectively in the reference image; and

inpainting the at least one region to be inpainted according to the at least one reference block respectively to acquire a composite image;

wherein determining the at least one reference block matching the at least one region to be inpainted respectively in the reference image, includes:

determining priorities of all pixel points to be inpainted in a region to be inpainted;

acquiring a target pixel point with a highest priority in all the pixel points to be inpainted;

acquiring a first preset region in the region to be inpainted with the target pixel point as a center; and

matching the first preset region with regions of the reference image, and using a region of the reference image that matches the first preset region as a reference block;

wherein matching the first preset region with the regions of the reference image, and using the region of the reference image that matches the first preset region as the reference block, includes:

acquiring a first reference pixel point corresponding to the target pixel point in the reference image;

acquiring a second preset region of the reference image with the first reference pixel point as a center;

determining a first offset matrix according to the first preset region and the second preset region;

acquiring, for all second reference pixel points in the reference image, third preset regions of the reference image each with a second reference pixel point as a center; the second reference pixel point being any pixel point in the reference image;

determining a second offset matrix according to the first preset region and each third preset region;

determining an offset factor according to the first offset matrix and the second offset matrix;

acquiring a grayscale similarity factor between the second reference pixel point and the target pixel point;

determining a similarity between the first preset region and the third preset region according to the offset factor and the grayscale similarity factor; and

among the third preset regions determined according to all the second reference pixel points, using a preset region with a smallest similarity as the reference block.