US 12,204,610 B2
Learning parameters for generative inpainting neural networks utilizing object-aware training and masked regularization
Zhe Lin, Fremont, CA (US); Haitian Zheng, Rochester, NY (US); Jingwan Lu, Santa Clara, CA (US); Scott Cohen, Sunnyvale, CA (US); Jianming Zhang, Campbell, CA (US); Ning Xu, Milpitas, CA (US); Elya Shechtman, Seattle, WA (US); Connelly Barnes, Seattle, WA (US); and Sohrab Amirghodsi, Seattle, WA (US)
Assigned to Adobe Inc., San Jose, CA (US)
Filed by Adobe Inc., San Jose, CA (US)
Filed on Feb. 14, 2022, as Appl. No. 17/650,967.
Prior Publication US 2023/0259587 A1, Aug. 17, 2023
Int. Cl. G06K 9/00 (2022.01); G06F 18/214 (2023.01); G06N 3/08 (2023.01); G06T 5/77 (2024.01); G06T 7/11 (2017.01)
CPC G06F 18/2148 (2023.01) [G06N 3/08 (2013.01); G06T 5/77 (2024.01); G06T 7/11 (2017.01); G06T 2207/20081 (2013.01); G06T 2207/20084 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A non-transitory computer readable medium storing instructions thereon that, when executed by at least one processor, cause a computing device to perform operations comprising:
generating a set of object masks for objects within a digital image utilizing a segmentation model;
selecting a masked digital image from a set of masked digital images depicting masked object instances indicated by the set of object masks for the digital image;
generating an inpainted digital image from the masked digital image by filling a hole region indicated by a digital image mask of the masked digital image utilizing a generative inpainting neural network comprising a discriminator neural network;
comparing the inpainted digital image with the digital image utilizing a masked regularization from the digital image mask to penalize the discriminator neural network from overfitting by enforcing computation of gradient penalties on unmasked pixels outside of the digital image mask; and
modifying parameters of the generative inpainting neural network based on the inpainted digital image according to the masked regularization.