US 12,333,692 B2
Modifying digital images via multi-layered scene completion facilitated by artificial intelligence
Daniil Pakhomov, San Jose, CA (US); Qing Liu, Santa Clara, CA (US); Zhihong Ding, Fremont, CA (US); Scott Cohen, Sunnyvale, CA (US); Zhe Lin, Fremont, CA (US); Jianming Zhang, Campbell, CA (US); Zhifei Zhang, San Jose, CA (US); Ohiremen Dibua, Newark, NJ (US); Mariette Souppe, San Jose, CA (US); Krishna Kumar Singh, San Jose, CA (US); and Jonathan Brandt, Santa Cruz, CA (US)
Assigned to Adobe Inc., San Jose, CA (US)
Filed by Adobe Inc., San Jose, CA (US)
Filed on Sep. 1, 2023, as Appl. No. 18/460,365.
Application 18/460,365 is a continuation in part of application No. 18/320,664, filed on May 19, 2023.
Application 18/320,664 is a continuation in part of application No. 18/190,500, filed on Mar. 27, 2023.
Application 18/190,500 is a continuation in part of application No. 18/058,601, filed on Nov. 23, 2022.
Application 18/190,500 is a continuation in part of application No. 18/058,630, filed on Nov. 23, 2022, granted, now 12,045,963.
Application 18/190,500 is a continuation in part of application No. 18/058,575, filed on Nov. 23, 2022.
Application 18/190,500 is a continuation in part of application No. 18/058,554, filed on Nov. 23, 2022.
Application 18/190,500 is a continuation in part of application No. 18/058,622, filed on Nov. 23, 2022.
Application 18/190,500 is a continuation in part of application No. 18/058,538, filed on Nov. 23, 2022.
Claims priority of provisional application 63/378,616, filed on Oct. 6, 2022.
Prior Publication US 2024/0135514 A1, Apr. 25, 2024
Int. Cl. G06T 5/00 (2024.01); G06F 3/04845 (2022.01); G06T 5/70 (2024.01); G06T 5/77 (2024.01); G06T 7/11 (2017.01); G06T 7/194 (2017.01); G06T 7/70 (2017.01)
CPC G06T 5/77 (2024.01) [G06F 3/04845 (2013.01); G06T 5/70 (2024.01); G06T 7/11 (2017.01); G06T 7/194 (2017.01); G06T 7/70 (2017.01); G06T 2200/24 (2013.01); G06T 2207/20021 (2013.01); G06T 2207/20084 (2013.01); G06T 2207/20092 (2013.01)] 18 Claims
OG exemplary drawing
 
1. A system comprising:
at least one memory device comprising a segmentation neural network and a diffusion neural network; and
at least one processor configured to cause the system to:
generate, using the segmentation neural network, an occluder mask for a first object portrayed in a digital image and an incomplete object corresponding to a second object portrayed in the digital image, a portion of the second object being occluded by the first object;
generate, using the diffusion neural network, an at least partially denoised image based on the occluder mask, the incomplete object corresponding to the second object, and a noise input;
generate, from the at least partially denoised image, a completed second object mask that comprises a completed mask portion corresponding to the portion of the second object occluded by the first object;
generate, from the at least partially denoised image, a completed second object that includes a completed portion corresponding to the portion of the second object occluded by the first object; and
modify, in response to one or more user interactions, the digital image by moving or deleting the first object and exposing the completed portion of the completed second object.