US 12,288,279 B2
Modifying digital images via scene-based editing using image understanding facilitated by artificial intelligence
Jonathan Brandt, Santa Cruz, CA (US); Scott Cohen, Sunnyvale, CA (US); Zhe Lin, Fremont, CA (US); Zhihong Ding, Fremont, CA (US); Darshan Prasad, San Jose, CA (US); Matthew Joss, Seattle, WA (US); Celso Gomes, Bellevue, WA (US); Jianming Zhang, Campbell, CA (US); Olena Soroka, Hamburg (DE); Klaas Stoeckmann, Hamburg (DE); Michael Zimmermann, Hamburg (DE); and Thomas Muehrke, Hamburg (DE)
Assigned to Adobe Inc., San Jose, CA (US)
Filed by Adobe Inc., San Jose, CA (US)
Filed on Nov. 23, 2022, as Appl. No. 18/058,538.
Prior Publication US 2024/0169624 A1, May 23, 2024
Int. Cl. G06T 11/60 (2006.01); G06F 3/048 (2013.01); G06F 3/04842 (2022.01); G06F 3/04845 (2022.01); G06T 11/40 (2006.01)
CPC G06T 11/60 (2013.01) [G06F 3/04842 (2013.01); G06F 3/04845 (2013.01); G06T 11/40 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A system comprising:
at least one memory device; and
at least one processor configured to cause the system to:
receive a digital image portraying a plurality of objects;
pre-process the digital image in response to receiving the digital image and before receiving a user interaction with the digital image by:
generating, using a segmentation neural network, an object mask for each object of the plurality of objects; and
generating, using a hole filling model, a completed background for the digital image that includes generated background pixels behind each object;
detect user input to move or delete an object of the plurality of objects; and
modify the digital image in response to the user input by:
moving or deleting the object in accordance with the user input; and
exposing a set of background pixels behind the object from the completed background.