US 12,367,626 B2
Modifying two-dimensional images utilizing three-dimensional meshes of the two-dimensional images
Radomir Mech, Mountain View, CA (US); Nathan Carr, San Jose, CA (US); and Matheus Gadelha, San Jose, CA (US)
Assigned to Adobe Inc., San Jose, CA (US)
Filed by Adobe Inc., San Jose, CA (US)
Filed on Nov. 15, 2022, as Appl. No. 18/055,584.
Prior Publication US 2024/0161366 A1, May 16, 2024
This patent is subject to a terminal disclaimer.
Int. Cl. G06T 11/60 (2006.01); G06T 7/70 (2017.01); G06T 17/20 (2006.01); G06T 19/20 (2011.01)
CPC G06T 11/60 (2013.01) [G06T 7/70 (2017.01); G06T 17/20 (2013.01); G06T 19/20 (2013.01); G06T 2219/2004 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A method comprising:
determining, by at least one processor utilizing one or more neural networks, pixel depth values representing estimated depths for pixels of a two-dimensional image;
generating, by the at least one processor utilizing the one or more neural networks, a three-dimensional mesh by incorporating depth information from the pixel depth values representing the estimated depths at the pixels of the two-dimensional image into a tessellation of the two-dimensional image according to camera parameters extracted from the two-dimensional image;
detecting, by the at least one processor, a displacement input at a two-dimensional position on the two-dimensional image;
modifying, by the at least one processor in response to the displacement input at the two-dimensional position on the two-dimensional image, the three-dimensional mesh by determining a displaced portion of the three-dimensional mesh according to a mapping of the two-dimensional position of the displacement input to a three-dimensional position of the three-dimensional mesh; and
generating, by the at least one processor, a modified two-dimensional image comprising a displacement of a portion of the two-dimensional image by re-rendering the two-dimensional image according to the displaced portion of the three-dimensional mesh and the camera parameters extracted from the two-dimensional image.