US 12,266,039 B2
Target-augmented material maps
Valentin Deschaintre, London (GB); Yiwei Hu, New Haven, CT (US); Paul Guerrero, London (GB); and Milos Hasan, San Jose, CA (US)
Assigned to Adobe Inc., San Jose, CA (US)
Filed by Adobe Inc., San Jose, CA (US)
Filed on Nov. 11, 2022, as Appl. No. 17/985,579.
Prior Publication US 2024/0161362 A1, May 16, 2024
Int. Cl. G06T 11/60 (2006.01); G06T 3/40 (2024.01); G06T 9/00 (2006.01); G06V 10/74 (2022.01)
CPC G06T 11/60 (2013.01) [G06T 3/40 (2013.01); G06T 9/00 (2013.01); G06V 10/761 (2022.01)] 20 Claims
OG exemplary drawing
 
1. A method comprising:
accessing a scene, a target image including a target material appearance, and an input material map defining a coarse structure of the scene;
accessing a material generation prior produced using a generative adversarial network (GAN), wherein the material generation prior includes a statistical function;
encoding, based on the material generation prior and using the GAN, an input material appearance from the input material map to produce a projected latent vector;
optimizing a current value of the projected latent vector based on the material generation prior to minimize a statistical difference between the target image and a renderable image associated with the current value of the projected latent vector; and
rendering, responsive to the optimizing, the scene based on a final value of the projected latent vector and an output material map providing the target material appearance applied to the coarse structure of the scene.