US 12,333,427 B2
Multi-scale output techniques for generative adversarial networks
Cameron Smith, Santa Cruz, CA (US); Ratheesh Kalarot, San Jose, CA (US); Wei-An Lin, San Jose, CA (US); Richard Zhang, San Francisco, CA (US); Niloy Mitra, London (GB); Elya Shechtman, Seattle, WA (US); Shabnam Ghadar, Menlo Park, CA (US); Zhixin Shu, San Jose, CA (US); Yannick Hold-Geoffrey, San Jose, CA (US); Nathan Carr, San Jose, CA (US); Jingwan Lu, Santa Clara, CA (US); Oliver Wang, Seattle, WA (US); and Jun-Yan Zhu, San Jose, CA (US)
Assigned to Adobe Inc., San Jose, CA (US)
Filed by Adobe Inc., San Jose, CA (US)
Filed on Jul. 23, 2021, as Appl. No. 17/384,283.
Claims priority of provisional application 63/092,980, filed on Oct. 16, 2020.
Prior Publication US 2022/0122222 A1, Apr. 21, 2022
Int. Cl. G06N 3/08 (2023.01); G06F 3/04845 (2022.01); G06F 3/04847 (2022.01); G06F 18/21 (2023.01); G06F 18/211 (2023.01); G06F 18/214 (2023.01); G06F 18/40 (2023.01); G06N 3/045 (2023.01); G06N 20/20 (2019.01); G06T 3/02 (2024.01); G06T 3/18 (2024.01); G06T 3/40 (2024.01); G06T 3/4038 (2024.01); G06T 3/4046 (2024.01); G06T 5/20 (2006.01); G06T 5/77 (2024.01); G06T 11/00 (2006.01); G06T 11/60 (2006.01); G06V 10/28 (2022.01); G06V 10/98 (2022.01)
CPC G06N 3/08 (2013.01) [G06F 3/04845 (2013.01); G06F 3/04847 (2013.01); G06F 18/211 (2023.01); G06F 18/214 (2023.01); G06F 18/2163 (2023.01); G06F 18/40 (2023.01); G06N 3/045 (2023.01); G06N 20/20 (2019.01); G06T 3/02 (2024.01); G06T 3/18 (2024.01); G06T 3/40 (2013.01); G06T 3/4038 (2013.01); G06T 3/4046 (2013.01); G06T 5/20 (2013.01); G06T 5/77 (2024.01); G06T 11/001 (2013.01); G06T 11/60 (2013.01); G06V 10/28 (2022.01); G06V 10/98 (2022.01); G06T 2207/10024 (2013.01); G06T 2207/20081 (2013.01); G06T 2207/20084 (2013.01); G06T 2207/20221 (2013.01); G06T 2210/22 (2013.01)] 14 Claims
OG exemplary drawing
 
1. A computer-implemented method comprising:
producing a latent space representation of an input image;
accessing a generator neural network comprising an input layer, an output layer, a plurality of intermediate layers, and coupled to an auxiliary neural network comprising a set of residual layers, wherein the auxiliary neural network is coupled to an intermediate layer of the plurality of intermediate layers;
generating a first output image at a first resolution by providing the latent space representation of the input image as input to the generator neural network to generate a first initial output of the intermediate layer, performing optimization including minimizing a loss function with respect to the first initial output to generate a first optimized output, and providing the first optimized output to the auxiliary neural network via the intermediate layer, which outputs the first output image;
causing display of the first output image as a rapid preview image;
generating a second output image at a second resolution higher than the first resolution by providing the latent space representation of the input image as input to the generator neural network to generate a second initial output of the output layer, performing optimization including minimizing a loss function with respect to the second initial output to generate a second optimized output, and providing the second optimized output to the output layer of the generator neural network, which outputs the first output image; and
subsequent to causing display of the first output image as a rapid preview image, causing display of the second output image as a higher resolution output image.