US 11,734,797 B2
Iterative multiscale image generation using neural networks
Nal Emmerich Kalchbrenner, Amsterdam (NL); Daniel Belov, London (GB); Sergio Gomez Colmenarejo, London (GB); Aaron Gerard Antonius van den Oord, London (GB); Ziyu Wang, Markham (CA); Joao Ferdinando Gomes de Freitas, London (GB); and Scott Ellison Reed, Atlanta, GA (US)
Assigned to DeepMind Technologies Limited, London (GB)
Filed by DeepMind Technologies Limited, London (GB)
Filed on May 23, 2022, as Appl. No. 17/751,359.
Application 17/751,359 is a continuation of application No. 16/324,061, granted, now 11,361,403, previously published as PCT/EP2018/054614, filed on Feb. 26, 2018.
Claims priority of provisional application 62/463,538, filed on Feb. 24, 2017.
Prior Publication US 2022/0284546 A1, Sep. 8, 2022
This patent is subject to a terminal disclaimer.
Int. Cl. G06T 3/40 (2006.01); G06N 20/00 (2019.01); G06N 3/045 (2023.01)
CPC G06T 3/4046 (2013.01) [G06N 3/045 (2023.01); G06N 20/00 (2019.01); G06T 3/4076 (2013.01)] 34 Claims
OG exemplary drawing
 
1. A method of generating an output image having an output resolution, each pixel in the output image having a respective value for each of one or more channels, the method comprising:
obtaining a low-resolution version of the output image; and
upscaling the low-resolution version of the output image to generate the output image having the output resolution by repeatedly performing the following operations until an image with an output resolution is obtained:
obtaining a current version of the output image having a current resolution; and
processing the current version of the output image using a set of convolutional neural networks that are specific to the current resolution to generate an updated version of the output image having an updated resolution that is higher than the current resolution,
wherein the set of convolutional neural networks that are specific to the current resolution comprises:
a first convolutional neural network that is configured to receive a first input comprising the current version of the image and to generate a first output image that includes columns of pixels from an intermediate version of the output image having an intermediate resolution that is higher than the current resolution but lower than the updated resolution, and
a second convolutional neural network that is configured to receive a second input comprising the intermediate version of the output image and to generate a second output image that includes rows of pixels from the updated version of the output image.