US 12,033,084 B2
Computing photorealistic versions of synthetic images
Stephan Joachim Garbin, London (GB); Marek Adam Kowalski, Cambridge (GB); Matthew Alastair Johnson, Cambridge (GB); Tadas Baltrusaitis, Cambridge (GB); Martin De La Gorce, Cambridge (GB); Virginia Estellers Casas, Cambridge (GB); Sebastian Karol Dziadzio, Cambridge (GB); and Jamie Daniel Joseph Shotton, Cambridge (GB)
Assigned to Microsoft Technology Licensing, LLC, Redmond, WA (US)
Filed by Microsoft Technology Licensing, LLC, Redmond, WA (US)
Filed on May 23, 2022, as Appl. No. 17/751,429.
Application 17/751,429 is a continuation of application No. 16/916,009, filed on Jun. 29, 2020, granted, now 11,354,846.
Claims priority of provisional application 63/019,872, filed on May 4, 2020.
Prior Publication US 2022/0284655 A1, Sep. 8, 2022
This patent is subject to a terminal disclaimer.
Int. Cl. G06T 15/00 (2011.01); G06F 18/2135 (2023.01); G06F 18/22 (2023.01); G06N 3/088 (2023.01); G06V 10/82 (2022.01); G06V 40/16 (2022.01)
CPC G06N 3/088 (2013.01) [G06F 18/2135 (2023.01); G06F 18/22 (2023.01); G06T 15/005 (2013.01); G06V 10/82 (2022.01); G06V 40/171 (2022.01)] 20 Claims
OG exemplary drawing
 
1. An image processing apparatus comprising:
a memory storing a region of interest of a synthetic image depicting an object from a class of objects;
a trained neural image generator having been trained to map embeddings from a multi-dimensional latent space to photorealistic images of objects in the class; and
a processor arranged to:
compute a first embedding from the latent space, the first embedding being computed by selecting an initial point embedding in the latent space which generates an image similar to the region of interest and by iteratively combining the initial point embedding with other samples from the latent space to produce the first embedding;
compute a second embedding from the latent space, the second embedding corresponding to the synthetic image; and
blend the first embedding and the second embedding to form a blended embedding,
wherein the trained neural image generator further generates an output image from the blended embedding.