US 11,720,994 B2
High-resolution portrait stylization frameworks using a hierarchical variational encoder
Linjie Luo, Los Angeles, CA (US); Guoxian Song, Singapore (SG); Jing Liu, Los Angeles, CA (US); and Wanchun Ma, Los Angeles, CA (US)
Assigned to Lemon Inc., Grand Cayman (KY)
Filed by Lemon Inc., Grand Cayman (KY)
Filed on May 14, 2021, as Appl. No. 17/321,384.
Prior Publication US 2022/0375024 A1, Nov. 24, 2022
Int. Cl. G06T 3/00 (2006.01); G06T 11/00 (2006.01); G06T 5/00 (2006.01); G06N 3/08 (2023.01); G06F 18/214 (2023.01); G06N 3/045 (2023.01)
CPC G06T 3/0012 (2013.01) [G06F 18/214 (2023.01); G06N 3/045 (2023.01); G06N 3/08 (2013.01); G06T 3/0006 (2013.01); G06T 5/00 (2013.01); G06T 11/00 (2013.01); G06T 2207/20016 (2013.01); G06T 2207/20081 (2013.01); G06T 2207/20084 (2013.01); G06T 2207/30201 (2013.01)] 17 Claims
OG exemplary drawing
 
1. A method for generating a stylized image, the method comprising:
receiving an input image;
encoding the input image using a variational autoencoder to obtain a latent vector by:
passing the received input image through a headless pyramid network to produce multiple levels of features maps at different sizes;
encoding, for each of the levels of features maps at different sizes, each level's respective feature map at the different size with a separate encoder of a plurality of encoders to produce a code, and
combining the encoded code of each level's respective feature map to obtain the latent vector;
providing the latent vector to a pre-trained generative adversarial network (GAN) model;
generating, by the pre-trained GAN model, a stylized image from the pre-trained GAN model, the generated stylized image being a cartoon style image of the input image; and
providing the stylized image as an output,
wherein the pre-trained GAN model includes a multi-path structure corresponding to two or more different attributes.