US 12,236,510 B2
Enhanced system for generation of facial models and animation
Hau Nghiep Phan, Montreal (CA)
Assigned to Electronic Arts Inc., Redwood City, CA (US)
Filed by Electronic Arts Inc., Redwood City, CA (US)
Filed on Jun. 10, 2021, as Appl. No. 17/344,471.
Prior Publication US 2022/0398795 A1, Dec. 15, 2022
Int. Cl. G06T 13/40 (2011.01); G06N 3/08 (2023.01); G06T 17/20 (2006.01); G06V 40/16 (2022.01)
CPC G06T 13/40 (2013.01) [G06N 3/08 (2013.01); G06T 17/20 (2013.01); G06V 40/168 (2022.01); G06V 40/172 (2022.01); G06V 40/174 (2022.01)] 17 Claims
OG exemplary drawing
 
1. A computer-implemented method comprising:
accessing an autoencoder trained based on a plurality of expressions of one or more real-world persons and identity information associated with the real-world persons, each expression being defined based on location information associated with a plurality of facial features, and the identity information, wherein the autoencoder was trained to reconstruct, via a latent variable space, expressions based on conditional information;
obtaining one or more two-dimensional (2D) images depicting a first set of one or more expressions of a first real-world person and first identity information of the first real-world person;
obtaining at least one 2D image depicting a second real-world person;
obtaining second identity information of the second real-world person, wherein the first identity information and the second identity information each comprise an identity vector that is representative of an invariant identity of the respective first or second real-world person; and
generating at least one 2D image depicting the second real-world person having an expression of the one or more expressions of the first person, wherein the generating comprises:
encoding, by an encoder, data associated with the first set of one or more expressions into the latent variable space;
decoding, by a decoder, latent variable space of the first set of one or more expressions based on the first identity information and the second identity information; and
outputting, by the decoder, the at least one 2D image depicting the second real-world person having an expression of the one or more expressions of the first person; and
accessing a machine learning model trained to generate a three-dimensional facial mesh based at least in part on the second identity information of the second real-world person and the at least one 2D image; and
generating, by the machine learning model, a first three-dimensional mesh of a face of the second real-world person based at least in part on the second identity information and the at least one 2D image.