US 11,836,905 B2
Image reenactment with illumination disentanglement
Mohamed N. Moustafa, Metuchen, NJ (US); Ahmed A. Ewais, New Cairo (EG); and Amr A. Ali, Cairo (EG)
Assigned to Spree3D Corporation, Incline Village, NV (US)
Filed by Spree3D Corporation, Incline Village, NV (US)
Filed on Jun. 3, 2021, as Appl. No. 17/338,196.
Prior Publication US 2022/0392034 A1, Dec. 8, 2022
Int. Cl. G06T 5/50 (2006.01); G06T 7/70 (2017.01); G06N 3/088 (2023.01); G06N 3/045 (2023.01)
CPC G06T 5/50 (2013.01) [G06N 3/045 (2023.01); G06N 3/088 (2013.01); G06T 7/70 (2017.01); G06T 2207/10016 (2013.01); G06T 2207/20081 (2013.01); G06T 2207/20084 (2013.01); G06T 2207/20221 (2013.01); G06T 2207/30201 (2013.01)] 22 Claims
OG exemplary drawing
 
1. Apparatus for inserting identity information from a source image into a destination image while mimicking illumination of the destination image, said apparatus comprising:
an identity encoder configured to encode just identity information of the source image and to produce an identity vector, where the identity encoder does not encode any pose information or illumination information of the source image;
a pose encoder configured to encode pose information from the totality of the destination image and to produce a pose vector;
separate from and independent of the pose encoder, an illumination encoder configured to encode illumination information from the totality of the destination image and to produce an illumination vector; and
a neural network generator coupled to the identity encoder, pose encoder, and illumination encoder, and having three inputs: the identity vector, the pose vector, and the illumination vector; wherein
the generator is configured to generate, in response to the three inputs, a composite image comprising identity information of the source image inserted into the destination image, wherein the composite image has substantially the same illumination as the destination image.