US 11,854,579 B2
Video reenactment taking into account temporal information
Mohamed N. Moustafa, Metuchen, NJ (US); Ahmed A. Ewais, New Cairo (EG); and Amr A. Ali, Cairo (EG)
Assigned to Spree3D Corporation, Incline Village, NV (US)
Filed by Spree3D Corporation, Incline Village, NV (US)
Filed on Jul. 12, 2021, as Appl. No. 17/373,605.
Application 17/373,605 is a continuation in part of application No. 17/338,196, filed on Jun. 3, 2021.
Prior Publication US 2022/0392490 A1, Dec. 8, 2022
Int. Cl. G11B 27/02 (2006.01); G06N 3/08 (2023.01); G06T 9/00 (2006.01); G06N 3/045 (2023.01)
CPC G11B 27/02 (2013.01) [G06N 3/045 (2023.01); G06N 3/08 (2013.01); G06T 9/002 (2013.01)] 21 Claims
OG exemplary drawing
 
1. An apparatus for inserting identity information from a source image of a first subject into a destination video of a second subject different than the first subject while mimicking motion in the destination video, said apparatus comprising:
an identity encoder configured to encode identity information from the source image and to produce an identity vector;
a driver encoder comprising a pose encoder configured to encode pose information from the destination video and to produce a pose vector, and a separate and independent motion encoder configured to encode motion information from the destination video and to produce a motion vector; and
a neural network generator having three inputs: the identity vector, the pose vector, and the motion vector; wherein
the neural network generator is configured to generate, in response to the three inputs, a composite video comprising identity information from the source video inserted into the destination video, where the composite video has substantially the same temporal information as in the destination video.