US 11,687,384 B2
Real-time synthetically generated video from still frames
Austin Walters, Savoy, IL (US); Vincent Pham, Champaign, IL (US); Kate Key, Effingham, IL (US); Kenneth Taylor, Champaign, IL (US); Anh Truong, Champaign, IL (US); Fardin Abdi Taghi Abad, Champaign, IL (US); and Jeremy Goodsitt, Champaign, IL (US)
Assigned to Capital One Services, LLC, McLean, VA (US)
Filed by CAPITAL ONE SERVICES, LLC, McLean, VA (US)
Filed on May 4, 2021, as Appl. No. 17/307,361.
Application 17/307,361 is a continuation of application No. 16/457,670, filed on Jun. 28, 2019, granted, now 11,032,585.
Application 16/457,670 is a continuation of application No. 16/173,374, filed on Oct. 29, 2018, granted, now 10,382,799, issued on Aug. 13, 2019.
Claims priority of provisional application 62/694,968, filed on Jul. 6, 2018.
Prior Publication US 2021/0255907 A1, Aug. 19, 2021
Int. Cl. G06T 7/246 (2017.01); G06F 9/54 (2006.01); G06N 20/00 (2019.01); G06F 17/16 (2006.01); G06N 3/04 (2023.01); G06F 11/36 (2006.01); G06N 3/088 (2023.01); G06F 21/62 (2013.01); G06N 5/04 (2023.01); G06F 17/15 (2006.01); G06T 7/194 (2017.01); G06T 7/254 (2017.01); G06F 16/2455 (2019.01); G06F 16/22 (2019.01); G06F 16/28 (2019.01); G06F 16/906 (2019.01); G06F 16/93 (2019.01); G06F 16/903 (2019.01); G06F 16/9038 (2019.01); G06F 16/9032 (2019.01); G06F 16/25 (2019.01); G06F 16/335 (2019.01); G06F 16/242 (2019.01); G06F 16/248 (2019.01); G06F 30/20 (2020.01); G06F 40/166 (2020.01); G06F 40/117 (2020.01); G06F 40/20 (2020.01); G06F 8/71 (2018.01); G06F 17/18 (2006.01); G06F 21/55 (2013.01); G06F 21/60 (2013.01); G06N 7/00 (2023.01); G06Q 10/04 (2023.01); G06T 11/00 (2006.01); H04L 9/40 (2022.01); H04L 67/306 (2022.01); H04L 67/00 (2022.01); H04N 21/234 (2011.01); H04N 21/81 (2011.01); G06N 5/00 (2023.01); G06N 5/02 (2023.01); G06V 30/196 (2022.01); G06F 18/22 (2023.01); G06F 18/23 (2023.01); G06F 18/24 (2023.01); G06F 18/40 (2023.01); G06F 18/213 (2023.01); G06F 18/214 (2023.01); G06F 18/21 (2023.01); G06F 18/20 (2023.01); G06F 18/2115 (2023.01); G06F 18/2411 (2023.01); G06F 18/2415 (2023.01); G06N 3/044 (2023.01); G06N 3/045 (2023.01); G06N 7/01 (2023.01); G06V 30/194 (2022.01); G06V 10/98 (2022.01); G06V 10/70 (2022.01); G06N 3/08 (2023.01)
CPC G06F 9/541 (2013.01) [G06F 8/71 (2013.01); G06F 9/54 (2013.01); G06F 9/547 (2013.01); G06F 11/3608 (2013.01); G06F 11/3628 (2013.01); G06F 11/3636 (2013.01); G06F 16/2237 (2019.01); G06F 16/2264 (2019.01); G06F 16/248 (2019.01); G06F 16/2423 (2019.01); G06F 16/24568 (2019.01); G06F 16/254 (2019.01); G06F 16/258 (2019.01); G06F 16/283 (2019.01); G06F 16/285 (2019.01); G06F 16/288 (2019.01); G06F 16/335 (2019.01); G06F 16/906 (2019.01); G06F 16/9038 (2019.01); G06F 16/90332 (2019.01); G06F 16/90335 (2019.01); G06F 16/93 (2019.01); G06F 17/15 (2013.01); G06F 17/16 (2013.01); G06F 17/18 (2013.01); G06F 18/213 (2023.01); G06F 18/214 (2023.01); G06F 18/217 (2023.01); G06F 18/2115 (2023.01); G06F 18/2148 (2023.01); G06F 18/2193 (2023.01); G06F 18/22 (2023.01); G06F 18/23 (2023.01); G06F 18/24 (2023.01); G06F 18/2411 (2023.01); G06F 18/2415 (2023.01); G06F 18/285 (2023.01); G06F 18/40 (2023.01); G06F 21/552 (2013.01); G06F 21/60 (2013.01); G06F 21/6245 (2013.01); G06F 21/6254 (2013.01); G06F 30/20 (2020.01); G06F 40/117 (2020.01); G06F 40/166 (2020.01); G06F 40/20 (2020.01); G06N 3/04 (2013.01); G06N 3/044 (2023.01); G06N 3/045 (2023.01); G06N 3/08 (2013.01); G06N 3/088 (2013.01); G06N 5/00 (2013.01); G06N 5/02 (2013.01); G06N 5/04 (2013.01); G06N 7/00 (2013.01); G06N 7/01 (2023.01); G06N 20/00 (2019.01); G06Q 10/04 (2013.01); G06T 7/194 (2017.01); G06T 7/246 (2017.01); G06T 7/248 (2017.01); G06T 7/254 (2017.01); G06T 11/001 (2013.01); G06V 10/768 (2022.01); G06V 10/993 (2022.01); G06V 30/194 (2022.01); G06V 30/1985 (2022.01); H04L 63/1416 (2013.01); H04L 63/1491 (2013.01); H04L 67/306 (2013.01); H04L 67/34 (2013.01); H04N 21/23412 (2013.01); H04N 21/8153 (2013.01); G06T 2207/10016 (2013.01); G06T 2207/20081 (2013.01); G06T 2207/20084 (2013.01)] 19 Claims
OG exemplary drawing
 
1. A system for generating a synthetic video, the system comprising:
one or more memory units for storing instructions; and
one or more processors configured to execute the instructions to perform operations comprising:
normalizing a sequence of training images from a video;
generating a sequence of training difference images by performing a method of image subtraction on the normalized sequence of training images;
using the sequence of training images from the video to train an encoder model to generate synthetic difference images;
using the sequence of training difference images derived from the video to train a decoder model to generate synthetic images; and
starting from a seed image or a seed difference image, generating a sequence of synthetic images by iteratively:
applying the encoder model to an output of the decoder model; and
applying the decoder model to an output of the encoder model.