US 12,217,494 B2
Keyframe extractor
Andreas Schmidt, San Pablo, CA (US)
Assigned to Gracenote, Inc., New York, NY (US)
Filed by Gracenote, Inc., Emeryville, CA (US)
Filed on Apr. 6, 2021, as Appl. No. 17/224,034.
Claims priority of provisional application 63/008,477, filed on Apr. 10, 2020.
Prior Publication US 2021/0319230 A1, Oct. 14, 2021
Int. Cl. G06V 10/82 (2022.01); G06F 18/24 (2023.01); G06F 18/2413 (2023.01); G06N 3/049 (2023.01); G06N 3/08 (2023.01); G06V 10/80 (2022.01); G06V 20/40 (2022.01); H04N 21/234 (2011.01); H04N 21/81 (2011.01)
CPC G06V 10/82 (2022.01) [G06F 18/24 (2023.01); G06F 18/2413 (2023.01); G06N 3/049 (2013.01); G06N 3/08 (2013.01); G06V 10/806 (2022.01); G06V 20/41 (2022.01); G06V 20/46 (2022.01); H04N 21/23424 (2013.01); H04N 21/812 (2013.01)] 17 Claims
OG exemplary drawing
 
1. A method comprising:
determining, by a computing system, a first blur score for a first frame of a video based on a discrete cosine transform (DCT) of pixel intensity values of the first frame by:
converting the first frame to grayscale;
downscaling the grayscale first frame;
calculating a first DCT of the grayscale, downscaled first frame to determine a first matrix of DCT coefficients;
transposing the first matrix of DCT coefficients;
calculating a second DCT of the transposed first matrix of DCT coefficients to determine a second matrix of DCT coefficients; and
calculating the first blur score for the first frame by subtracting a sum of the absolute values of the upper-left quarter of the second matrix of DCT coefficients from a sum of the absolute values of the coefficients of the second matrix of DCT coefficients;
determining, by the computing system, a second blur score for a second frame of the video based on a DCT of pixel intensity values of the second frame by:
converting the second frame to grayscale;
downscaling the grayscale second frame;
calculating a third DCT of the grayscale, downscaled second frame to determine a third matrix of DCT coefficients;
transposing the third matrix of DCT coefficients;
calculating a fourth DCT of the transposed third matrix of DCT coefficients to determine a fourth matrix of DCT coefficients; and
calculating the second blur score for the second frame by subtracting a sum of the absolute values of the upper-left quarter of the fourth matrix of DCT coefficients from a sum of the absolute values of the coefficients of the fourth matrix of DCT coefficients;
determining, by the computing system, a blur delta that quantifies a difference between a level of blurriness of the first frame of the video represented by the first blur score and a level of blurriness of the second frame of the video represented by the second blur score, wherein the second frame is subsequent to and adjacent to the first frame;
determining, by the computing system, a contrast delta that quantifies a difference between a contrast of the first frame and a contrast of the second frame;
determining, by the computing system, a fingerprint distance between a first image fingerprint of the first frame and a second image fingerprint of the second frame;
determining, by the computing system, a keyframe score using the blur delta, the contrast delta, and the fingerprint distance;
based on the keyframe score, determining, by the computing system, that the second frame is a keyframe; and
outputting, by the computing system, data indicating that the second frame is a keyframe.