US 12,141,199 B2
Feature-based video annotation
Balakrishnan Varadarajan, Sunnyvale, CA (US); George Dan Toderici, Mountain View, CA (US); Apostol Natsev, Sunnyvale, CA (US); Nitin Khandelwal, Milpitas, CA (US); Sudheendra Vijayanarasimhan, Mountain View, CA (US); Weilong Yang, Fremont, CA (US); and Sanketh Shetty, Sunnyvale, CA (US)
Assigned to Google LLC, Mountain View, CA (US)
Filed by Google LLC, Mountain View, CA (US)
Filed on Dec. 13, 2021, as Appl. No. 17/548,859.
Application 17/548,859 is a continuation of application No. 16/687,118, filed on Nov. 18, 2019, granted, now 11,200,423.
Application 16/687,118 is a continuation of application No. 15/722,756, filed on Oct. 2, 2017, granted, now 10,482,328, issued on Nov. 19, 2019.
Application 15/722,756 is a continuation of application No. 14/823,946, filed on Aug. 11, 2015, granted, now 9,779,304, issued on Oct. 3, 2017.
Prior Publication US 2022/0207873 A1, Jun. 30, 2022
This patent is subject to a terminal disclaimer.
Int. Cl. G06K 9/62 (2022.01); G06F 16/78 (2019.01); G06F 16/783 (2019.01); G06F 18/214 (2023.01); G06F 18/22 (2023.01); G06F 18/2413 (2023.01); G06V 20/40 (2022.01); G06V 20/70 (2022.01); H04N 5/265 (2006.01)
CPC G06F 16/783 (2019.01) [G06F 16/7867 (2019.01); G06F 18/214 (2023.01); G06F 18/22 (2023.01); G06F 18/2413 (2023.01); G06V 20/41 (2022.01); G06V 20/70 (2022.01); H04N 5/265 (2013.01)] 18 Claims
OG exemplary drawing
 
1. A computer-implemented method for annotating videos, the method comprising:
identifying, using a hardware processor, a plurality of entities identifying characteristics of a plurality of media content items;
selecting, using the hardware processor, first features correlated with each of the plurality of entities;
determining, using the hardware processor, a plurality of relationships each between a given first feature from the selected first features and a probability of a corresponding entity from the plurality of entities existing in a given media content item from the plurality of media content items;
receiving, using the hardware processor, a particular media content item and second features associated with the particular media content item;
determining, using the hardware processor and based on processing the second features, one or more entities from the plurality of entities that are included in the particular media content item;
determining, using the hardware processor, one or more corresponding probabilities of the one or more entities being central to at least one frame of a plurality of frames of the particular media content item based on one or more relationships from the plurality of relationships that correspond to the one or more entities;
labelling, using the hardware processor, the at least one frame of the particular media content item with a particular entity of the one or more entities and with the probability of the particular entity being central to the at least one frame.