US 11,961,300 B2
Dynamic media content categorization method
Gabriel Autès, Lausanne (CH); Sami Arpa, Lausanne (CH); and Sabine Süsstrunk, Lausanne (CH)
Assigned to ECOLE POLYTECHNIQUE FEDERALE DE LAUSANNE (EPFL), Lausanne (CH)
Appl. No. 17/607,393
Filed by ECOLE POLYTECHNIQUE FEDERALE DE LAUSANNE (EPFL), Lausanne (CH)
PCT Filed Apr. 29, 2019, PCT No. PCT/IB2019/053484
§ 371(c)(1), (2) Date Oct. 28, 2021,
PCT Pub. No. WO2020/222027, PCT Pub. Date Nov. 5, 2020.
Prior Publication US 2022/0207864 A1, Jun. 30, 2022
Int. Cl. G06V 10/00 (2022.01); G06V 10/44 (2022.01); G06V 10/764 (2022.01); G06V 10/82 (2022.01); G06V 20/40 (2022.01)
CPC G06V 20/47 (2022.01) [G06V 10/454 (2022.01); G06V 10/765 (2022.01); G06V 10/82 (2022.01); G06V 20/40 (2022.01)] 24 Claims
OG exemplary drawing
 
1. A computer implemented method of classifying a media item into one or more user profile suitability classes in a classification system comprising a set of pre-trained artificial neural networks, the method comprising:
receiving a media file;
extracting a first data stream and a second data stream from the media file, the first media stream comprising first media content of a first media content form, the second data stream comprising second media content of a second, different media content form, the first and second data streams being comprised in the media item, the first content form being one of the following content forms: audio, image and text, while the second media content form being any one of the remaining two content forms;
generating a first sequence of first feature vectors describing the first media content, the first sequence of first feature vectors being generated by a first set of convolution neural networks;
generating a second sequence of second feature vectors describing the second media content, the second sequence of second feature vectors being generated by a second set of convolution neural networks or by a set of word embedding matrices;
generating at least a first single feature vector representing the first sequence of first feature vectors and the second sequence of second feature vectors, or generating at least a first single feature vector representing at least the first sequence of first feature vectors, and generating a second single feature vector representing at least the second sequence of second feature vectors;
generating a probability vector at least from the first single feature vector in the case where the first single feature vector represents the first sequence of first feature vectors and the second sequence of second feature vectors, or at least from the first and second single feature vectors by first concatenating the first and second single feature vectors in the case where the first single feature vector represents at least the first sequence of first feature vectors, and the second single feature vector represents at least the second sequence of second feature vectors, the probability vector being generated by a feedforward artificial neural network; and
assigning a user profile suitability class to the media item based on the probability vector.