CPC G06N 3/084 (2013.01) [G06F 16/483 (2019.01); G06N 3/04 (2013.01); G06Q 10/0639 (2013.01); G06Q 30/0271 (2013.01); G06Q 50/01 (2013.01); G06F 40/289 (2020.01)] | 25 Claims |
1. A system for determining quality level of labeling of a plurality of multimedia data items, wherein the labels relate to reputation polarity, wherein reputation polarity comprises determining an impact of a statement on a reputation of a brand, perception of a brand, or a combination thereof; wherein said brand comprises a person, company, organization or other identifiable entity; the system comprising a plurality of worker computational devices for labeling the items, each worker computational device comprising a processor for executing instructions for labeling the items and a memory for storing the instructions; an analysis computational device for analyzing the labels to determine accuracy by calculating cross polarity accuracy scores and neutral bias scores for a plurality of labels for each item, and for providing feedback through said worker computational devices to increase accuracy of labeling; wherein said cross polarity accuracy score is determined at least partially according to a number of correct labels determined at each worker computational device; wherein said neutral bias score is at least partially determined according to a number of correct reputation polarity labels determined at each worker computational device; wherein said analysis computational device comprises an analysis processor for executing instructions for analyzing the labels and for providing feedback, and an analysis memory for storing the instructions; and a computer network for communication between said computational devices;
the system further comprising a model computational device for being trained on labeled multimedia data items to train a model to automatically determine reputation polarity by classifying an unlabeled multimedia data item, wherein said model computational device comprises a model processor for executing a plurality of instructions for automatically training a model to automatically determine reputation polarity of an unlabeled multimedia data item, and a model memory for storing said instructions; wherein said model to automatically determine reputation polarity receives said plurality of labeled multimedia data items and classifies said multimedia data items according to reputation polarity; wherein an error in correctly classifying said multimedia data items by said model is calculated and is used to adjust said model; wherein said training is repeated until a sufficiently low error is reached.
|