CPC A61B 5/165 (2013.01) [A61B 5/1128 (2013.01); A61B 5/117 (2013.01); A61B 5/7264 (2013.01); A61B 5/7275 (2013.01); G16H 10/60 (2018.01); G16H 20/70 (2018.01); G16H 50/50 (2018.01)] | 20 Claims |
1. An emotion analysis and recognition system, comprising:
a query system configured to interact with a user; and
an automated recognition of bodily expression of emotion (ARBEE) system communicatively coupled to the query system, the ARBEE system having a processor and a memory storing program instructions, the program instructions executable by the processor to:
receive a plurality of body movement models, each body movement model generated based on a crowdsourced body language dataset;
calculate at least one evaluation metric for each body movement model, the at least one evaluation metric to evaluate a classification performance for each of a plurality of categorical emotions and/or to evaluate regression performance for each of a plurality of dimensional emotions, the categorical and dimensional emotions being determined based on a combination of movements for body parts of subjects, facial expressions of the subjects, associated audio, associated context, and a pixel analysis of cropped images of a human body;
select a highest ranked body movement model based on the at least one evaluation metric calculated for each body movement model;
combine the highest ranked body movement model with at least one other body movement model of the plurality of body movement models;
calculate at least one evaluation metric for each combination of body movement models;
determine a highest ranked combination of body movement models;
based on the determined highest ranked combination of body movement models, predict a bodily expression of emotion;
receive, from the query system, a video;
apply the received video as input to the highest ranked combination of body movement models; and
transmit an emotion result to the query system, the emotion result providing information usable by the query system for determining an emotion of a target,
wherein the query system interacts with the user based on the emotion result received from the ARBEE system.
|