US 12,142,078 B2
Emotion recognition and notification system
Abdullah Alghamdi, Najran (SA); Asadullah Shaikh, Najran (SA); Mohammad Tabrez Quasim, Bisha (SA); Mesfer Alrizq, Najran (SA); Surbhi Bhatia, Hofuf (SA); Mana Saleh Al Reshan, Najran (SA); and Shadab Alam, Jizan (SA)
Filed by Abdullah Alghamdi, Najran (SA); Asadullah Shaikh, Najran (SA); Mohammad Tabrez Quasim, Bisha (SA); Mesfer Alrizq, Najran (SA); Surbhi Bhatia, Hofuf (SA); Mana Saleh Al Reshan, Najran (SA); and Shadab Alam, Jizan (SA)
Filed on Jun. 28, 2022, as Appl. No. 17/809,428.
Prior Publication US 2022/0335752 A1, Oct. 20, 2022
Int. Cl. G06V 40/16 (2022.01); A61B 5/00 (2006.01); A61B 5/16 (2006.01); G06V 10/82 (2022.01)
CPC G06V 40/174 (2022.01) [A61B 5/165 (2013.01); A61B 5/74 (2013.01); G06V 10/82 (2022.01); G06V 40/165 (2022.01); G06V 40/171 (2022.01)] 8 Claims
OG exemplary drawing
 
1. An emotion recognition and notification system, the system comprises:
an input unit includes a camera for capturing a facial image of a user's face;
a pre-processing unit to remove noise and enhance the details of the captured facial image;
a face detection unit to detect a face from the captured facial image;
a features extraction unit to extract a set of image features from detected face from the captured facial image;
a central processing unit equipped with a two channel facial expression recognition network based on CNN (Convolutional Neural Network) and LSTM (Long and Short Term Memory Network) to recognize an emotion of a user upon comparing the set of image features from a pre-stored image features, wherein the central processing unit executes a process for acquiring the recognition value corresponding to an expression of the face on the basis of differences between relative positions of pre-stored image features extracted from the face using an image previously received from the input unit at a timing when the input unit inputs an image, and relative positions derived in advance as references for the pre-stored image features extracted from one or more of the faces; and
a control unit equipped with a communication module to alert a registered user about a recognized emotion.