US 12,268,504 B2
Cognitive disorder human-computer interaction method and system based on emotion monitoring
Shiyi Li, Beijing (CN); Yan Xu, Beijing (CN); Zhujiang Ma, Beijing (CN); and Xiaoyi Wang, Beijing (CN)
Assigned to BEIJING WISPIRIT TECHNOLOGY CO., LTD, Beijing (CN)
Filed by BEIJING WISPIRIT TECHNOLOGY CO., LTD, Beijing (CN)
Filed on Dec. 30, 2022, as Appl. No. 18/148,825.
Application 18/148,825 is a continuation of application No. PCT/CN2022/120987, filed on Sep. 23, 2022.
Prior Publication US 2023/0148925 A1, May 18, 2023
Int. Cl. A61B 5/16 (2006.01); A61B 5/00 (2006.01); G06F 3/01 (2006.01); G06V 40/16 (2022.01)
CPC A61B 5/165 (2013.01) [A61B 5/0077 (2013.01); G06F 3/012 (2013.01); G06F 2203/011 (2013.01); G06V 40/174 (2022.01)] 8 Claims
OG exemplary drawing
 
1. A cognitive disorder human-computer interaction method based on emotion monitoring, comprising the following steps:
obtaining a cognitive ability level of a user;
generating a human-computer interaction scheme according to the cognitive ability level;
generating an Nth human-computer interaction task according to the human-computer interaction scheme, N being a positive integer;
carrying out emotion monitoring on the performance of the user in the Nth human-computer interaction task, and returning to the previous step of generating an Nth human-computer interaction task according to the human-computer interaction scheme, whereby N=N+1 until all tasks in the human-computer interaction scheme are completed; and
carrying out, in the process of human-computer interaction, a relaxed human-computer interaction between two adjacent human-computer interaction tasks, and selecting a task type of the relaxed human-computer interaction according to an emotion comprehensive index of the user in the previous human-computer interaction task, wherein when the emotion comprehensive index is positive, a human-computer interaction task of a high strength challenge class is selected; when the emotion comprehensive index is negative, a human-computer interaction task of a high relaxation and stress relief class is selected; and when the emotion comprehensive index is neutral, a human-computer interaction task is randomly selected;
wherein the emotion comprehensive index being obtained by the following steps:
acquiring expression information of the user in each collection time period;
acquiring expression features in the expression information, and comparing the expression features with an Asian face database so as to obtain proportions of a positive emotion, a negative emotion and a neutral emotion in the expression information, the emotion with the maximum proportion being a current emotion of the user;
determining proportions of positive emotions, negative emotions and neutral emotions in all the current emotions of the user in the whole process of human-computer interaction;
acquiring a current emotion state of the user based on the proportions of the positive emotions, the negative emotions and the neutral emotions in all the current emotions of the user;
acquiring an emotion change state of the user based on an emotion proportion and a time change in each collection time period;
acquiring an emotion fluctuation state of the user based on the emotion proportion and the time change in each collection time period; and
obtaining the emotion comprehensive index of the user according to at least two of the current emotion state, the emotion change state and the emotion fluctuation state.