US 12,419,556 B2
Method and system for detecting short-term stress and generating alerts inside the indoor environment
Chun Kit Hung, Hong Kong (HK); Fu Tuen Leung, Hong Kong (HK); Ka Fai Suk, Hong Kong (HK); and Kar-Wing Edward Lor, Hong Kong (HK)
Assigned to Hong Kong Applied Science and Technology Research Institute Company Limited, Hong Kong (HK)
Filed by Hong Kong Applied Science and Technology Research Institute Company Limited, Hong Kong (HK)
Filed on Aug. 31, 2022, as Appl. No. 17/899,632.
Prior Publication US 2024/0065596 A1, Feb. 29, 2024
Int. Cl. A61B 5/16 (2006.01); A61B 5/024 (2006.01); A61B 5/0533 (2021.01); A61B 5/11 (2006.01); G06V 10/80 (2022.01); G06V 20/40 (2022.01); G06V 40/20 (2022.01)
CPC A61B 5/165 (2013.01) [A61B 5/02438 (2013.01); A61B 5/0533 (2013.01); A61B 5/1118 (2013.01); G06V 10/809 (2022.01); G06V 20/46 (2022.01); G06V 40/23 (2022.01)] 20 Claims
OG exemplary drawing
 
1. A method for detecting short-term stress of a subject person and providing alerts in an indoor environment, comprising:
receiving heart rate, galvanic skin response data, and accelerometer data from a wearable device;
capturing image frames using a camera;
performing, by a processor, a bio-signals analysis in accordance with the heart rate and the galvanic skin response data, comprising:
generating heart rate statistical data of the heart rate;
generating galvanic skin response statistical data of the galvanic skin response data;
mapping the heart rate statistical data into a heart rate index;
mapping the galvanic skin response statistical data into a skin conductance index; and
data fusing the heart rate index and skin conductance index to obtain a bio-signals index, wherein the bio-signals index indicates changes in the heart rate and the galvanic skin response data;
performing, by the processor, a skeletal motion analysis in accordance with the image frames to generate one or more motion states and activeness, comprising:
determining a set of joint coordinates of joint pose of the subject person from each of the image frames;
identifying a set of landmarks from the set of joint coordinates;
if the set of landmarks cannot be identified, performing a reconstruction of landmarks using an approximation of motion of the subject person from the image frames with the accelerometer data to reconstruct the set of landmarks;
generating a skeletal motion sequence from the set of landmarks; and
predicting the motion states and activeness of the subject person, comprising applying a graphic convolution network (GCN) in classifying the skeletal motion sequence into different motion states and activeness based on a trained model; and
performing, by the processor, data fusion to determine the short-term stress in accordance with the bio-signals index and the motion states and activeness.