| CPC A61B 5/0077 (2013.01) [G06T 7/0012 (2013.01); G06T 7/70 (2017.01); G06V 10/764 (2022.01); G06V 10/82 (2022.01); G06V 40/161 (2022.01); G06V 40/171 (2022.01); G06V 40/174 (2022.01); G06V 40/176 (2022.01); G06V 40/20 (2022.01); G06T 2207/10016 (2013.01); G06T 2207/30201 (2013.01)] | 31 Claims |
|
1. A system for facial analysis comprising:
a camera configured to take a video of a subject's face, the video having a plurality of frames;
a data storage device configured to receive and store the video; and
a data processing system having a processor and program code which when executed:
(a) generates a pose data stream by extracting, from the plurality of frames, a plurality of poses of the subject's face, each pose of the plurality of poses including an angle and a position of the subject's face,
(b) generates a facial gestures state data stream by extracting, from the plurality of frames, a plurality of representations of the subject's facial gesture state, each representation of the plurality of representations including facial keypoints, the facial keypoints being a collection of points on the subject's face,
(c) concatenates the pose data stream and the facial gestures state data stream to align the data streams in time,
(d) extracts a plurality of facial syllables from the aligned data streams,
(e) compiles the plurality of facial syllables into a series of state sequences, and
(f) extracts a behavioral fingerprint for the subject based on the series of state sequences, the behavioral fingerprint providing a summary of the subject's state over a given period of time, wherein extracting the behavioral fingerprint includes calculating a histogram of state frequencies based on the series of state sequences, the histogram indicative of how often each of the plurality of facial syllables occurs in the series of state sequences.
|