US 12,334,221 B2
System, method and computer readable medium for video-based facial weakness analysis for detecting neurological deficits
Gustavo Rohde, Dunn Loring, VA (US); Andrew M. Southerland, Charlottesville, VA (US); Yan Zhuang, Charlottesville, VA (US); Mark McDonald, Charlottesville, VA (US); Omar Uribe, Sterling, VA (US); Chad M. Aldridge, Charlottesville, VA (US); and Mohamed Abul Hassan, Davis, CA (US)
Assigned to University of Virginia Patent Foundation, Charlottesville, VA (US)
Filed by University of Virginia Patent Foundation, Charlottesville, VA (US)
Filed on Feb. 4, 2022, as Appl. No. 17/665,334.
Claims priority of provisional application 63/146,220, filed on Feb. 5, 2021.
Prior Publication US 2022/0319707 A1, Oct. 6, 2022
Int. Cl. G06V 10/00 (2022.01); G16H 30/20 (2018.01); G16H 50/20 (2018.01)
CPC G16H 50/20 (2018.01) [G16H 30/20 (2018.01)] 39 Claims
OG exemplary drawing
 
1. A system for analyzing facial weakness for predicting presence of one or more neurological deficits, comprising:
a camera;
one or more memory devices configured to store instructions; and
one or more processors are configured to execute the instructions to:
extract the facial landmarks from a video feed received from said camera;
perform landmarks and intensity normalization that removes translation, rotation, and scaling variations from said extracted facial landmarks;
detect facial movement by employing an optical flow method to measure the face movement intensity and locate a target video segment where a smile configuration is evident to obtain desired a video segment;
extract shape features and appearance-based features from target frames inside said desired video segment, wherein said shape features and appearance-based features are high-dimensional;
project said high-dimensional shape features and appearance-based features onto a low-dimensional subspace;
classify the input video using said low-dimensional representation of shape features and appearance-based features via a recurrent neural network; and
in response to said classification, predict a presence of one or more neurological deficits and transmit said predication to one or more secondary source devices.