CPC G16H 50/30 (2018.01) [G16H 10/60 (2018.01); G16H 50/20 (2018.01)] | 21 Claims |
1. A system for evaluating a user, the system comprising:
a microphone;
a camera positioned to capture an image of the user and configured to output video data;
a memory containing machine readable medium comprising machine executable code having stored thereon instructions for performing a method of evaluating the user; and
a control system coupled to the memory comprising one or more processors, the control system configured to execute the machine executable code to cause the control system to:
record, by the camera, a set of test video data during a time period;
record, by the microphone, a set of test audio data during the time period;
process the video data to assign a plurality of pixels to a face of the user;
analyze the plurality of pixels to determine whether the face of the user is within a frame captured by the camera;
in response to determining that the face of the user is within the frame captured by the camera, process the plurality of pixels to output video features associated with the user;
process the audio data to identify sounds representing a voice of the user and output audio features associated with the user;
process, using a machine learning model, the audio and video features, wherein the machine learning model was previously trained with a set of training data comprising audio and video data recorded from a plurality of individuals with labels indicating whether each of the plurality of individuals has one of a plurality of characteristics; and
output an indication of whether the user has at least one of the plurality of characteristics.
|