| CPC A61B 5/7275 (2013.01) [A61B 5/112 (2013.01); A61B 5/1128 (2013.01); A61B 5/7267 (2013.01); G06N 3/08 (2013.01); G06T 7/251 (2017.01); G06V 10/75 (2022.01); G06V 40/25 (2022.01); G16H 50/20 (2018.01); G16H 50/30 (2018.01); G16H 50/50 (2018.01); G16H 50/70 (2018.01); G06T 2207/10016 (2013.01); G06T 2207/20081 (2013.01); G06T 2207/20084 (2013.01)] | 17 Claims |
|
1. A computer-implemented method for human gait analysis based on a video stream obtained from a monocular camera device, the video stream including a plurality of frames reflecting a walk of a human individual, the method comprising:
inferring, from the video stream, three-dimensional gait information, wherein the three-dimensional gait information includes estimates of joint locations of the individual including at least foot locations of the individual on each frame, the estimates being derived by matching, for each frame, two-dimensional joint coordinates of the respective frame with respective three-dimensional model information of a body of the individual, wherein the inferring further comprises:
deriving heat-maps and location-maps for joint location estimation by using a convolutional neural network trained on three-dimensional human pose datasets, wherein a particular heat-map describes, for a corresponding frame, probability values that a particular joint is associated with respective pixels of the corresponding frame, and wherein a particular set of location-maps includes a plurality of location-maps, with each of the location-maps describing a distance of a particular joint to a root location for the corresponding frame in a respective spatial dimension;
receiving a selection of at least a frame sequence of the video stream during which the joints of the human individual move over time;
estimating a skeleton model of the individual by determining, for each frame of the selected sequence, a loss for each joint of a default skeleton model in each spatial coordinate, and adjusting the default skeleton model to compensate the determined losses to provide an adjusted skeleton model; and
performing kinematic skeleton fitting per video frame using the adjusted skeleton model to determine a plurality of joint locations including at least the foot locations of feet of the individual on each frame; and
determining one or more gait parameters of the individual based on the foot locations of the individual in local extrema frames showing local extrema of a distance between one foot location of the individual and a corresponding reference joint location, wherein at least one of the determined gait parameters is associated with a score characterizing a risk of fall for the individual.
|