US 12,465,280 B1
System and methods for gait analysis and longitudinal health and aging assessments including musculoskeletal disorders using video-trained spatio-temporal graph neural networks
Bahram Jalali, Los Angeles, CA (US); and William R Ryan, Los Angeles, CA (US)
Assigned to iMagine Design LLC, Tuscon, AZ (US)
Filed by iMagine Design LLC, Tuscon, AZ (US)
Filed on Jun. 6, 2025, as Appl. No. 19/230,571.
Int. Cl. A61B 5/00 (2006.01); A61B 5/11 (2006.01); A61B 5/256 (2021.01); A61B 5/389 (2021.01)
CPC A61B 5/4528 (2013.01) [A61B 5/112 (2013.01); A61B 5/1128 (2013.01); A61B 5/256 (2021.01); A61B 5/389 (2021.01); A61B 5/7267 (2013.01); A61B 5/7275 (2013.01)] 19 Claims
OG exemplary drawing
 
1. A method for pose and gait classification and motion prediction using spatio-temporal relationships between body joints, the method comprising:
capturing a sequence of images or video frames of a subject;
applying a neural network-based pose estimation algorithm to the sequence of images or video frames to detect landmark positions of one or more anatomical joints;
constructing a spatio-temporal graph from the detected landmark positions of the one or more anatomical joints, wherein:
nodes of the spatio-temporal graph correspond to the one or more anatomical joints and the landmark positions,
spatial edges of the spatio-temporal graph represent anatomical connections between the one or more anatomical joints within a single image or frame, and
temporal edges connect the one or more anatomical joints across successive images or frames of the sequence of images or video frames; and
inputting the constructed spatio-temporal graph into a spatio-temporal graph convolutional network (ST-GCN) to classify pose and gait patterns and predict motion or stability states for the constructed spatio-temporal graph; and
optionally enhancing the ST-GCN with a spatio-temporal attention module to emphasize the one or more relevant anatomical joints and a temporal attention module to emphasize critical timeframes.