US 12,033,435 B2
Vision-based motion capture system for rehabilitation training
Shih-Yao Lin, Palo Alto, CA (US); Tao Yang, Mountain View, CA (US); Chao Huang, Palo Alto, CA (US); Zhen Qian, Santa Clara, CA (US); and Wei Fan, New York, NY (US)
Assigned to TENCENT AMERICA LLC, Palo Alto, CA (US)
Filed by TENCENT AMERICA LLC, Palo Alto, CA (US)
Filed on May 4, 2021, as Appl. No. 17/307,533.
Prior Publication US 2022/0358309 A1, Nov. 10, 2022
Int. Cl. G06T 7/246 (2017.01); A63B 24/00 (2006.01); A63B 71/06 (2006.01); G06N 3/04 (2023.01); G06V 20/40 (2022.01); G06V 40/10 (2022.01); G06V 40/20 (2022.01)
CPC G06V 40/23 (2022.01) [A63B 24/0062 (2013.01); A63B 71/0622 (2013.01); A63B 71/0669 (2013.01); G06N 3/04 (2013.01); G06T 7/251 (2017.01); G06V 20/46 (2022.01); G06V 40/103 (2022.01); A63B 2071/0636 (2013.01)] 16 Claims
OG exemplary drawing
 
1. A method for video-based motion capture performed by at least one processor, the method comprising:
obtaining, from a monocular camera of a smartphone, video data including at least one body part of a person, the video data being spatially only two-dimensional;
selecting keypoints of the at least one body part based on a predetermined rehabilitation category;
extracting a motion feature of the at least one body part from the video data;
scoring the motion feature based on the predetermined rehabilitation category; and
generating a display illustrating the motion feature and said scoring of the motion feature,
wherein said selecting the keypoints of the at least one body part based on the predetermined rehabilitation category comprises predicting the predetermined rehabilitation category by a deep neural network (DNN) configured to predict N possible regions representing possible locations of the keypoints with respect to the at least one body part,
wherein N is an integer,
wherein said predicting the predetermined rehabilitation category by the DNN comprises comparing the video data including at least one body part of the person to a plurality of anchor poses, and
wherein the plurality of anchor poses each comprise poses of ones of predetermined rehabilitation categories, including the predetermined rehabilitation category.