US 12,424,027 B2
Joint motion estimation based method for estimating continuous human postures
Canjun Yang, Hangzhou (CN); Xin Wu, Hangzhou (CN); Zhangpeng Tu, Hangzhou (CN); Weitao Wu, Hangzhou (CN); and Yuanchao Zhu, Hangzhou (CN)
Assigned to ZHEJIANG UNIVERSITY, Hangzhou (CN)
Filed by ZHEJIANG UNIVERSITY, Hangzhou (CN)
Filed on Apr. 13, 2023, as Appl. No. 18/134,547.
Claims priority of application No. 202210418358.0 (CN), filed on Apr. 20, 2022.
Prior Publication US 2023/0343140 A1, Oct. 26, 2023
Int. Cl. G06V 40/20 (2022.01); G06T 7/20 (2017.01); G06V 10/75 (2022.01)
CPC G06V 40/20 (2022.01) [G06T 7/20 (2013.01); G06V 10/751 (2022.01); G06T 2207/10016 (2013.01)] 10 Claims
OG exemplary drawing
 
1. A key joint motion estimation based method for estimating continuous human postures, wherein a system for estimation comprises two estimators:
an estimator 1 that is a pretrained deep neural network posture estimator, and
an estimator 2 that is a video encoding standard H.264 based motion estimator; and
the key joint motion estimation based method for estimating continuous human postures comprises three stages:
a first stage: after the system starts to operate, taking a first imported video frame as a key frame I0, recognizing a human posture in the video frame by using the estimator 1, so as to obtain initial human key joint coordinates; during operation of the estimator 1, enabling the system to continue to acquire a plurality of video frames, and temporarily storing the video frames in a computer memory queue; and after the operation of the estimator 1, transmitting obtained key joint coordinate data and all the temporarily stored video frames into the estimator 2 for operation in order, to obtain a human key joint estimation result of each video frame;
a second stage: operating a next video frame I1 subsequently acquired by the system after the operation of the estimator 1 in the first stage by using both the estimator 1 and the estimator 2, and performing real-time operation on a subsequently acquired new video frame by using the estimator 2 in a process that the operation of the estimator 1 is not completed yet; and
a third stage: after the operation of the estimator 1 in the second stage, comparing estimation results of key joints in the video frame I1 by the estimator 1 and the estimator 2, and if a sum of all key joint coordinate errors of the two estimators is less than a set threshold ε, repeating the steps in the second stage for subsequent video frames; and if an error between the two estimators is greater than the threshold, updating, based on the estimation results of the key joints in the video frame I1 by the estimator 1, the estimation results of the key joints in the video frame obtained by the estimator 2 during the operation of the estimator 1 by using the estimator 2 again, and after these results are updated, repeating the steps in the second stage for the subsequent video frames.