US 12,073,600 B2
Method and apparatus for detecting keypoints of human body, electronic device and storage medium
Wentao Liu, Shenzhen (CN); Yujing Guo, Shenzhen (CN); Yongwang Wang, Shenzhen (CN); Chen Qian, Shenzhen (CN); and Jiahua Li, Shenzhen (CN)
Assigned to SHENZHEN SENSETIME TECHNOLOGY CO., LTD., Guangdong (CN)
Filed by SHENZHEN SENSETIME TECHNOLOGY CO., LTD., Shenzhen (CN)
Filed on Oct. 22, 2021, as Appl. No. 17/507,850.
Application 17/507,850 is a continuation of application No. PCT/CN2020/080231, filed on Mar. 19, 2020.
Claims priority of application No. 201910635763.6 (CN), filed on Jul. 15, 2019.
Prior Publication US 2022/0044056 A1, Feb. 10, 2022
Int. Cl. G06V 10/46 (2022.01); G06F 18/25 (2023.01); G06N 3/08 (2023.01); G06V 10/24 (2022.01); G06V 40/10 (2022.01)
CPC G06V 10/462 (2022.01) [G06F 18/25 (2023.01); G06N 3/08 (2013.01); G06V 10/24 (2022.01); G06V 40/10 (2022.01)] 15 Claims
OG exemplary drawing
 
1. A method for detecting body keypoints, comprising;
extracting, in response to detecting that an image comprises a body, two-dimensional (2D) coordinate data for identifying body keypoint positions in the image to obtain 2D pose data; and
performing body keypoint feature fusion on the 2D pose data and depth data corresponding to the body keypoint positions to obtain three-dimensional (3D) pose data for identifying the body keypoint positions,
the method further comprising:
acquiring a first body motion state; describing, through first 3D pose data, a change of body keypoint positions corresponding to the first body motion state; and generating a first control instruction according to the first 3D pose data, and sending the first control instruction to a receiving side device, so as to display an action simulating operation corresponding to the first body motion state on a display screen of the receiving side device,
or
acquiring a second body motion state; describing, through second 3D pose data, a change of body keypoint positions corresponding to the second body motion state; comparing the second 3D pose data with pre-configured pose data, and generating a second control instruction in response to an inconsistent comparison result; and sending prompt information according to the second control instruction, so as to adjust the second body motion state to a target state according to the prompt information,
or
acquiring a third body motion state; describing, through third 3D pose data, a change of body keypoint positions corresponding to the third body motion state; and sending the third 3D pose data to the receiving side device, so as to display, on the display screen of the receiving side device, operations of a virtual avatar sampling the third 3D pose data.