US 11,940,774 B2
Action imitation method and robot and computer readable storage medium using the same
Miaochen Guo, Shenzhen (CN); Jingtao Zhang, Shenzhen (CN); Dong Wang, Shenzhen (CN); Shuping Hu, Shenzhen (CN); Jianxin Pang, Shenzhen (CN); and Youjun Xiong, Shenzhen (CN)
Assigned to UBTECH ROBOTICS CORP LTD, Shenzhen (CN)
Filed by UBTECH ROBOTICS CORP LTD, Shenzhen (CN)
Filed on Dec. 8, 2020, as Appl. No. 17/115,735.
Claims priority of application No. 201911397998.2 (CN), filed on Dec. 30, 2019.
Prior Publication US 2021/0200190 A1, Jul. 1, 2021
Int. Cl. G05B 19/408 (2006.01); G05B 19/4155 (2006.01); G06F 18/214 (2023.01); G06F 18/22 (2023.01); G06N 3/08 (2023.01); G06V 10/764 (2022.01); G06V 20/10 (2022.01); G06V 20/64 (2022.01); G06V 40/20 (2022.01)
CPC G05B 19/4086 (2013.01) [G05B 19/4155 (2013.01); G06F 18/214 (2023.01); G06F 18/22 (2023.01); G06N 3/08 (2013.01); G06V 10/764 (2022.01); G06V 20/10 (2022.01); G06V 20/647 (2022.01); G06V 40/20 (2022.01); G05B 2219/33027 (2013.01); G05B 2219/50391 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A computer-implemented action imitation method for a robot, comprising steps of:
collecting at least a two-dimensional image of a to-be-imitated object;
obtaining two-dimensional coordinates of each key point of the to-be-imitated object in the two-dimensional image and a pairing relationship between key points of the to-be-imitated object, wherein the pairing relationship between the key points of the to-be-imitated object is for representing a relative positional relationship between adjacent joints; and
converting the two-dimensional coordinates of each key point of the to-be-imitated object in the two-dimensional image into space three-dimensional coordinates corresponding to the key points of the to-be-imitated object through a pre-trained first neural network model, and generating an action control instruction of the robot based on the space three-dimensional coordinates corresponding to the key points of the to-be-imitated object and the pairing relationship between the key points, wherein the action control instruction is for controlling the robot to imitate an action of the to-be-imitated object;
wherein the step of obtaining the two-dimensional coordinates of each key point of the to-be-imitated object in the two-dimensional image and the pairing relationship between the key points of the to-be-imitated object comprises:
inputting the two-dimensional image into a pre-trained second neural network model, and obtaining the two-dimensional coordinates of each key point of the to-be-imitated object in the two-dimensional image and the pairing relationship between the key points of the to-be-imitated object outputted by the pre-trained second neural network model.