CPC G06T 17/00 (2013.01) [G06N 3/04 (2013.01); G06T 7/251 (2017.01); G06T 7/344 (2017.01); G06T 7/50 (2017.01); G06T 19/20 (2013.01); G06V 20/64 (2022.01); G06T 2207/10016 (2013.01); G06T 2207/20081 (2013.01); G06T 2207/20084 (2013.01); G06T 2207/30236 (2013.01); G06T 2207/30241 (2013.01); G06T 2207/30252 (2013.01); G06T 2210/12 (2013.01); G06T 2219/004 (2013.01); G06T 2219/2004 (2013.01); G06V 2201/08 (2022.01)] | 16 Claims |
1. A method for monocular 3D object modeling and auto-labeling with 2D semantic keypoints, comprising:
predicting, using a continuously traversable coordinate shape space (CSS) network, a normalized object coordinate space (NOCS) image and a shape vector corresponding to an input 2D labeled object;
lifting linked, 2D semantic keypoints of a 2D structured object geometry of the input 2D label object into a 3D structured object geometry;
geometrically and projectively aligning the 2D NOCS image to the 3D structured vehicle geometry and a 3D object model decoded from the shape vector;
back projecting the 2D semantic keypoints to auto-label 3D bounding boxes from the 3D object model;
enforcing geometric and projective verification constraints on the auto-labeled 3D bounding boxes to identify verified auto-labeled 3D bounding boxes and unverified auto-labeled 3D bounding boxes;
saving the verified auto-labeled 3D bounding boxes in a CSS pool and discarding the unverified auto-labeled 3D bounding boxes; and
retraining the CSS network using the saved, verified auto-labeled 3D bounding boxes.
|