US 12,073,575 B2
Object-centric three-dimensional auto labeling of point cloud data
Ruizhongtai Qi, Mountain View, CA (US); Yin Zhou, San Jose, CA (US); Dragomir Anguelov, San Francisco, CA (US); and Pei Sun, Palo Alto, CA (US)
Assigned to Waymo LLC, Mountain View, CA (US)
Filed by Waymo LLC, Mountain View, CA (US)
Filed on Aug. 20, 2021, as Appl. No. 17/407,795.
Claims priority of provisional application 63/114,461, filed on Nov. 16, 2020.
Claims priority of provisional application 63/068,966, filed on Aug. 21, 2020.
Prior Publication US 2022/0058818 A1, Feb. 24, 2022
Int. Cl. G06T 7/521 (2017.01); G06T 7/20 (2017.01)
CPC G06T 7/521 (2017.01) [G06T 7/20 (2013.01); G06T 2207/10028 (2013.01); G06T 2207/20084 (2013.01)] 17 Claims
OG exemplary drawing
 
1. A method comprising:
obtaining a sensor data segment comprising a temporal sequence of three-dimensional point clouds generated from sensor readings of an environment by one or more sensors, each three-dimensional point cloud comprising a respective plurality of points in a first coordinate system;
identifying, from the sensor data segment, (i) a plurality of object tracks that each corresponds to a different object in the environment and (ii) for each object track, respective initial three-dimensional regions in each of one or more of the point clouds in which the corresponding object appears, wherein each initial three-dimensional region is an initial estimate of the three-dimensional region of the point cloud that includes points that are measurements of the corresponding object, wherein the identifying comprises:
processing each of the point clouds in the temporal sequence using an object detector to obtain, for each point cloud, a detector output that identifies a plurality of three-dimensional regions in the point cloud that are predicted to correspond to objects; and
processing the detector output using an object tracker to obtain an object tracker output that associates each of at least a subset of the three-dimensional regions in each of the point clouds with a respective one of the plurality of object tracks;
generating, for each object track, extracted object track data that includes at least the points in the respective initial three-dimensional regions for the object track; and
generating, for each object track and from the extracted object track data for the object track, an auto labeling output that defines respective refined three-dimensional regions in each of the one or more point clouds that is a refined estimate of the three-dimensional region of the point cloud that includes points that are measurements of the corresponding object.