US 11,940,804 B2
Automated object annotation using fused camera/LiDAR data points
Zhongtao Liu, Pittsburgh, PA (US); James Esper, Pittsburgh, PA (US); and Jong Ho Lee, Pittsburgh, PA (US)
Assigned to Motional AD LLC, Boston, MA (US)
Filed by MOTIONAL AD LLC, Boston, MA (US)
Filed on Dec. 9, 2020, as Appl. No. 17/116,081.
Claims priority of provisional application 62/949,345, filed on Dec. 17, 2019.
Prior Publication US 2021/0181745 A1, Jun. 17, 2021
Int. Cl. G06V 20/58 (2022.01); B60W 30/08 (2012.01); B60W 40/02 (2006.01); G01C 21/16 (2006.01); G01C 21/36 (2006.01); G01S 7/41 (2006.01); G01S 7/48 (2006.01); G01S 13/86 (2006.01); G01S 13/93 (2020.01); G01S 13/931 (2020.01); G01S 17/86 (2020.01); G01S 17/89 (2020.01); G01S 17/931 (2020.01); G05D 1/00 (2006.01); G06F 18/00 (2023.01); G08G 1/0962 (2006.01); G08G 1/16 (2006.01)
CPC G05D 1/024 (2013.01) [G01S 17/89 (2013.01); G05D 1/0274 (2013.01); G06V 20/584 (2022.01); G08G 1/16 (2013.01); G05D 2201/0213 (2013.01)] 19 Claims
OG exemplary drawing
 
1. A system comprising:
one or more computer processors; and
one or more storage media storing instructions which, when executed by the one or more computer processors, cause performance of operations comprising:
obtaining map data used for navigating a vehicle, wherein the map data identifies a plurality of objects in an environment of the vehicle;
obtaining image data of an image of the environment of the vehicle, wherein the image comprises one or more objects;
obtaining, using a LiDAR, LiDAR data points corresponding to the one or more objects;
projecting, using a processing circuit, the LiDAR data points onto the image data to generate fused data, the fused data comprising a LiDAR data portion and an image data portion;
detecting, using the processing circuit, a first target object and a second target object in the fused data;
extracting, using the processing circuit, a first subset of the LiDAR data points that corresponds to the detected first target object;
extracting, using the processing circuit, a second subset of the LiDAR data points that corresponds to the detected second target object;
automatically annotating, using the processing circuit, the map data, wherein the automatically annotating comprises:
in accordance with a determination that the extracted first subset of the LiDAR data points that corresponds to the detected first target object satisfies registration criteria associated with the detected first target object, automatically annotating the map data corresponding to the detected first target object, and
in accordance with a determination that the extracted second subset of the LiDAR data points that corresponds to the detected second target object does not satisfy the registration criteria associated with the detected second target object, not annotating map data corresponding to the detected second target object; and
navigating, using a control circuit, the vehicle in the environment according to the annotated map data.