US 12,117,519 B2
Object detection using RADAR and LiDAR fusion
Xiaoli Meng, Singapore (SG); Lubing Zhou, Singapore (SG); and Karan Rajendra Shetti, Singapore (SG)
Assigned to Motional AD LLC, Boston, MA (US)
Filed by Motional AD LLC, Boston, MA (US)
Filed on Oct. 7, 2021, as Appl. No. 17/496,495.
Prior Publication US 2023/0109909 A1, Apr. 13, 2023
Int. Cl. G01S 13/86 (2006.01); G01S 7/41 (2006.01); G01S 13/42 (2006.01); G06F 18/23 (2023.01); G06F 18/25 (2023.01); G06N 20/00 (2019.01); G06T 7/162 (2017.01); G06V 20/56 (2022.01)
CPC G01S 13/865 (2013.01) [G01S 7/412 (2013.01); G01S 7/417 (2013.01); G01S 13/42 (2013.01); G06F 18/23 (2023.01); G06F 18/253 (2023.01); G06N 20/00 (2019.01); G06T 7/162 (2017.01); G06V 20/56 (2022.01); G06T 2207/10028 (2013.01); G06T 2207/20081 (2013.01); G06T 2207/20084 (2013.01); G06T 2207/30252 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A method comprising:
obtaining, with at least one processor, a first point cloud captured by a light detection and ranging (LiDAR) sensor of a vehicle operating in an environment;
obtaining, with the at least one processor, a second point cloud captured by a radio detection and ranging (RADAR) sensor of the vehicle;
transforming, with the at least one processor, the first point cloud to a vehicle-centric reference frame;
transforming, with the at least one processor, the second point cloud to the vehicle-centric reference frame;
generating, with the at least one processor, a first set of clusters of points in the first point cloud;
generating, with the at least one processor, a first set of anchor boxes for the first set of clusters;
generating, with the at least one processor, a second set of clusters of points in the second point cloud;
generating, with the at least one processor, a second set of anchor boxes for the second set of clusters;
generating, with the at least one processor, an association of the first set of clusters and the second set of clusters in the first set of anchor boxes and the second set of anchor boxes;
generating, with the at least one processor, a third set of clusters and a third anchor box based on the association of the first set of clusters and the second set of clusters in the first set of anchor boxes and the second set of anchor boxes, wherein generating the third set of clusters includes combining features at a feature level to create fused LiDAR/RADAR-generated features using features obtained from each of a LiDAR branch and a RADAR branch in a network;
generating, with the at least one processor and using the fused LiDAR/RADAR-generated features, an object label, a bounding box, and a velocity for each cluster in the third set of clusters based on a machine learning model; and
causing, with the at least one processor, the vehicle to traverse the environment based on the object label, the bounding box, and the velocity for each cluster in the third set of clusters.