US 12,214,717 B2
Vehicular multi-sensor system using a camera and lidar sensor to detect objects
Martin Solar, Erlenbach (DE); and Christopher L Van Dan Elzen, Rochester, MI (US)
Assigned to Magna Electronics Inc., Auburn Hills, MI (US)
Filed by MAGNA ELECTRONICS INC., Auburn Hills, MI (US)
Filed on Sep. 18, 2023, as Appl. No. 18/468,776.
Application 18/468,776 is a continuation of application No. 18/164,783, filed on Feb. 6, 2023, granted, now 11,760,255.
Application 18/164,783 is a continuation of application No. 17/807,763, filed on Jun. 20, 2022, granted, now 11,572,013, issued on Feb. 7, 2023.
Application 17/807,763 is a continuation of application No. 16/699,893, filed on Dec. 2, 2019, granted, now 11,364,839, issued on Jun. 21, 2022.
Application 16/699,893 is a continuation of application No. 15/413,463, filed on Jan. 24, 2017, granted, now 10,493,899, issued on Dec. 3, 2019.
Application 15/413,463 is a continuation of application No. 15/083,894, filed on Mar. 29, 2016, granted, now 9,555,736, issued on Jan. 31, 2017.
Claims priority of provisional application 62/142,705, filed on Apr. 3, 2015.
Prior Publication US 2024/0001838 A1, Jan. 4, 2024
This patent is subject to a terminal disclaimer.
Int. Cl. B60Q 1/14 (2006.01); B60Q 1/00 (2006.01); B60R 11/04 (2006.01); B60W 30/14 (2006.01); G08G 1/16 (2006.01)
CPC B60Q 1/143 (2013.01) [B60Q 1/0023 (2013.01); B60R 11/04 (2013.01); B60W 30/14 (2013.01); G08G 1/161 (2013.01); G08G 1/163 (2013.01); G08G 1/164 (2013.01); G08G 1/166 (2013.01); B60Q 2300/132 (2013.01); B60Q 2300/41 (2013.01); B60Q 2300/42 (2013.01); B60Q 2300/45 (2013.01); B60W 2420/403 (2013.01); B60W 2420/408 (2024.01); B60W 2556/65 (2020.02)] 44 Claims
OG exemplary drawing
 
1. A vehicular multi-sensor system, said vehicular multi-sensor system comprising:
a plurality of sensors disposed at a vehicle equipped with said vehicular multi-sensor system;
wherein said plurality of sensors at least comprises (i) a forward-viewing camera, said forward-viewing camera disposed behind a windshield of the equipped vehicle and viewing forward of the equipped vehicle through the windshield and (ii) a forward-sensing 3D point-cloud LIDAR sensor having a field of sensing forward of the equipped vehicle;
wherein said forward-viewing camera is operable to capture image data;
wherein said forward-sensing 3D point-cloud LIDAR sensor is operable to capture 3D point-cloud LIDAR data;
wherein said forward-viewing camera comprises a photosensor array having at least one million photosensor elements arranged in a two-dimensional array of columns and rows;
wherein said forward-viewing camera views (i) a traffic lane of a multi-lane road being traveled along by the equipped vehicle and (ii) another traffic lane of the multi-lane road;
wherein the field of sensing of said forward-sensing 3D point-cloud LIDAR sensor at least encompasses the other traffic lane of the multi-lane road;
an electronic control unit (ECU) comprising electronic circuitry;
wherein image data captured by said forward-viewing camera is transferred from said forward-viewing camera to the ECU and is processed at the ECU;
wherein 3D point-cloud LIDAR data captured by said forward-sensing 3D point-cloud LIDAR sensor is transferred from said forward-sensing 3D point-cloud LIDAR sensor to the ECU and is processed at the ECU;
wherein, responsive at least in part to processing at the ECU of 3D point-cloud LIDAR data captured by said forward-sensing 3D point-cloud LIDAR sensor, a traffic participant present forward of the equipped vehicle on the other traffic lane of the multi-lane road is detected;
wherein image data captured by said forward-viewing camera is processed at the ECU for a headlamp control system of the equipped vehicle and for at least one selected from the group consisting of (i) a lane departure warning system of the equipped vehicle, (ii) a traffic sign recognition system of the equipped vehicle and (iii) an adaptive cruise control system of the equipped vehicle; and
wherein the headlamp control system of the equipped vehicle adjusts a headlamp of the equipped vehicle based at least in part on detection of the traffic participant responsive at least in part to processing at the ECU of 3D point-cloud LIDAR data captured by said forward-sensing 3D point-cloud LIDAR sensor.