US 12,112,497 B2
LIDAR-camera fusion where LIDAR and camera validly see different things
Chaim Blau, New York, NY (US); Ofer Springer, Jerusalem (IL); Kevin Rosenblum, Jerusalem (IL); Alon Ziv, Motza Illit (IL); Erez Dagan, Rishpon (IL); David Boublil, Jerusalem (IL); Nadav Shaag, Jerusalem (IL); David Neuhof, Jerusalem (IL); Jeffrey Moskowitz, Tel Aviv (IL); Gal Topel, Jerusalem (IL); and Yotam Stern, Jerusalem (IL)
Assigned to MOBILEYE VISION TECHNOLOGIES LTD., Jerusalem (IL)
Filed by MOBILEYE VISION TECHNOLOGIES LTD., Jerusalem (IL)
Filed on Jun. 29, 2022, as Appl. No. 17/809,612.
Application 17/809,612 is a continuation of application No. PCT/US2020/067753, filed on Dec. 31, 2020.
Claims priority of provisional application 63/082,619, filed on Sep. 24, 2020.
Claims priority of provisional application 62/957,000, filed on Jan. 3, 2020.
Prior Publication US 2022/0324437 A1, Oct. 13, 2022
Int. Cl. G01S 17/42 (2006.01); B60W 10/04 (2006.01); B60W 10/18 (2012.01); B60W 10/20 (2006.01); B60W 30/09 (2012.01); B60W 60/00 (2020.01); G01B 11/22 (2006.01); G01C 21/00 (2006.01); G01C 21/16 (2006.01); G01C 21/30 (2006.01); G01S 7/481 (2006.01); G01S 17/58 (2006.01); G01S 17/86 (2020.01); G01S 17/89 (2020.01); G01S 17/931 (2020.01); G06T 7/55 (2017.01); H04N 23/698 (2023.01); H04N 23/90 (2023.01)
CPC G06T 7/55 (2017.01) [B60W 10/04 (2013.01); B60W 10/18 (2013.01); B60W 10/20 (2013.01); B60W 30/09 (2013.01); B60W 60/001 (2020.02); G01B 11/22 (2013.01); G01C 21/1652 (2020.08); G01C 21/1656 (2020.08); G01C 21/30 (2013.01); G01C 21/3885 (2020.08); G01S 7/4817 (2013.01); G01S 17/42 (2013.01); G01S 17/58 (2013.01); G01S 17/86 (2020.01); G01S 17/89 (2013.01); G01S 17/931 (2020.01); H04N 23/698 (2023.01); H04N 23/90 (2023.01); B60W 2420/403 (2013.01); B60W 2420/408 (2024.01); B60W 2554/20 (2020.02); B60W 2554/4042 (2020.02); B60W 2554/802 (2020.02); B60W 2556/45 (2020.02); B60W 2720/10 (2013.01); B60W 2720/24 (2013.01); G06T 2207/10028 (2013.01); G06T 2207/20081 (2013.01); G06T 2207/20084 (2013.01); G06T 2207/30252 (2013.01)] 23 Claims
OG exemplary drawing
 
1. A navigation system for a host vehicle, the navigation system comprising:
at least one processor programmed to:
receive from a camera onboard the host vehicle at least one captured image representative of an environment of the host vehicle, wherein the camera is positioned at a first location relative to the host vehicle;
receive point cloud information from a LIDAR system onboard the host vehicle, wherein the point cloud information is representative of distances to various objects in an environment of the host vehicle, wherein the LIDAR system is positioned at a second location relative to the host vehicle, wherein the second location is different from the first location, and wherein a field of view of the camera at least partially overlaps with a field of view of the LIDAR system to provide a shared field of view region;
analyze the at least one captured image and the received point cloud information to detect one or more objects in the shared field of view region, wherein the detected one or more objects are represented in only one of the at least one captured image or the received point cloud information;
determine whether a vantage point difference between the first location of the camera and the second location of the LIDAR system accounts for the one or more detected objects being represented in only one of the at least one captured image or the received point cloud information;
if the vantage point difference does not account for the one or more detected objects being represented in only one of the at least one captured image or the received point cloud information, then cause at least one remedial action to be taken; and
if the vantage point difference does account for the one or more detected objects being represented in only one of the at least one captured image or the received point cloud information, then determine at least one navigational action to be taken by the host vehicle based on the one or more detected objects and cause the at least one navigational action to be taken by the host vehicle.