US 12,148,223 B2
Shared vision system backbone
Arjun Bhargava, San Francisco, CA (US); Chao Fang, Sunnyvale, CA (US); Charles Christopher Ochoa, San Francisco, CA (US); Kun-Hsin Chen, San Francisco, CA (US); Kuan-Hui Lee, Jose, CA (US); and Vitor Guizilini, Santa Clara, CA (US)
Assigned to TOYOTA RESEARCH INSTITUTE, INC., Los Altos, CA (US); and TOYOTA JIDOSHA KABUSHIKI KAISHA, Aichi-Ken (JP)
Filed by TOYOTA RESEARCH INSTITUTE, INC., Los Altos, CA (US)
Filed on Apr. 28, 2022, as Appl. No. 17/732,421.
Prior Publication US 2023/0351767 A1, Nov. 2, 2023
Int. Cl. G06V 20/58 (2022.01); B60W 60/00 (2020.01); G06V 20/40 (2022.01)
CPC G06V 20/58 (2022.01) [B60W 60/001 (2020.02); G06V 20/49 (2022.01); B60W 2420/403 (2013.01); B60W 2420/408 (2024.01)] 20 Claims
OG exemplary drawing
 
1. A method for generating a dense light detection and ranging (LiDAR) representation by a vision system of a vehicle, comprising:
receiving, at a sparse depth network, one or more sparse representations of an environment within a vicinity of the vehicle;
generating, at a depth estimation network, a depth estimate of the environment depicted in an image captured by an image capturing sensor integrated with the vehicle based on receiving the one or more sparse representation;
generating, via the sparse depth network, one or more sparse depth estimates based on receiving the one or more sparse representations of the environment, each sparse depth estimate associated with a respective sparse representation of the one or more sparse representations;
fusing, at a depth fusion network, the depth estimate and the one or more sparse depth estimates to generate a dense depth estimate;
generating the dense LiDAR representation based on the dense depth estimate; and
controlling an action of the vehicle based on identifying a three-dimensional object in the dense LiDAR representation.