US 11,783,591 B2
Using a multimode neural network to determine a representation of a spatial structure objects in an image
Sudeep Pillai, Santa Clara, CA (US); Vitor Guizilini, Santa Clara, CA (US); Rares A. Ambrus, San Fransisco, CA (US); and Adrien David Gaidon, Mountain View, CA (US)
Assigned to Toyota Research Institute, Inc., Los Altos, CA (US)
Filed by Toyota Research Institute, Inc., Los Altos, CA (US)
Filed on Jun. 11, 2020, as Appl. No. 16/898,541.
Prior Publication US 2021/0387649 A1, Dec. 16, 2021
Int. Cl. G06V 20/56 (2022.01); G06N 3/08 (2023.01); G06F 18/214 (2023.01); G06N 3/045 (2023.01); B60W 60/00 (2020.01); G06V 10/82 (2022.01)
CPC G06V 20/56 (2022.01) [B60W 60/0025 (2020.02); G06F 18/214 (2023.01); G06N 3/045 (2023.01); G06N 3/08 (2013.01); G06V 10/82 (2022.01); B60W 2556/50 (2020.02)] 20 Claims
OG exemplary drawing
 
1. A system for determining a representation of a spatial structure of objects in an image, the system comprising:
one or more processors; and
a memory communicably coupled to the one or more processors and storing:
a setting module including instructions that when executed by the one or more processors cause the one or more processors to set values of weights to be applied to inputs of nodes of a first neural network and thereby determine a mode of the first neural network in response to a receipt of the image and a receipt of a facing direction of a camera that produced the image, wherein the mode accounts for the facing direction, the facing direction comprises at least one of a first facing direction of a first camera disposed on a vehicle and a second facing direction of a second camera disposed on the vehicle, wherein:
first values of the weights are applied to the inputs of the nodes in response to the facing direction of the camera that produced the image being the first facing direction, and
second values of the weights are applied to the inputs of the nodes in response to the facing direction of the camera that produced the image being the second facing direction;
a neural network execution module including instructions that when executed by the one or more processors cause the one or more processors to determine, in response to the mode having been set and via the first neural network network, the representation of the spatial structure of the objects in the image; and
a communications module including instructions that when executed by the one or more processors cause the one or more processors to transmit the representation of the spatial structure of the objects in the image to an automotive navigation system, wherein the automotive navigation system is configured to use the representation of the spatial structure of the objects in the image to determine a distance between the vehicle and a specific object in the image.