US 11,983,625 B2
Robust multimodal sensor fusion for autonomous driving vehicles
Nilesh Ahuja, Cupertino, CA (US); Ignacio J. Alvarez, Portland, OR (US); Ranganath Krishnan, Hillsboro, OR (US); Ibrahima J. Ndiour, Portland, OR (US); Mahesh Subedar, Laveen, AZ (US); and Omesh Tickoo, Portland, OR (US)
Assigned to Intel Corporation, Santa Clara, CA (US)
Filed by Intel Corporation, Santa Clara, CA (US)
Filed on Jun. 24, 2020, as Appl. No. 16/911,100.
Prior Publication US 2020/0326667 A1, Oct. 15, 2020
Int. Cl. G06N 3/08 (2023.01); G05B 13/02 (2006.01); G06F 18/2431 (2023.01); G06F 18/25 (2023.01); G06N 5/046 (2023.01); G06N 7/01 (2023.01)
CPC G06N 3/08 (2013.01) [G05B 13/026 (2013.01); G05B 13/027 (2013.01); G06F 18/2431 (2023.01); G06F 18/251 (2023.01); G06N 5/046 (2013.01); G06N 7/01 (2023.01)] 21 Claims
OG exemplary drawing
 
1. An electronic control unit (ECU), comprising:
a plurality of a neural networks, each one of the plurality of neural networks having an input coupled to a respective autonomous vehicle (AV) sensor and configured to output distribution data representing a set of different classes used in accordance with an AV application environmental model;
a plurality of uncertainty estimation units, each one of the plurality of uncertainty estimation units being configured to calculate uncertainty estimate values comprising an aleatoric uncertainty value and an epistemic uncertainty value using the distribution data output from a respectively coupled one of the plurality of neural networks, the aleatoric uncertainty value being with respect to data provided by the AV sensor coupled to the respectively coupled one of the plurality of neural networks; and
a control unit configured to:
gate a contribution of data provided by each AV sensor to the distribution data output by each one of the plurality of neural networks in accordance with a gating function that utilizes respective uncertainty estimate values calculated for each one of the plurality of neural networks; and
generate the AV application environmental model using the gated distribution data.