US 11,989,861 B2
Deep learning-based real-time detection and correction of compromised sensors in autonomous machines
Wenlong Yang, Shanghai (CN); Tomer Rider, Naahryia (IL); and Xiaopei Zhang, Shanghai (CN)
Assigned to INTEL CORPORATION, Santa Clara, CA (US)
Filed by Intel Corporation, Santa Clara, CA (US)
Filed on Nov. 28, 2017, as Appl. No. 15/824,808.
Prior Publication US 2019/0025773 A1, Jan. 24, 2019
Int. Cl. G06N 3/08 (2023.01); G05B 13/02 (2006.01); G06F 18/214 (2023.01); G06F 18/24 (2023.01); G06F 18/2413 (2023.01); G06F 18/25 (2023.01); G06N 3/045 (2023.01); G06N 3/084 (2023.01); G06N 5/04 (2023.01); G06T 5/77 (2024.01); G06T 7/00 (2017.01); G06V 10/764 (2022.01); G06V 10/80 (2022.01); G06V 10/82 (2022.01); G06V 10/98 (2022.01); G05D 1/00 (2006.01); G06N 3/044 (2023.01)
CPC G06T 5/77 (2024.01) [G05B 13/0265 (2013.01); G06F 18/214 (2023.01); G06F 18/24 (2023.01); G06F 18/24133 (2023.01); G06F 18/251 (2023.01); G06N 3/045 (2023.01); G06N 3/08 (2013.01); G06N 3/084 (2013.01); G06N 5/04 (2013.01); G06T 7/0002 (2013.01); G06V 10/764 (2022.01); G06V 10/803 (2022.01); G06V 10/82 (2022.01); G06V 10/993 (2022.01); G05D 1/00 (2013.01); G06N 3/044 (2023.01); G06T 2207/20081 (2013.01); G06T 2207/20084 (2013.01); G06T 2207/30168 (2013.01)] 9 Claims
OG exemplary drawing
 
1. An apparatus comprising:
one or more processors to:
capture, via one or more sensors associated with an image capturing device, one or more images of a scene, wherein an image of the one or more images is determined to be an unclear image when it is obscured or indecipherable due to one or more technical defects associated with a sensor or a physical obstruction obscuring the sensor;
receive one or more data inputs associated with the one or more images to concatenate the one or more data inputs into a single data input to be processed by a deep learning model, wherein the one or more data inputs are collected from the one or more sensors and concatenated into the single data input prior to being processed by the deep learning model;
classify data associated with the processed single data input to generate results including one or more of identifying the sensor being defective or obstructed, notifying the sensor being identified as defective or obstructed, wherein the sensor identified as defective or obstructed is further classified as a compromised sensor;
auto-correct, on-the-fly, based on identifying and notifying of the sensor being defective or obstructed, the defective or obstructed sensor associated with the unclear image such that the sensor is auto-corrected to overcome a technical defect or the physical obstruction without interruption and while one or more tasks associated with the image capturing device are in progress, wherein auto-correcting includes real-time and on-the-fly issuing of one or more alerts to warn and real-time and on-the-fly fixing of the compromised sensor; and
receive the single data input to perform one or more deep learning processes including a training process and an inferencing process to obtain real-time identification of the sensor associated with the unclear image, wherein the results including the real-time identification of the sensor being defective or obstructed are propagated with weight updates to predict one or more technical defects associated with the one or more sensors, wherein the one or more technical defects are at least partially responsible for the sensor being detective or obstructed.