US 12,351,182 B2
System and method to neutralize the presence of adverse weather artifacts from automotive camera imagery
Sai Vishnu Aluru, Commerce Township, MI (US); and Daniel Woodall, Ann Arbor, MI (US)
Assigned to GM GLOBAL TECHNOLOGY OPERATIONS LLC, Detroit, MI (US)
Filed by GM GLOBAL TECHNOLOGY OPERATIONS LLC, Detroit, MI (US)
Filed on Mar. 1, 2023, as Appl. No. 18/176,790.
Prior Publication US 2024/0294171 A1, Sep. 5, 2024
Int. Cl. B60W 40/02 (2006.01); G01W 1/10 (2006.01); G06T 7/254 (2017.01); G06V 20/56 (2022.01)
CPC B60W 40/02 (2013.01) [G01W 1/10 (2013.01); G06T 7/254 (2017.01); G06V 20/56 (2022.01); B60W 2420/403 (2013.01); B60W 2555/20 (2020.02)] 20 Claims
OG exemplary drawing
 
1. A method comprising:
obtaining camera images from a camera of a vehicle;
obtaining weather data as to a location in which the vehicle is travelling;
determining, via a processor of the vehicle, whether precipitation is present at the location, based on the weather data; and
when it is determined by the processor that precipitation is present:
processing the camera images, via a processor, generating one or more processed frames from the camera images, based on the weather data, at sequential points in time, generated a plurality of sequential frames comprising a first frame at a first point in time and a second frame at a second point in time;
generating, via the processor, a difference frame that includes specific pixels that appear in one of the plurality of sequential frames and not the other;
normalizing and filtering image pixels of the difference frame via the processor;
applying, via the processor, morphological openings to the image pixels of the difference frame via the processor in conjunction with the normalized pixel images, when it is determined via the processor that precipitation is present;
determining, via the processor, a ratio of pixels in the difference frame compared to pixels in the first frame, pixels in the second frame, or both, and whether the ratio exceeds a predetermined threshold;
when it is determined that the ratio of pixels in the difference frame does not exceed the predetermined threshold, then repeating the normalizing, filtering, and applying of the morphological openings via the processor without substituting the frames of the camera images in which the precipitation is present and without removing the pixels that are present in the difference frame and that reflect the precipitation, until a subsequent determination is made by the processor that the quantity of pixels in the difference frame exceeds the predetermined threshold;
when it is instead determined that the ratio of pixels in the difference frame exceeds the predetermined threshold, then substituting the frames of the camera images in which the precipitation is present and removing the pixels that are present in the difference frame and that reflect the precipitation, generating one or more enhanced frames, via the processor; and
performing a control action for the vehicle, in accordance with the instructions provided by the processor in accordance with instructions provided by the processor, based on the one or more enhanced frames.