US 11,941,815 B2
Method and a system training a model to perform semantic segmentation on foggy images
Hiroaki Shimizu, Brussels (BE); Dengxin Dai, Zurich (CH); Christos Sakaridis, Zurich (CH); and Luc Van Gool, Zurich (CH)
Assigned to TOYOTA MOTOR EUROPE, Brussels (BE); and ETH ZURICH, Zurich (CH)
Appl. No. 17/258,519
Filed by TOYOTA MOTOR EUROPE, Brussels (BE); and ETH ZURICH, Zurich (CH)
PCT Filed Jul. 24, 2018, PCT No. PCT/EP2018/070074
§ 371(c)(1), (2) Date Jan. 7, 2021,
PCT Pub. No. WO2020/020446, PCT Pub. Date Jan. 30, 2020.
Prior Publication US 2021/0158098 A1, May 27, 2021
Int. Cl. G06K 9/00 (2022.01); G06F 18/214 (2023.01); G06N 3/08 (2023.01); G06T 7/11 (2017.01); G06V 10/44 (2022.01); G06V 10/776 (2022.01); G06V 20/10 (2022.01); G06V 20/56 (2022.01); G06V 20/58 (2022.01); G06V 20/70 (2022.01)
CPC G06T 7/11 (2017.01) [G06F 18/2148 (2023.01); G06N 3/08 (2013.01); G06V 10/44 (2022.01); G06V 10/776 (2022.01); G06V 20/10 (2022.01); G06V 20/56 (2022.01); G06V 20/58 (2022.01); G06V 20/70 (2022.01); G06T 2207/10004 (2013.01); G06T 2207/20081 (2013.01); G06T 2207/20084 (2013.01)] 12 Claims
OG exemplary drawing
 
1. A method for training a model to be used for semantic segmentation of images, comprising:
a—obtaining a first plurality of foggy images having different densities of synthetic fog,
b—training a classification model for estimating fog density using the first plurality of foggy images and the associated different densities,
c—classifying a second plurality of images using the classification model of step b according to the fog density in the images, so as to obtain a second plurality of foggy images each having a fog density comprised within a first fog density threshold and a second fog density threshold,
d—obtaining a third plurality of foggy images each having a fog density of synthetic fog comprised within the first fog density threshold and the second fog density threshold,
e—training a semantic segmentation model using the third plurality of foggy images and predefined semantic segmentations of the third plurality of images,
f—applying the semantic segmentation model of step e to the second plurality of foggy images to obtain semantic segmentations of the second plurality of foggy images,
g—obtaining a fourth plurality of foggy images each having a fog density of synthetic fog comprised within a third fog density threshold and a fourth fog density threshold, the third and fourth fog density thresholds being both greater than the first and second fog density thresholds,
h—training the semantic segmentation model of step e using:
the fourth plurality of foggy images and predefined semantic segmentations of the fourth plurality of foggy images,
the second plurality of foggy images and the semantic segmentations of the plurality of foggy images obtained in step f.