	US 12,406,473 B2
	Method and system for enhancing online reflected light ferrograph image
Shuo Wang, Xi'an (CN); Jing Liu, Xi'an (CN); Tonghai Wu, Xi'an (CN); Miao Wan, Xi'an (CN); Yaguo Lei, Xi'an (CN); and Junyi Cao, Xi'an (CN)
Assigned to Xi'an Jiaotong University, xi'an (CN)
Filed by Xi'an Jiaotong University, Xi'an (CN)
Filed on Jan. 13, 2023, as Appl. No. 18/154,760.
Claims priority of application No. 202210550282.7 (CN), filed on May 20, 2022.
Prior Publication US 2023/0154158 A1, May 18, 2023
Int. Cl. G06V 10/77 (2022.01); G06N 3/0455 (2023.01); G06N 3/048 (2023.01); G06T 5/00 (2024.01); G06T 5/50 (2006.01); G06T 5/70 (2024.01); G06T 7/00 (2017.01); G06T 7/73 (2017.01); G06V 10/778 (2022.01); G06V 10/82 (2022.01)

CPC G06V 10/7715 (2022.01) [G06N 3/0455 (2023.01); G06N 3/048 (2023.01); G06T 5/50 (2013.01); G06T 5/70 (2024.01); G06T 7/001 (2013.01); G06T 7/74 (2017.01); G06V 10/7796 (2022.01); G06V 10/82 (2022.01); G06T 2207/20192 (2013.01); G06T 2207/20221 (2013.01); G06T 2207/30164 (2013.01)]

6 Claims

1. A method for enhancing an online reflected light ferrograph image, comprising:

(S1) fusing a U-shaped encoder-decoder network (Unet) architecture and a SqueezeNet network, based on contour markers of wear particles in the online reflected light ferrograph image, to construct a SqueezeNet-Unet-based wear particle position prediction network through steps of:

(S101) marking contours of the wear particles in the online reflected light ferrograph image to construct a wear particle position marking map;

(S102) constructing an encoder of the SqueezeNet-Unet-based wear particle position prediction network based on the Unet architecture by using the SqueezeNet network combined with short-cut;

(S103) performing up-sampling on a wear particle feature map by using bicubic interpolation to construct a decoder of the SqueezeNet-Unet-based wear particle position prediction network obtained in step (S102); and

(S104) taking a Sigmoid activation function as an output layer of the SqueezeNet-Unet-based wear particle position prediction network obtained in step (S102); and transforming an input image of the output layer into wear particle pixel position probability map to achieve automatic localization of the wear particles in the online reflected light ferrograph image;

(S2) constructing a ResNeXt-cycle-consistent generative adversarial network (CycleGAN) image transformation network based on a CycleGAN architecture; and subjecting the SqueezeNet-Unet-based wear particle position prediction network constructed in step (S1) and the ResNeXt-CycleGAN image transformation network to concatenate fusion to construct an online reflected light ferrograph image enhancement model;

wherein

a mode of the concatenate fusion is weighted fusion based on an output of the SqueezeNet-Unet-based wear particle position prediction network and the original online reflected light ferrograph image; and a fusion result is taken as an input of the ResNeXt-CycleGAN image transformation network;

an input layer of a generator of the ResNext-CycleGAN image transformation network is formed by one Conv-GN-ReLU block, and is configured to adjust the number of channels of an input image to the number of input channels of a feature extraction part; an encoder of the generator of the ResNeXt-CycleGAN image transformation network is constructed by using two Conv-GN-ReLU blocks; a feature transformation layer of the generator of the ResNeXt-CycleGAN image transformation network is constructed by using ResNeXt; a decoder of the generator of the ResNext-CycleGAN image transformation network is constructed by using two Deconv-GN-ReLU blocks; and an output layer of the generator of the ResNeXt-CycleGAN image transformation network is formed by one Conv-GN-Tan H block; and

a PatchGAN structure of a Pix2pix network is used as a discriminator of the ResNeXt-CycleGAN image transformation network; a dropout layer is introduced, and fully connected (FC)-Sigmoid is used as an output layer of the discriminator to output an image pixel discrimination probability;

(S3) determining a loss function of the SqueezeNet-Unet-based wear particle position prediction network constructed in step (S1); optimizing a cycle-consistency loss function of the ResNeXt-CycleGAN image transformation network constructed in step (S2) by combining structural similarity (SSIM) loss and L1 loss; and designing an overall loss function of the online reflected light ferrograph image enhancement model by weighted fusion;

wherein the overall loss function of the online reflected light ferrograph image enhancement model is designed through steps of:

taking a Focal loss as the loss function of the SqueezeNet-Unet-based wear particle position prediction network; and taking a cross-entropy loss as an adversarial loss function of the ResNext-CycleGAN image transformation network;

matching luminance and contrast information of a cycle-consistency reconstructed image of the ResNeXt-CycleGAN image transformation network and an input image of the ResNeXt-CycleGAN image transformation network by using the SSIM loss; and combining the SSIM loss L_SSIMand the loss L_L_₁as the cycle-consistency loss function of the ResNeXt-CycleGAN image transformation network; and

obtaining the overall loss function of the online reflected light ferrograph image enhancement model by weighted sum based on the Focal loss L_{Focal_loss}of the SqueezeNet-Unet-based wear particle position prediction network and an adversarial loss L_GANand the cycle-consistency loss function of the ResNeXt-CycleGAN image transformation network; and

(S4) with the overall loss function of the online reflected light ferrograph image enhancement model obtained in step (S3) as an optimization object, successively adopting a training sample set consisting of an original online reflected light ferrograph image and a traditional algorithm-enhanced online reflected light ferrograph image, and a training sample set consisting of the original online reflected light ferrograph image and an offline reflected light ferrograph image to optimize the online reflected light ferrograph image enhancement model constructed in step (S2), so as to enhance features of the wear particles in the online reflected light ferrograph image;

wherein step (S4) is performed through steps of:

(S401) acquiring a transmitted light image and a reflected light image; enhancing the reflected light image by using color restoration algorithm, and superimposing the transmitted light image on an enhanced reflected light image to obtain an initial sample required for training the online reflected light ferrograph image enhancement model;

(S402) taking the offline reflected light ferrograph image acquired by offline ferrography system as a final sample;

(S403) training the SqueezeNet-Unet-based wear particle position prediction network on an ImageNet dataset to obtain network parameters; and migrating the network parameters as pre-training parameters for an encoder of the SqueezeNet-Unet-based wear particle position prediction network;

(S404) training the online reflected light ferrograph image enhancement model by using an Adam algorithm with the reflected light image and the initial sample obtained in step (S401) as training sample; and

(S405) training the online reflected light ferrograph image enhancement model by using a Stochastic Gradient Descent (SGD) algorithm with the reflected light image and the final sample obtained in step (S402) as training sample, to achieve construction of the online reflected light ferrograph image enhancement model and enhancement of the online reflected light ferrograph image.