| CPC G06T 11/006 (2013.01) [A61B 5/0515 (2013.01); G06T 3/4046 (2013.01); G06T 2210/41 (2013.01); G06T 2211/441 (2023.08)] | 8 Claims |
|
1. A system for reconstructing a magnetic particle image based on a pre-trained model, comprising a magnetic particle imaging (MPI) device, a signal processor, and a control processor, wherein
a wired or wireless communication exists between the MPI device, the signal processor, and the control processor;
the control processor is configured to adjust a parameter of the MPI device and control the MPI device to scan a magnetic particle sample, through the wired or wireless communication; and
the signal processor comprises:
a simulation system generation module, configured to generate simulation system matrices of the magnetic particle imaging system with different parameters;
a first neural network model parameter acquisition module, configured to pre-train a pre-constructed neural network model according to the simulation system matrices, take a pre-trained neural network model as a first neural network model, and acquire a parameter of the first neural network model;
wherein the first neural network model parameter acquisition module comprises: a matrix conversion module, configured to convert the simulation system matrices obtained from the simulation system generation module into real-domain matrices as first matrices, initialize all-1 matrices of a same size as the first matrices, set a value of a first preset percentage in the all-1 matrices to zero to acquire mask matrices, and multiply the mask matrices with the first matrices to acquire masked simulation system matrices as second matrices;
a recovered system matrix acquisition module, configured to take the first matrices as true value labels, divide the second matrices into a plurality of matrix blocks with an equal size, and take the plurality of matrix blocks as an input into the pre-constructed neural network model, wherein the pre-constructed neural network model comprises a first encoder and a first decoder; and recovered system matrices are acquired as third matrices based on the plurality of matrix blocks through the pre-constructed neural network model;
a first loss function calculation module, configured to calculate a first loss function between the recovered system matrices and the true value labels, and adjust a parameter of the pre-constructed neural network model according to the first loss function;
a first loop module, configured to loop through the recovered system matrix acquisition module and the first loss function calculation module according to a set number of training epochs until training of the pre-constructed neural network is completed, take a trained pre-constructed neural network as the first neural network model, and acquire the parameter of the first neural network model;
wherein the recovered system matrices are acquired as the third matrices based on the plurality of matrix blocks through the pre-constructed neural network model by:
converting the plurality of matrix blocks into one-dimensional vectors, encoding the one-dimensional vectors to acquire matrix block vectors, and adding learnable position embedding to the matrix block vectors to acquire encoded matrix block vectors with the learnable position embedding, as first vectors;
inputting the first vectors into the first encoder to acquire first feature vectors; and
mapping a channel number of the first feature vectors to a dimension of the first decoder, inputting the channel mapped first feature vectors into the first decoder to acquire second feature vectors, converting the second feature vectors into a plurality of two-dimensional matrix blocks, and splicing the plurality of two-dimensional matrix blocks into the third matrices;
wherein the first encoder is configured to encode the plurality of matrix blocks to acquire the first feature vectors, and the first encoder comprises a first plurality of consecutive self-attention layers;
each self-attention layer of the first plurality of consecutive self-attention layers comprises a self-attention layer input terminal, a multi-head attention layer, a first addition unit, a first layer normalization layer, a feedforward network, a second addition unit, a second layer normalization layer, and a self-attention layer output terminal that are sequentially connected;
the self-attention layer input terminal is in a residual connection to the first addition unit, and an output terminal of the first layer normalization layer is in a residual connection to the second addition unit;
the multi-head attention layer comprises a multi-head attention layer input terminal, Q parallel dot product attention blocks, a feature connection layer, a first fully connected layer, and a multi-head attention layer output terminal that are sequentially connected, wherein Q is an integer;
each of the Q parallel dot product attention blocks comprises a dot-product first fully connected layer, a dot-product second fully connected layer, and a dot-product third fully connected layer that are arranged in parallel; an output of the dot-product first fully connected layer and an output of the dot-product second fully connected layer are jointly connected to a matrix multiplication unit, and are sequentially connected to a normalization layer and a softmax layer; and an output of the softmax layer and the dot-product third fully connected layer are jointly connected to the matrix multiplication unit, and are connected to a dot product attention block output terminal; and
the first decoder is configured to decode the first feature vectors to acquire the third matrices, and the first decoder comprises a second plurality of consecutive self-attention layers;
a third neural network model acquisition module, configured to generate a data set corresponding to a downstream task, pre-construct a neural network model corresponding to the downstream task as a second neural network model, input the parameter of the first neural network model into the second neural network model, and train the second neural network model loaded with the parameter of the first neural network model through the data set to acquire a third neural network model; and
an image reconstruction module, configured to input acquired real input data collected by the MPI device into the third neural network model for enhancement, play an auxiliary role to acquire a reconstructed MPI image, and accurately locate a tumor or target based on the reconstructed MPI image, wherein when the downstream task is an X-space reconstruction related method, the input data is divided frames of one-dimensional frequency-domain signals acquired by performing Fourier transform on acquired real noisy one-dimensional time-domain signals; and when the downstream task is a system matrix reconstruction related method, the input data is a collected low-quality system matrix.
|