US 12,475,532 B2
Image deblurring using a multi-layer LSTM network
Arlene Cole-Rhodes, Bel Air, MD (US); and Greig Richmond, Severn, MD (US)
Assigned to Morgan State University, Baltimore, MD (US)
Filed by Morgan State University, Baltimore, MD (US)
Filed on Mar. 17, 2023, as Appl. No. 18/185,862.
Claims priority of provisional application 63/321,012, filed on Mar. 17, 2022.
Prior Publication US 2023/0298147 A1, Sep. 21, 2023
Int. Cl. G06T 5/20 (2006.01); G06T 5/73 (2024.01)
CPC G06T 5/20 (2013.01) [G06T 5/73 (2024.01); G06T 2207/20081 (2013.01); G06T 2207/20084 (2013.01)] 9 Claims
OG exemplary drawing
 
1. A computer implemented method for deblurring an image, comprising,
a. inputting an image file into a computer memory,
b. using a computer processor to retrieve an image file from said computer memory,
c. using said computer processor to pass said image file through a neural network comprising at least three dilated image filters in parallel to produce an output file for each at least three dilated image filters, each of said at least three dilated image filters having a different resolution from a most coarse resolution to a most fine resolution, including one or more intermediate resolutions;
d. using said computer processor to supply said most coarse resolution output file from said at least three dilated image filters as a first input to a long-short term memory “LSTM”) cell of said neural network, followed by supplying an intermediate resolution output file from said at least three dilated image filters as a second input to the LSTM cell, followed by supplying said most fine resolution output file from said at least three dilated image filters as a third input to the LSTM cell;
e. adding an output of the LSTM cell to said image file via a residual connection to produce an LSTM inception block output file;
f. using said LSTM inception block output file as a new image file and repeating steps c. through e. at least three times;
wherein said neural network is trained using a standard mean squared error loss (MSE):

OG Complex Work Unit Math
where n is a number of pixels in a training image, X is a target output, and X  is a recovered output from the network, where a learning rate (or step-size for the weight updates) for training the network is 1e-5 and an optimization algorithm used to train the network is adaptive moment estimation algorithm (Adam).