US 12,143,698 B2
Method for creating hyperspectral high speed camera image using generative adversarial network algorithm
Sang Woo Noh, Gyeongsangnam-do (KR); Tae Hwan Kim, Gyeongsangnam-do (KR); Jin Woo Ahn, Gyeongsangnam-do (KR); and Han Gyu Kim, Seoul (KR)
Assigned to Defense Agency for Technology and Quality, Gyeongsangnam-do (KR)
Filed by Defense Agency for Technology and Quality, Gyeongsangnam-do (KR)
Filed on Dec. 22, 2021, as Appl. No. 17/558,835.
Claims priority of application No. 10-2020-0183353 (KR), filed on Dec. 24, 2020.
Prior Publication US 2022/0210347 A1, Jun. 30, 2022
Int. Cl. H04N 23/11 (2023.01); G06T 7/00 (2017.01)
CPC H04N 23/11 (2023.01) [G06T 7/97 (2017.01); G06T 2207/10036 (2013.01); G06T 2207/20081 (2013.01)] 1 Claim
OG exemplary drawing
 
1. A method for creating a hyperspectral high speed camera image, comprising:
a first step of taking images with a spectral scanning type hyperspectral high speed camera having a function of viewing data values for each spectrum, wherein data acquired by the spectral scanning type hyperspectral high speed camera are formed in the form of a three-dimensional data cube by including spectra for each pixel in addition to a two-dimensional spatial information of an image taken with a general camera, and then separately acquiring the acquired three-dimensional data cubes for each specific time point that each spectrum has been taken to create three-dimensional data cubes as many as the number of spectra;
a second step of separating a region corresponding to a moving object and a region corresponding to a background from the created data cubes through a process of detecting the moving object to separate it from the background; and
a third step of acquiring spectral values according to spatial locations of the images by using the values of each spectrum corresponding to the same coordinates in the separated background region, and reconstructing a three-dimensional data cube with all spectral values for each specific time point with regard to the region with motion through a generative adversarial network algorithm,
in the third step, the generative adversarial network is performed using the following Equations 1 to 3,
in the third step, to learn mapping GY: X→Y by Equation 1 below, GY is adjusted so as to minimize a reconstruction error in paired data samples {(xi, yi)}, assuming that a recurrent loss is obtained by n-th successive substitution of P(x) by using Equation 2 below, in Equation 3, a generator function Gi learns input data xtn so that the input data xtn become a value xt(n+i) of a next spectrum belonging into the same frame, finally, by using the identifier Gi, all spectra at a specific spectrum n time point are restored:

OG Complex Work Unit Math
In Equations 1, 2 and 3, xiϵX, yiϵY, min is a minimum value function, GY is a generator function, LT is a loss function, PN is a prediction function, T is the total time of the image, N means the number of spectra, xtn is n-th data value at time t, xt(n+N) is n+N-th data value at time t, Pi is a prediction function, Gi is a generator function, and xt(n+i) is n+i-th data value at time t.