US 12,346,809 B2
Method, device, and storage medium for deep learning based domain adaptation with data fusion for aerial image data analysis
Jingyang Lu, Germantown, MD (US); Erik Blasch, Arlington, VA (US); Roman Ilin, Dayton, OH (US); Hua-mei Chen, Germantown, MD (US); Dan Shen, Germantown, MD (US); Nichole Sullivan, Germantown, MD (US); and Genshe Chen, Germantown, MD (US)
Assigned to INTELLIGENT FUSION TECHNOLOGY, INC., Germantown, MD (US)
Filed by Intelligent Fusion Technology, Inc., Germantown, MD (US)
Filed on Sep. 21, 2021, as Appl. No. 17/480,999.
Claims priority of provisional application 63/081,036, filed on Sep. 21, 2020.
Prior Publication US 2022/0092420 A1, Mar. 24, 2022
Int. Cl. G06N 3/08 (2023.01); G06F 18/214 (2023.01); G06F 18/25 (2023.01); G06V 20/00 (2022.01)
CPC G06N 3/08 (2013.01) [G06F 18/214 (2023.01); G06F 18/253 (2023.01); G06V 20/00 (2022.01)] 12 Claims
OG exemplary drawing
 
1. A domain adaptation for efficient learning fusion (DAELF) method, comprising:
acquiring data from a plurality of data sources of a plurality of sensors;
for each of the plurality of sensors, training an auxiliary classifier generative adversarial network (AC-GAN) by a hardware processor, wherein the AC-GAN includes a feature extraction network, a label prediction network, a generator network, and a discriminator network, with data from each data source of the plurality of data sources, thereby obtaining a trained feature extraction network and a trained label prediction network for each data source;
using the trained feature extraction network and the trained label prediction network for each data source on a sensor side, and a corresponding centralized fusion network on a fusion center side to form a decision-level fusion network; or using the trained feature extraction network for each data source on the sensor side and a corresponding centralized fusion network on the fusion center side to form a feature-level fusion network;
training the decision-level fusion network or the feature-level fusion network with a source-only mode or a generate to adapt (GTA) mode, wherein:
at the source-only mode, the trained feature extraction network for each data source and the corresponding centralized fusion network are trained with labeled source data, and
at the GTA mode, the trained feature extraction network for each data source and the corresponding centralized fusion network are trained separately, wherein the trained feature extraction network for each data source is trained with the labeled source data and unlabeled target data; and the corresponding centralized fusion network is trained with the labeled source data only; and
applying the trained decision-level fusion network or the trained feature-level fusion network to detect a target of interest.