US 12,354,322 B1
Multimodal image matching method and system, terminal device, and storage medium
Qingsong Wang, Guangdong (CN); Yijun Liu, Guangdong (CN); Haifeng Huang, Guangdong (CN); and Tao Lai, Guangdong (CN)
Assigned to SUN YAT-SEN UNIVERSITY, Guangzhou (CN)
Filed by SUN YAT-SEN UNIVERSITY, Guangdong (CN)
Filed on Nov. 21, 2024, as Appl. No. 18/954,565.
Application 18/954,565 is a continuation in part of application No. PCT/CN2024/103035, filed on Jul. 2, 2024.
Claims priority of application No. 202410034059.6 (CN), filed on Jan. 10, 2024.
Int. Cl. G06V 10/75 (2022.01); G06T 7/174 (2017.01); G06V 10/32 (2022.01)
CPC G06V 10/757 (2022.01) [G06T 7/174 (2017.01); G06V 10/32 (2022.01); G06T 2207/10044 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A multimodal image matching method, comprising:
performing self-supervised feature extraction on an optical image and a synthetic aperture radar (SAR) image to obtain a repetitive feature point between the optical image and the SAR image;
segmenting the optical image and the SAR image into a first image block sequence based on the repetitive feature point, and performing feature extraction on the first image block sequence through a dual-branch network to obtain feature description vectors of the optical image and the SAR image respectively, wherein the dual-branch network comprises a first branch network for extracting a global feature and a second branch network for extracting a local feature; and
performing feature matching on the optical image and the SAR image based on the feature description vectors to obtain a matching point pair between the optical image and the SAR image.