US 12,462,335 B2
Systems and methods for multi-modal multi-dimensional image registration
Pingkun Yan, Clifton Park, NY (US); Hengtao Guo, Troy, NY (US); Xinrui Song, Troy, NY (US); and Xuanang Xu, Troy, NY (US)
Assigned to Rensselaer Polytechnic Institute, Troy, NY (US)
Filed by Rensselaer Polytechnic Institute, Troy, NY (US)
Filed on Jan. 3, 2023, as Appl. No. 18/092,531.
Claims priority of provisional application 63/316,096, filed on Mar. 3, 2022.
Prior Publication US 2023/0281751 A1, Sep. 7, 2023
Int. Cl. G06T 3/14 (2024.01); G06T 7/33 (2017.01)
CPC G06T 3/14 (2024.01) [G06T 7/337 (2017.01); G06T 2207/10088 (2013.01); G06T 2207/10132 (2013.01); G06T 2207/20084 (2013.01); G06T 2207/20221 (2013.01)] 15 Claims
OG exemplary drawing
 
1. A method of multi-modal image registration, the method comprising:
receiving as input a fixed image from a first imaging device;
receiving as input a moving image from a second imaging device;
performing feature extraction on the fixed image via a first feature extractor to generate a fixed image feature map;
performing feature extraction on the moving image via second feature extractor to generate a moving image feature map;
performing cross-modal attention on the fixed image feature map and the moving image feature map to generate cross-modal feature attention data;
performing deep registration on the cross-modal feature attention data via a deep registrator; and
outputting a multi-modal registered image.