US 12,482,059 B2
Method and apparatus for generating panoramic image based on deep learning network
Hee-Kyung Lee, Daejeon (KR); Gi-Mun Um, Seoul (KR); Dae-Young Song, Daejeon (KR); and Dong-Hyeon Cho, Daejeon (KR)
Assigned to ELECTRONICS AND TELECOMMUNICATIONS RESEARCH INSTITUTE, Daejeon (KR); and THE INDUSTRY & ACADEMIC COOPERATION IN CHUNGNAM NATIONAL UNIVERSITY (IAC), Daejeon (KR)
Filed by ELECTRONICS AND TELECOMMUNICATIONS RESEARCH INSTITUTE, Daejeon (KR); and THE INDUSTRY & ACADEMIC COOPERATION IN CHUNGNAM NATIONAL UNIVERSITY (IAC), Daejeon (KR)
Filed on Sep. 23, 2022, as Appl. No. 17/951,899.
Claims priority of application No. 10-2021-0126502 (KR), filed on Sep. 24, 2021; and application No. 10-2022-0065357 (KR), filed on May 27, 2022.
Prior Publication US 2023/0098276 A1, Mar. 30, 2023
Int. Cl. G06T 3/4038 (2024.01); G06T 3/18 (2024.01); G06T 5/50 (2006.01)
CPC G06T 3/4038 (2013.01) [G06T 3/18 (2024.01); G06T 5/50 (2013.01); G06T 2207/10024 (2013.01); G06T 2207/20081 (2013.01); G06T 2207/20084 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A method for generating a panoramic image, comprising:
configuring a data set for training a deep learning network based on a number of images, K, simultaneously acquired from multiple cameras;
extracting an encoding feature map, a skip connection feature map, and a decoding feature map based on K/2 images, among the K images;
estimating multiple homographies based on the encoding feature map;
estimating a flow adjustment map, a preprocessing blending map, a weight map, and a post-processing blending map based on the decoding feature map;
deriving a flow map based on the multiple homographies and the flow adjustment map;
generating a preprocessed image based on the preprocessing blending map;
generating multiple warped images based on the preprocessed image and the flow map;
generating a matching image based on the multiple warped images and the weight map; and
correcting the matching image based on the matching image and the post-processing blending map, and then generating an output of the panoramic image.