US 12,394,076 B2
System and method for facial un-distortion in digital images using multiple imaging sensors
Zeeshan Nadir, Allen, TX (US); Numair Khan, Providence, RI (US); and Hamid Sheikh, Allen, TX (US)
Assigned to Samsung Electronics Co., Ltd., Suwon-si (KR)
Filed by Samsung Electronics Co., Ltd., Suwon-si (KR)
Filed on Jan. 31, 2022, as Appl. No. 17/589,014.
Prior Publication US 2023/0245330 A1, Aug. 3, 2023
Int. Cl. G06T 7/33 (2017.01); G06T 3/147 (2024.01); G06T 5/77 (2024.01); G06T 7/40 (2017.01); G06T 7/50 (2017.01)
CPC G06T 7/337 (2017.01) [G06T 3/147 (2024.01); G06T 5/77 (2024.01); G06T 7/40 (2013.01); G06T 7/50 (2017.01); G06T 2207/20081 (2013.01); G06T 2207/20084 (2013.01); G06T 2207/30201 (2013.01)] 24 Claims
OG exemplary drawing
 
1. A method comprising:
aligning landmark points between multiple distorted images to generate multiple aligned images, wherein the multiple distorted images exhibit perspective distortion in at least one face appearing in the multiple distorted images, and wherein the landmark points are aligned to correct baseline disparities between the multiple distorted images caused by differences in at least one of: sensor sensitivities, calibrations, focal lengths, and baseline distance between imaging sensors;
predicting a depth map using a disparity estimation neural network that receives the multiple aligned images as input;
generating a warp field using a selected one of the multiple aligned images;
performing a two-dimensional (2D) image projection on the selected aligned image using the depth map and the warp field to generate an undistorted image, wherein the undistorted image includes one or more missing pixels as a result of the 2D image projection; and
filling in the one or more missing pixels in the undistorted image using an inpainting neural network to generate a final undistorted image.