US 12,008,738 B2
Defocus blur removal and depth estimation using dual-pixel image data
Rahul Garg, Sunnyvale, CA (US); Neal Wadhwa, Cambridge, MA (US); Pratul Preeti Srinivasan, San Francisco, CA (US); Tianfan Xue, Sunnyvale, CA (US); Jiawen Chen, San Ramon, CA (US); Shumian Xin, Pittsburgh, PA (US); and Jonathan T. Barron, Alameda, CA (US)
Assigned to Google LLC, Mountain View, CA (US)
Appl. No. 17/626,069
Filed by Google LLC, Mountain View, CA (US)
PCT Filed Nov. 13, 2020, PCT No. PCT/US2020/060517
§ 371(c)(1), (2) Date Jan. 11, 2022,
PCT Pub. No. WO2022/103400, PCT Pub. Date May 19, 2022.
Prior Publication US 2022/0375042 A1, Nov. 24, 2022
Int. Cl. G06T 5/73 (2024.01); G06T 5/50 (2006.01); G06T 7/50 (2017.01)
CPC G06T 5/73 (2024.01) [G06T 5/50 (2013.01); G06T 7/50 (2017.01)] 20 Claims
OG exemplary drawing
 
1. A computer-implemented method comprising:
obtaining dual-pixel image data comprising a first sub-image and a second sub-image;
determining (i) an in-focus image, (ii) a first blur kernel corresponding to the first sub-image, and (iii) a second blur kernel corresponding to the second sub-image;
determining a loss value using a loss function comprising one or more of: an equivalence loss term configured to determine a difference between (i) a convolution of the first sub-image with the second blur kernel and (ii) a convolution of the second sub-image with the first blur kernel, or a data loss term configured to determine a sum of (i) a difference between the first sub-image and a convolution of the in-focus image with the first blur kernel and (ii) a difference between the second sub-image and a convolution of the in-focus image with the second blur kernel;
based on the loss value and the loss function, updating one or more of: (i) the in-focus image, (ii) the first blur kernel, or (iii) the second blur kernel; and
generating image data based on one or more of: (i) the in-focus image as updated, (ii) the first blur kernel as updated, or (iii) the second blur kernel as updated.