US 11,810,313 B2
Real-time stereo matching using a hierarchical iterative refinement network
Vladimir Tankovich, Seattle, WA (US); Christian Haene, Berkeley, CA (US); Sean Ryan Francesco Fanello, San Francisco, CA (US); Yinda Zhang, Daly City, CA (US); Shahram Izadi, Tiburon, CA (US); Sofien Bouaziz, Los Gatos, CA (US); Adarsh Prakash Murthy Kowdle, San Francisco, CA (US); and Sameh Khamis, Oakland, CA (US)
Assigned to GOOGLE LLC, Mountain View, CA (US)
Filed by GOOGLE LLC, Mountain View, CA (US)
Filed on Feb. 19, 2021, as Appl. No. 17/249,095.
Claims priority of provisional application 62/979,928, filed on Feb. 21, 2020.
Prior Publication US 2021/0264632 A1, Aug. 26, 2021
Int. Cl. G06T 7/593 (2017.01); H04N 13/20 (2018.01); G06T 3/00 (2006.01); G06T 3/40 (2006.01); G06T 5/30 (2006.01); H04N 13/00 (2018.01)
CPC G06T 7/593 (2017.01) [G06T 3/0093 (2013.01); G06T 3/40 (2013.01); G06T 5/30 (2013.01); H04N 13/20 (2018.05); G06T 2207/20016 (2013.01); G06T 2207/20084 (2013.01); H04N 2013/0081 (2013.01)] 21 Claims
OG exemplary drawing
 
1. A real-time active stereo system comprising:
a capture system configured to capture stereo data, the stereo data including a first input image and a second input image; and
a depth sensing computing system configured to predict a depth map, the depth sensing computing system including:
a feature extractor configured to extract features from the first and second input images at a plurality of resolutions;
an initialization engine configured to generate a plurality of depth estimations, each of the plurality of depth estimations corresponding to a different resolution and including a three-dimensional (3D) slanted plane hypothesis for a region of a respective depth estimation, the 3D slanted plane hypothesis including a disparity value and a location of a slanted plane; and
a propagation engine configured to iteratively refine the plurality of depth estimations based on image warping and spatial propagation.