US 12,288,348 B2
Depth estimation based on ego-motion estimation and residual flow estimation
Jiexiong Tang, Stockholm (SE); Rares Andrei Ambrus, San Francisco, CA (US); Vitor Guizilini, Santa Clara, CA (US); and Adrien David Gaidon, Mountain View, CA (US)
Assigned to TOYOTA RESEARCH INSTITUTE, INC., Los Altos, CA (US)
Filed by TOYOTA RESEARCH INSTITUTE, INC., Los Altos, CA (US)
Filed on Jun. 29, 2023, as Appl. No. 18/344,700.
Application 18/344,700 is a continuation of application No. 17/230,941, filed on Apr. 14, 2021, granted, now 11,727,588.
Claims priority of provisional application 63/009,935, filed on Apr. 14, 2020.
Prior Publication US 2023/0342960 A1, Oct. 26, 2023
Int. Cl. G06T 7/55 (2017.01); G06T 3/18 (2024.01); G06T 3/4007 (2024.01); G06T 3/4046 (2024.01); G06T 7/70 (2017.01); G06T 19/00 (2011.01); G06V 20/56 (2022.01)
CPC G06T 7/55 (2017.01) [G06T 3/18 (2024.01); G06T 3/4007 (2013.01); G06T 3/4046 (2013.01); G06T 7/70 (2017.01); G06T 19/00 (2013.01); G06V 20/56 (2022.01); G06T 2207/30244 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A method for depth estimation performed by a depth estimation system associated with an agent, comprising:
determining a first depth of a first image and a second depth of a second image, the first image and the second image being captured by a sensor associated with the agent;
generating a first three-dimensional (3D) image of the first image based on the first depth, a first pose associated with the sensor, and the second image;
generating a warped depth image based on transforming the first depth in accordance with the first pose;
determining a residual pose estimate between the warped depth image and the second depth, the residual pose estimate being parameterized by a six-degree of freedom (DoF) transformation;
updating the first pose based on the residual pose estimate;
updating the first 3D image in accordance with the updated first pose; and
controlling an action of the agent in accordance with the updated first 3D image.