US 12,333,744 B2
Scale-aware self-supervised monocular depth with sparse radar supervision
Vitor Guizilini, Santa Clara, CA (US); and Charles Christopher Ochoa, San Francisco, CA (US)
Assigned to TOYOTA RESEARCH INSTITUTE, INC., Los Altos, CA (US)
Filed by TOYOTA RESEARCH INSTITUTE, INC., Los Altos, CA (US)
Filed on Jan. 19, 2022, as Appl. No. 17/578,830.
Prior Publication US 2023/0230264 A1, Jul. 20, 2023
Int. Cl. G06T 7/50 (2017.01); G01B 15/00 (2006.01); G01S 13/89 (2006.01); G06T 3/40 (2024.01)
CPC G06T 7/50 (2017.01) [G01B 15/00 (2013.01); G01S 13/89 (2013.01); G06T 3/40 (2013.01); G06T 2207/10028 (2013.01); G06T 2207/20081 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A method for depth estimation from monocular images, comprising:
receiving an image captured by an image sensor, the image comprising pixels representing a scene of an environment;
deriving a depth map for the image based on a depth model, the depth map comprising a plurality of predicted depth values for a plurality of the pixels of the image;
estimating a first scale comprising a pixel distance between depth values for the image based the plurality of predicted depth values;
receiving depth data captured by a range sensor, the depth data comprising a point cloud representing the scene of the environment, the point cloud comprising depth measures for a plurality of points of the point cloud;
determining a second scale comprising a measure of central tendency for the first scale based on the depth measures, wherein the measure of central tendency, for the first scale, comprises a measure of a central value for probability distribution of the first scale based on the depth measure;
determining a scale factor based on a ratio between the second scale and the first scale; and
updating the depth model based on the scale factor, wherein the depth model generates metrically accurate depth estimates based on the scale factor.