US 12,333,750 B2
Systems and methods for generic visual odometry using learned features via neural camera models
Vitor Guizilini, Santa Clara, CA (US); Igor Vasiljevic, Los Altos, CA (US); Rares A. Ambrus, San Francisco, CA (US); Sudeep Pillai, Los Altos, CA (US); and Adrien Gaidon, Los Altos, CA (US)
Assigned to TOYOTA RESEARCH INSTITUTE, INC., Los Altos, CA (US)
Filed by TOYOTA RESEARCH INSTITUTE, INC., Los Altos, CA (US)
Filed on Apr. 17, 2022, as Appl. No. 17/722,360.
Application 17/722,360 is a continuation of application No. 17/021,968, filed on Sep. 15, 2020, granted, now 11,508,080.
Prior Publication US 2022/0245843 A1, Aug. 4, 2022
This patent is subject to a terminal disclaimer.
Int. Cl. G06T 7/55 (2017.01); G05D 1/00 (2024.01); G06T 3/18 (2024.01); G06T 7/33 (2017.01); G06T 7/80 (2017.01)
CPC G06T 7/55 (2017.01) [G06T 3/18 (2024.01); G06T 7/33 (2017.01); G06T 7/80 (2017.01); G05D 1/0088 (2013.01); G05D 1/0214 (2013.01); G05D 1/0223 (2013.01); G05D 1/0251 (2013.01); G06T 2207/20081 (2013.01); G06T 2207/30244 (2013.01); G06T 2207/30252 (2013.01)] 7 Claims
OG exemplary drawing
 
1. A method comprising:
estimating correspondences between keypoints of a target camera image and keypoints of a context camera image, wherein the target camera image and the context camera image are obtained from a monocular sequence;
using a ray surface decoder to predict a ray surface from the target image, wherein the predicted ray surface associates a respective pixel in the target image with a corresponding direction;
based on the keypoint correspondences and the predicted ray surface, lifting a set of 2D keypoints to 3D, using a neural camera model; and
projecting the 3D keypoints into the context camera image using the neural camera model.