US 11,682,140 B1
Methods and apparatus for calibrating stereo cameras using a time-of-flight sensor
Timothy P. Daly, Jr., San Jose, CA (US)
Assigned to PlusAI, Inc., Cupertino, CA (US)
Filed by PlusAI, Inc., Cupertino, CA (US)
Filed on Jun. 30, 2022, as Appl. No. 17/855,512.
Int. Cl. G06T 7/80 (2017.01)
CPC G06T 7/85 (2017.01) [G06T 2207/10016 (2013.01); G06T 2207/30252 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A system, comprising:
a stereo camera system onboard a vehicle;
a Time of Flight (ToF) sensor onboard the vehicle;
at least one processor; and
a memory operably coupled to the at least one processor and storing instructions to cause the system to perform operations comprising:
receiving, from the stereo camera system, stereo image data depicting a scene, the scene including a stationary object, the stereo image data depicting the scene captured by the stereo camera system at a first time;
receiving sensor data from the ToF sensor;
projecting the sensor data into a representation of the scene, to generate a plurality of ToF distance measurements;
identifying a plurality of ToF disparities based on the ToF distance measurements;
selecting a subset of ToF disparities from the plurality of ToF disparities based on proximities of the plurality of ToF disparities to an expected disparity value from a non-zero set of expected disparity values;
determining a distance between the stationary object and the vehicle, the distance being associated with the first time; and
applying a correction to the stereo image data by:
when the distance between the stationary object and the vehicle is below a predefined threshold, applying a first correction to the stereo image data based on a stereo disparity calculated based on the stereo image data, and
when the distance between the stationary object and the vehicle is not below the predefined threshold, applying a second correction to the stereo image data based on the subset of ToF disparities.