US 11,699,247 B2
System and method for runtime determination of camera miscalibration
Xiangyun Ye, Framingham, MA (US); David Y. Li, West Roxbury, MA (US); Guruprasad Shivaram, Chestnut Hill, MA (US); and David J. Michael, Wayland, MA (US)
Assigned to Cognex Corporation, Natick, MA (US)
Filed by Xiangyun Ye, Framingham, MA (US); David Y. Li, West Roxbury, MA (US); Guruprasad Shivaram, Chestnut Hill, MA (US); and David J. Michael, Wayland, MA (US)
Filed on Dec. 24, 2009, as Appl. No. 12/647,192.
Prior Publication US 2011/0157373 A1, Jun. 30, 2011
Int. Cl. G06T 7/80 (2017.01)
CPC G06T 7/85 (2017.01) [G06T 2207/10012 (2013.01); G06T 2207/30108 (2013.01)] 18 Claims
OG exemplary drawing
 
1. A method for determining camera miscalibration in a system of at least three cameras, comprising the steps of:
a) during a first calibration process and prior to a runtime vision inspection process, calibrating the at least three cameras, including estimating respective intrinsic calibration parameters and extrinsic calibration parameters for each of the at least three cameras and determining respective error statistics for each of the at least three cameras, wherein each of the at least three cameras is located at a different orientation with respect to a calibration object in a three-dimensional space;
b) during the runtime vision inspection process in which an inspection object is aligned or inspected based upon a training model, the intrinsic calibration parameters, the extrinsic calibration parameters, and the error statistics, wherein the inspection object is different from the calibration object, finding a first feature on the inspection object in the three-dimensional space with a first plurality of the at least three cameras and determining a first estimated location of the first feature;
c) during the runtime vision inspection process, comparing the first estimated location with a first predicted location from the training model to generate a first alignment score to be used in alignment or inspection of the inspection object;
d) during the runtime vision inspection process, finding a second feature on the inspection object in the three-dimensional space with a second plurality of the at least three cameras and determining a second estimated location of the second feature;
e) during the runtime vision inspection process, comparing the second estimated location with a second predicted location from the training model to generate a second alignment score to be used in alignment or inspection of the inspection object;
f) during the runtime vision inspection process, comparing the first alignment score and the second alignment score with respect to at least one of (i) an accuracy determined during step (a), or (ii) a desired system accuracy;
g) in response to step (f) and during the runtime vision inspection process, determining that a second calibration process is desired based upon a result of the step of comparing exceeding at least one of the (i) accuracy determined during step (a) and (ii) the desired system accuracy;
h) in response to step (g), ceasing the runtime vision inspection process; and
i) in response to step (h), performing the second calibration process to recalibrate at least one of the at least three cameras, wherein the second calibration process is performed with respect to a different object than the runtime vision inspection process, and wherein the second calibration process comprises estimating the respective intrinsic calibration parameters and the extrinsic calibration parameters for at least one of the at least three cameras and redetermining the respective error statistics for at least one of the at least three cameras.