US 12,345,806 B2
Extrinsic parameter calibration for 4D millimeter-wave radar and camera based on adaptive projection error
Sicong Lyu, Shanghai (CN); and Liangliang Wang, Shanghai (CN)
Assigned to Ambarella International LP, Santa Clara, CA (US)
Filed by Ambarella International LP, Santa Clara, CA (US)
Filed on Jul. 25, 2022, as Appl. No. 17/872,895.
Prior Publication US 2024/0027604 A1, Jan. 25, 2024
Int. Cl. G01S 13/86 (2006.01); G01S 7/40 (2006.01); G01S 13/931 (2020.01); G06N 3/08 (2023.01); G06T 7/70 (2017.01); G06T 7/80 (2017.01)
CPC G01S 13/867 (2013.01) [G01S 7/40 (2013.01); G01S 13/931 (2013.01); G01S 7/4082 (2021.05); G06N 3/08 (2013.01); G06T 7/70 (2017.01); G06T 7/80 (2017.01); G06T 2207/10016 (2013.01); G06T 2207/10044 (2013.01); G06T 2207/20081 (2013.01); G06T 2207/20084 (2013.01); G06T 2207/20104 (2013.01)] 20 Claims
OG exemplary drawing
 
1. An apparatus comprising:
an interface configured to receive (i) pixel data representing a first field of view of a camera and (ii) four-dimensional (4D) point cloud data representing a second field of view of a 4D imaging radar sensor; and
a processor configured to (i) process the pixel data arranged as video frames and the 4D point cloud data, (ii) generate a plurality of center points in response to detecting a radar corner reflector in each of said video frames, (iii) generate a plurality of detection coordinates of said radar corner reflector in said 4D point cloud data, and (iv) calibrate extrinsic parameters for the 4D imaging radar sensor and the camera in response to a projection error based on pairs of said plurality of center points and said plurality of detection coordinates, wherein
(a) said first field of view of said camera and said second field of view of said 4D imaging radar sensor overlap,
(b) said pairs of said plurality of center points and said plurality of detection coordinates are determined in response to positioning said radar corner reflector at different locations in said overlap of said first field of view of the camera and the second field of view of the 4D imaging radar sensor between a near-distance value and a long-distance value, and
(c) a scale of said projection error of said pairs are modified based on a size of the radar corner reflector in said video frames captured at said long-distance value.