US 11,989,905 B2
Systems and methods for synchronizing an image sensor
Landis Huffman, Chicago, IL (US); Joshua Michael Finken, Park City, UT (US); Amey Aroskar, North Aurora, IL (US); Sumedh Rasal, Chicago, IL (US); and Sanjay Kumar Boddhu, Aurora, IL (US)
Assigned to HERE GLOBAL B.V., Eindhoven (NL)
Filed by HERE GLOBAL B.V., Eindhoven (NL)
Filed on Apr. 27, 2021, as Appl. No. 17/241,487.
Prior Publication US 2022/0343531 A1, Oct. 27, 2022
Int. Cl. G06T 7/70 (2017.01); G01S 19/14 (2010.01); G06V 20/56 (2022.01); G06V 30/19 (2022.01); G06V 30/24 (2022.01)
CPC G06T 7/70 (2017.01) [G01S 19/14 (2013.01); G06V 20/56 (2022.01); G06V 30/1918 (2022.01); G06V 30/2552 (2022.01); G06T 2207/20048 (2013.01); G06T 2207/30241 (2013.01)] 10 Claims
OG exemplary drawing
 
10. A non-transitory computer-readable storage medium for synchronizing a camera sensor, carrying one or more sequences of one or more instructions which, when executed by one or more processors, cause a system to perform steps to:
receive image data captured using an image sensor that is moving along a pathway;
generate an image sensor trajectory by applying a structure-from-motion algorithm to the image data to generate a plurality of positions forming the image sensor trajectory, wherein the plurality of positions indicates a plurality of estimated positions in which images were captured by the image sensor as the image sensor move along the pathway, and wherein the image sensor trajectory indicates a plurality of timestamps in which the images were captured by the image sensor;
receive position data acquired along the pathway using a position sensor, wherein timestamps for the image data and position data are asynchronous;
assemble a position sensor trajectory using the position data, wherein the position sensor trajectory indicates a plurality of positions in which position measurements were captured by the position sensor as the position sensor move along the pathway and a plurality of timestamps in which the position measurements were captured by the position sensor, and wherein the position sensor trajectory is a Global Navigation Satellite System (GNSS) data acquired along the pathway;
generate a spatial transformation that aligns the image sensor trajectory and position sensor trajectory by applying a coherent point drift algorithm to the image sensor trajectory and position sensor trajectory, wherein the coherent point drift algorithm matches a shape, position, and scale of the image sensor trajectory to the position sensor trajectory; and
synchronize the image sensor using the spatial transformation.