US 12,392,628 B2
Localization based on multi-collect fusion
Luca Del Pero, London (GB); and Karim Tarek Mahmoud Elsayed Ahmed Shaban, London (GB)
Assigned to Lyft, Inc., San Francisco, CA (US)
Filed by Lyft, Inc., San Francisco, CA (US)
Filed on Jun. 30, 2020, as Appl. No. 16/917,803.
Prior Publication US 2021/0404834 A1, Dec. 30, 2021
Int. Cl. G01C 21/36 (2006.01); G01C 21/32 (2006.01); G06F 16/29 (2019.01); G06V 20/56 (2022.01)
CPC G01C 21/3635 (2013.01) [G01C 21/32 (2013.01); G01C 21/3614 (2013.01); G06F 16/29 (2019.01); G06V 20/56 (2022.01)] 20 Claims
OG exemplary drawing
 
17. A computing system comprising:
at least one processor;
a non-transitory computer-readable medium; and
program instructions stored on the non-transitory computer-readable medium that are executable by the at least one processor such that the computing system is configured to perform functions comprising:
generating a local map portion of a geographical environment based on sensor data captured by a device, wherein the sensor data comprises digital image data and the local map portion comprises local map structure data generated by processing the captured digital image data using a technique for determining structure from a first series of images within the captured digital image data, and wherein the local map structure data comprises (i) a first three-dimensional (3D) representation of the geographical environment that is generated during the processing of the captured digital image data using the technique for determining structure from the first series of images and (ii) first sequential pose data associated with the first series of images that is generated during the processing of the captured digital image data using the technique for determining structure from the first series of images;
accessing existing map structure data of an existing map, wherein the existing map structure data is aligned to a global coordinate system and is predetermined from map structure data that was previously generated by processing previously-captured digital image data using a technique for determining structure from a second series of images within the previously-captured digital image data, and wherein the existing map structure data comprises (i) a second 3D representation of the geographic environment that is generated during the processing of the previously-captured digital image data using the technique for determining structure from the second series of images and (ii) second sequential pose data associated with the second series of images that is generated during the processing of the previously-captured digital image data using the technique for determining structure from the second series of images;
identifying pose correlations between (i) poses included in the first sequential pose data that is generated during the processing of the captured digital image data using the technique for determining structure from the first series of images and (ii) poses included in the second sequential pose data that is generated during the processing of the previously-captured digital image data using the technique for determining structure from the second series of images;
identifying feature correlations between (i) visible features included in the first 3D representation environment that is generated during the processing of the captured digital image data using the technique for determining structure from the first series of images and (ii) visible features included in the second 3D representation that is generated during the processing of the previously-captured digital image data using the technique for determining structure from the second series of images;
determining a transformation of the local map portion relative to the existing map based on the identified pose correlations and the identified feature correlations; and
determining a localization of the device within the global coordinate system by using the determined transformation to align the local map portion with the existing map.