US 12,411,023 B2
Method and system for video-based positioning and mapping utilizing a pixel wise segmentation on an image of a detected object
Tinosch Kolagheichi-Ganjineh, Berlin (DE); Philipp Holzschneider, Berlin (DE); Dimitri Schachmann, Berlin (DE); Sergej Mann, Berlin (DE); Sebastian Ihlefeld, Berlin (DE); Michael Hofmann, Amsterdam (NL); Olaf Booij, Leiden (NL); and Nicolau Leal Werneck, Amsterdam (NL)
Assigned to TomTom Global Content B.V., Amsterdam (NL)
Filed by TomTom Global Content B.V., Amsterdam (NL)
Filed on Dec. 3, 2023, as Appl. No. 18/527,317.
Application 18/527,317 is a continuation of application No. 18/088,710, filed on Dec. 26, 2022, granted, now 11,835,358.
Application 18/088,710 is a continuation of application No. 16/467,977, granted, now 11,761,790, previously published as PCT/EP2017/082293, filed on Dec. 11, 2017.
Claims priority of application No. 16203258 (EP), filed on Dec. 9, 2016; and application No. 1714381 (GB), filed on Sep. 7, 2017.
Prior Publication US 2024/0167844 A1, May 23, 2024
This patent is subject to a terminal disclaimer.
Int. Cl. G06C 21/00 (2006.01); G01C 21/00 (2006.01); G01C 21/32 (2006.01); G06F 16/29 (2019.01); G06F 16/587 (2019.01); G06T 7/11 (2017.01); G06T 7/73 (2017.01); G06V 20/56 (2022.01); G06V 20/58 (2022.01)
CPC G01C 21/3848 (2020.08) [G01C 21/32 (2013.01); G01C 21/3811 (2020.08); G06F 16/29 (2019.01); G06F 16/587 (2019.01); G06T 7/11 (2017.01); G06T 7/74 (2017.01); G06V 20/582 (2022.01); G06V 20/588 (2022.01); G06T 2207/10016 (2013.01); G06T 2207/30244 (2013.01); G06T 2207/30256 (2013.01)] 16 Claims
OG exemplary drawing
 
1. A method, comprising:
obtaining, from at least one camera associated with a vehicle traveling on a road, a sequence of images of an environment of the road, each image being associated with a location where that image was obtained;
generating a local map representation of an area of the road using at least some images from the sequence of images and the locations associated therewith, the generating including:
processing the at least some of the images to detect an object in the environment of the road, the processing including, for each image of the at least some of the images:
performing a pixel wise segmentation on the image, the pixel wise segmentation resulting in each pixel being allocated an object class or object class vector indicating a probability of each object class for that pixel; and
processing the image to detect the object based at least in part on the object classes or object class vectors;
determining at least one transformation for mapping the object between the at least some of the images, the determining including determining a change in position and/or rotation for the object between sequential images based on a respective location of the at least one camera where each of the images was captured; and
based on the at least one transformation and the locations associated with the at least some of the images, generating a two- and/or three-dimensional representation for the object relative to the area of the road;
comparing the local map representation with some or all of a reference map to identify a corresponding section of the reference map; and
selectively updating the corresponding section of the reference map based on the local map representation.