US 12,366,459 B2
Localization processing service and observed scene reconstruction service
Brian Streem, Jackson Heights, NY (US); Ricardo Achilles Filho, Bauru (BR); Vijaykumar Sureshkumar, Jersey City, NJ (US); Marcelo Guimarães, Bauru (BR); Marcelo Eduardo Benencase, Cascavel (BR); Andresso da Silva, Campo Alegre (BR); Guilherme Soares Silvestre, Matão (BR); Gustavo Henrique Benso, Reginópolis (BR); Juan Carlos Conceição de Lima Sales, Campo Grande (BR); Natan Henrique Sanches, São Carlos (BR); Saulo Gabriel Felix, São Cristóvão (BR); Pedro Luiz Cason Caldato, Bauru (BR); Gustavo Henrique Stahl, Limeira (BR); and Igor Kenzo Ishikawa Oshiro Nakashima, Campinas (BR)
Assigned to AEROCINE VENTURES, INC., Jackson Heights, NY (US)
Filed by AEROCINE VENTURES, INC., Jackson Heights, NY (US)
Filed on Jun. 9, 2023, as Appl. No. 18/208,257.
Claims priority of provisional application 63/350,856, filed on Jun. 9, 2022.
Claims priority of provisional application 63/422,564, filed on Nov. 4, 2022.
Prior Publication US 2023/0400327 A1, Dec. 14, 2023
Int. Cl. G01C 21/00 (2006.01); G01C 21/30 (2006.01); G06T 7/70 (2017.01); G06T 7/73 (2017.01); G06T 17/05 (2011.01); G06V 10/56 (2022.01); G06V 10/74 (2022.01); G06V 10/764 (2022.01); G06V 10/771 (2022.01); G06V 20/62 (2022.01)
CPC G01C 21/3804 (2020.08) [G01C 21/30 (2013.01); G06T 7/70 (2017.01); G06T 7/74 (2017.01); G06T 17/05 (2013.01); G06V 10/56 (2022.01); G06V 10/761 (2022.01); G06V 10/764 (2022.01); G06V 10/771 (2022.01); G06V 20/625 (2022.01); G06T 2207/20084 (2013.01)] 27 Claims
OG exemplary drawing
 
1. A method of localizing an object in an environment using an observing subsystem comprising an image sensor component, a memory component, and a processing module communicatively coupled to the image sensor component and the memory component, the method comprising:
storing, with the memory component, a map feature database comprising a plurality of map feature entries, wherein:
each map feature entry of the plurality of map feature entries is respectively associated with a rendered map image of a plurality of rendered map images rendered from a georeferenced three-dimensional map; and
each rendered map image of the plurality of rendered map images is associated with a respective map location and a respective map orientation;
capturing, at a moment in time with the image sensor component, an observing image of the object;
determining, with the processing module, an observing orientation of the image sensor component at the moment in time;
determining, with the processing module, an observing location of the image sensor component at the moment in time;
defining, with the processing module, a similar image set, wherein:
the similar image set comprises the captured observing image and a particular rendered map image of the plurality of rendered map images; and
the defining comprises determining that:
the observing orientation of the captured observing image satisfies an orientation similarity comparison with the map orientation of the particular rendered map image; and
the observing location of the captured observing image satisfies a location similarity comparison with the map location of the particular rendered map image;
extracting, with a feature extractor model of the processing module, the following:
an image feature map from the captured observing image of the similar image set; and
a map feature map from the particular rendered map image of the similar image set;
performing, with a correlation module of the processing module, a tensor correlation between the extracted image feature map and the extracted map feature map to determine the position of the object on the particular rendered map image; and
based on the determined position of the object on the particular rendered map image, determining, with a raycasting module of the processing module, georeferenced three-dimensional coordinates of the object in the environment.