US 12,272,094 B2
Visual camera re-localization using graph neural networks and relative pose supervision
Mehmet Özgür Türkoǧlu, Zurich (CH); Aron Monszpart, London (GB); Eric Brachmann, Hanover (DE); and Gabriel J. Brostow, London (GB)
Assigned to Niantic, Inc., San Francisco, CA (US)
Filed by Niantic, Inc., San Francisco, CA (US)
Filed on Dec. 9, 2021, as Appl. No. 17/546,375.
Claims priority of provisional application 63/123,474, filed on Dec. 10, 2020.
Prior Publication US 2022/0189060 A1, Jun. 16, 2022
Int. Cl. G06T 7/73 (2017.01); G06T 7/11 (2017.01); G06T 7/13 (2017.01); G06V 10/77 (2022.01)
CPC G06T 7/74 (2017.01) [G06T 7/11 (2017.01); G06T 7/13 (2017.01); G06V 10/7715 (2022.01)] 18 Claims
OG exemplary drawing
 
1. A computer-implemented method for generating virtual content for display by a client device, the method comprising:
receiving an image of a scene captured by a camera of the client device;
inputting the image of the scene into a re-localization model, wherein the re-localization model performs steps comprising:
encoding the image into a feature map;
retrieving reference images from an image database based on the feature map of the image;
building a graph based on the image and the reference images comprising:
feature vector nodes generated from the feature map that represent the image and the reference images, and
edges between the feature vector nodes; and
predicting a pose of the image based on the graph;
receiving, from the re-localization model, the predicted pose for the image of the scene; and
generating, using the predicted pose, virtual content for display by the client device in conjunction with the scene.