US 12,033,081 B2
Systems and methods for virtual and augmented reality
Paul-Edouard Sarlin, Zurich (CH); Daniel DeTone, San Francisco, CA (US); Tomasz Jan Malisiewicz, Mountain View, CA (US); and Andrew Rabinovich, San Francisco, CA (US)
Assigned to Magic Leap, Inc.
Filed by Magic Leap, Inc., Plantation, FL (US)
Filed on Nov. 13, 2020, as Appl. No. 17/098,043.
Claims priority of provisional application 62/935,597, filed on Nov. 14, 2019.
Prior Publication US 2021/0150252 A1, May 20, 2021
Int. Cl. G06N 3/084 (2023.01); G06F 18/22 (2023.01); G06N 3/04 (2023.01); G06N 3/08 (2023.01); G06V 10/75 (2022.01); G06V 10/82 (2022.01); G06V 20/00 (2022.01); G06V 30/196 (2022.01)
CPC G06N 3/084 (2013.01) [G06F 18/22 (2023.01); G06N 3/04 (2013.01); G06N 3/08 (2013.01); G06V 10/757 (2022.01); G06V 10/82 (2022.01); G06V 20/36 (2022.01); G06V 30/1988 (2022.01)] 16 Claims
OG exemplary drawing
 
1. A computer system comprising:
a computer-readable medium;
a processor connected to the computer-readable medium; and
a set of instructions on the computer-readable medium, including:
a deep middle-end matcher architecture that includes:
an attentional graph neural network having:
a keypoint encoder to map keypoint positions p and their visual descriptors d into a single vector; and
alternating self-attention and cross-attention layers that, based on the vector, repeated L times to create representations f; and
an optimal matching layer that creates an M by N score matrix from the representations f and finds an optimal partial assignment based on the M by N score matrix,
where:
p are keypoint positions,
d are descriptors,
L are a plurality of times,
f are representations,
M by N is a score matrix with a length M and a width N.