US 12,007,564 B2
Methods and apparatuses for corner detection using neural network and corner detector
Anush Mohan, San Jose, CA (US); Guy Rosenthal, Tel Aviv (IL); Adi Perry, Tel Aviv (IL); and Gilboa Sheveki, Shoham (IL)
Assigned to Magic Leap, Inc., Plantation, FL (US)
Filed by MAGIC LEAP, INC., Plantation, FL (US)
Filed on May 11, 2023, as Appl. No. 18/315,766.
Application 18/315,766 is a continuation of application No. 16/875,499, filed on May 15, 2020, granted, now 11,686,941.
Claims priority of provisional application 62/849,386, filed on May 17, 2019.
Prior Publication US 2023/0288703 A1, Sep. 14, 2023
Int. Cl. G06T 7/13 (2017.01); G02B 27/01 (2006.01); G06F 18/214 (2023.01); G06N 3/08 (2023.01); G06T 3/18 (2024.01); G06V 10/25 (2022.01); G06V 10/44 (2022.01); G06V 10/764 (2022.01); G06V 10/774 (2022.01); G06V 10/82 (2022.01); G06V 20/20 (2022.01)
CPC G02B 27/017 (2013.01) [G06F 18/214 (2023.01); G06N 3/08 (2013.01); G06T 3/18 (2024.01); G06T 7/13 (2017.01); G06V 10/25 (2022.01); G06V 10/44 (2022.01); G06V 10/454 (2022.01); G06V 10/764 (2022.01); G06V 10/7753 (2022.01); G06V 10/82 (2022.01); G06V 20/20 (2022.01); G02B 2027/0138 (2013.01); G02B 2027/014 (2013.01)] 19 Claims
OG exemplary drawing
 
1. A method of training and using a neural network for image interest point detection, comprises:
generating a reference dataset comprising a plurality of reference sets, wherein each of the plurality of reference sets comprises:
an image, and
a set of reference interest points corresponding to the image; and
for each reference set of the plurality of reference sets:
generating a warped image by applying a homograph to the image,
generating a warped set of reference interest points by applying the homograph to the set of reference interest points,
calculating, by the neural network receiving the image as input, a set of calculated interest points and a calculated descriptor,
calculating, by the neural network receiving the warped image as input, a set of calculated warped interest points and a calculated warped descriptor,
calculating a loss based on the set of calculated interest points, the calculated descriptor, the set of calculated warped interest points, the calculated warped descriptor, the set of reference interest points, the warped set of reference interest points, and the homograph, and
modifying the neural network based on the loss; wherein the method further comprises: obtaining locations of features in an input image, wherein the locations of the features are identified by the neural network; determining a region of interest for one of the features in the input image, the region of interest having a size that is less than a size of the input image; and performing a corner detection using a corner detection algorithm to identify a corner in the region of interest.