US 12,354,295 B2
Eye tracking latency enhancements
Bradley Vincent Stuart, Fort Lauderdale, FL (US); Daniel Farmer, Verdi, NV (US); Tiejian Zhang, Santa Clara, CA (US); Shiuli Das, Milpitas, CA (US); Suraj Manjunath Shanbhag, Santa Clara, CA (US); and Erik Fonseka, Biel (CH)
Assigned to MAGIC LEAP, INC., Plantation, FL (US)
Appl. No. 17/597,026
Filed by Magic Leap, Inc., Plantation, FL (US)
PCT Filed Jul. 1, 2020, PCT No. PCT/US2020/040549
§ 371(c)(1), (2) Date Dec. 22, 2021,
PCT Pub. No. WO2021/007087, PCT Pub. Date Jan. 14, 2021.
Claims priority of provisional application 62/945,562, filed on Dec. 9, 2019.
Claims priority of provisional application 62/871,009, filed on Jul. 5, 2019.
Prior Publication US 2022/0301217 A1, Sep. 22, 2022
Int. Cl. G06T 7/70 (2017.01); G06T 7/11 (2017.01); G06T 7/13 (2017.01); G06T 7/60 (2017.01); G06V 10/22 (2022.01); G06V 10/766 (2022.01); G06V 40/18 (2022.01)
CPC G06T 7/70 (2017.01) [G06T 7/11 (2017.01); G06T 7/13 (2017.01); G06T 7/60 (2013.01); G06V 10/22 (2022.01); G06V 10/766 (2022.01); G06V 40/193 (2022.01); G06V 40/197 (2022.01); G06T 2207/10052 (2013.01); G06T 2207/20081 (2013.01); G06T 2207/20084 (2013.01); G06T 2207/30201 (2013.01)] 15 Claims
OG exemplary drawing
 
1. A head-mounted system comprising:
a camera configured to capture images of an eye of a user;
one or more light source configured to illuminate the eye of the user, such that glints are represented in images of the eye of user; and
one or more processors configured to:
obtain a first image of the eye of the user from the camera;
provide the first image as input to a machine learning model which has been trained to generate iris segmentation data and pupil segmentation data given an image of an eye;
obtain a second image of the eye of the user from the camera subsequent to the first image;
detect, based on the iris segmentation data, a set of one or more locations in the second image at which one or more glints are represented;
identify, based on the pupil segmentation data, a region of the second image at which the pupil of the eye of the user is represented;
determine a pose of the eye of the user based at least in part on the detected set of one or more glint locations in the second image and the identified region of the second image;
obtain, via the machine learning model, iris segmentation data and pupil segmentation data for the second image;
obtain a third image of the eye of the user from the camera;
detect a set of one or more locations in the third image at which one or more glints are shown, respectively, based on the iris segmentation data generated by the machine learning model for the second image;
identify a region of the third image at which the pupil of the eye of the user is represented based on the pupil segmentation data generated by the machine learning model for the second image; and
determine a pose of the eye of the user based at least in part on the detected set of one or more glint locations in the third image and the identified region of the third image.