US 11,775,058 B2
Method and system for performing eye tracking using an off-axis camera
Vijay Badrinarayanan, Mountain View, CA (US); Zhengyang Wu, Bellevue, WA (US); Srivignesh Rajendran, San Francisco, CA (US); and Andrew Rabinovich, San Francisco, CA (US)
Assigned to Magic Leap, Inc., Plantation, FL (US)
Filed by Magic Leap, Inc., Plantation, FL (US)
Filed on Dec. 21, 2020, as Appl. No. 17/129,669.
Application 17/129,669 is a continuation of application No. PCT/US2019/038693, filed on Jun. 24, 2019.
Claims priority of provisional application 62/688,983, filed on Jun. 22, 2018.
Prior Publication US 2021/0182554 A1, Jun. 17, 2021
Int. Cl. G06T 7/00 (2017.01); G06F 3/01 (2006.01); G06T 7/11 (2017.01); G06N 3/08 (2023.01); G06V 40/19 (2022.01); G06V 10/764 (2022.01); G06V 10/82 (2022.01); G06V 40/18 (2022.01)
CPC G06F 3/013 (2013.01) [G06N 3/08 (2013.01); G06T 7/0012 (2013.01); G06T 7/11 (2017.01); G06V 10/764 (2022.01); G06V 10/82 (2022.01); G06V 40/18 (2022.01); G06V 40/19 (2022.01); G06T 2207/20081 (2013.01); G06T 2207/20084 (2013.01); G06T 2207/30041 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A method of estimating a gaze vector for a user of an optical device, the method comprising:
receiving, from a camera mounted to the optical device, an input image of an eye of the user;
providing the input image of the eye of the user to a neural network having a plurality of layers, the neural network having been previously trained using a plurality of training input images;
generating, using the neural network, network output data that includes two-dimensional (2D) pupil data and cornea center data, wherein a first subset of the plurality of layers are used to generate the 2D pupil data and a second subset of the plurality of layers are used to generate the cornea center data, wherein the first subset and the second subset share at least one common layer, and wherein each of the first subset and the second subset includes at least one unique layer; and
computing, based on the network output data, the gaze vector corresponding to the eye of the user.