US 11,861,062 B2
Blink-based calibration of an optical see-through head-mounted display
Ehsan Azimi, Baltimore, MD (US); Long Qian, Baltimore, MD (US); Peter Kazanzides, Lutherville, MD (US); and Nassir Navab, Fairfax, VA (US)
Assigned to The Johns Hopkins University, Baltimore, MD (US)
Appl. No. 15/733,453
Filed by The Johns Hopkins University, Baltimore, MD (US)
PCT Filed Jan. 31, 2019, PCT No. PCT/US2019/016000
§ 371(c)(1), (2) Date Jul. 31, 2020,
PCT Pub. No. WO2019/152619, PCT Pub. Date Aug. 8, 2019.
Claims priority of provisional application 62/625,978, filed on Feb. 3, 2018.
Prior Publication US 2020/0363867 A1, Nov. 19, 2020
Int. Cl. G09G 5/00 (2006.01); G06F 3/01 (2006.01); H04N 13/327 (2018.01); H04N 13/332 (2018.01); H04N 13/366 (2018.01); G02B 27/01 (2006.01)
CPC G06F 3/013 (2013.01) [G02B 27/0179 (2013.01); H04N 13/327 (2018.05); H04N 13/332 (2018.05); H04N 13/366 (2018.05); G02B 2027/0187 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A method, comprising:
receiving, at a calibration device, information from a positional tracking device that relates to a position of at least one point on a three-dimensional real-world object;
causing, by the calibration device, an optical see-through head-mounted display device to display a virtual image having at least one feature in a display space of the optical see-through head-mounted display device;
receiving, at the calibration device, information from an eye tracking device indicating that a user wearing the optical see-through head-mounted display device performed a voluntary eye blink,
wherein the voluntary eye blink is a calibration input to indicate that the at least one feature of the virtual image appears to the user to be aligned with the at least one point on the three-dimensional real-world object in the display space of the optical see-through head-mounted display device;
determining, by the calibration device and using one or more machine learning models, whether a blink, associated with the information from the eye tracking device, is voluntary, spontaneous, or reflexive based on using a peak velocity and a particular duration associated with the blink,
wherein the one or more machine learning models are trained to use information associated with the virtual image and historical information associated with historical blink frequencies and durations for spontaneous blinks and reflex blinks to determine whether the blink is voluntary, spontaneous, or reflexive,
wherein the information includes one or more of:
information associated with a manner in which the virtual image is displayed, or
user information;
recording, by the calibration device, an alignment measurement based on the position of the at least one point on the three-dimensional real-world object in a real-world coordinate system based on a threshold amount of time before or after the user performed the voluntary eye blink,
wherein recording the alignment measurement is based on using the one or more machine learning models to determine whether the blink is voluntary, spontaneous, or reflexive; and
generating, by the calibration device, a function providing a mapping between three-dimensional points in the real-world coordinate system and corresponding points in the display space of the optical see-through head-mounted display device based on the alignment measurement.