US 12,299,193 B2
Gaze demarcating system, method, device and non-transitory computer readable storage medium
Yachong Xue, Beijing (CN); Xue Dong, Beijing (CN); Zhongyuan Wu, Beijing (CN); Jiankang Sun, Beijing (CN); Guixin Yan, Beijing (CN); Guang Bai, Beijing (CN); and Jiyang Shao, Beijing (CN)
Assigned to BEIJING SHIYAN TECHNOLOGY CO., LTD., Beijing (CN)
Appl. No. 18/029,672
Filed by Beijing Shiyan Technology Co., Ltd., Beijing (CN)
PCT Filed May 27, 2022, PCT No. PCT/CN2022/095755
§ 371(c)(1), (2) Date Mar. 31, 2023,
PCT Pub. No. WO2023/226034, PCT Pub. Date Nov. 30, 2023.
Prior Publication US 2024/0361830 A1, Oct. 31, 2024
Int. Cl. G06F 3/01 (2006.01); G06T 7/73 (2017.01)
CPC G06F 3/013 (2013.01) [G06F 3/017 (2013.01); G06T 7/73 (2017.01); G06T 2207/10024 (2013.01); G06T 2207/10048 (2013.01); G06T 2207/30201 (2013.01)] 18 Claims
OG exemplary drawing
 
1. A gaze demarcating system, comprising: an eye movement tracking apparatus, a gesture recognition apparatus, a display interaction apparatus, and a gaze behavior determining apparatus, wherein;
the eye movement tracking apparatus is configured to acquire pupil position coordinates;
the gesture recognition apparatus is configured to acquire gesture position coordinates;
the display interaction apparatus is configured to interact with a gesture;
the gaze behavior determining apparatus is respectively connected with the eye movement tracking apparatus, the gesture recognition apparatus, and the display interaction apparatus, and is configured to, in a process of the display interaction apparatus interacting with the gesture, obtain a plurality of groups of effective gaze coordinates and a plurality of groups of effective gaze pupil position coordinates corresponding to the plurality of groups of effective gaze coordinates according to the acquired gesture position coordinates and pupil position coordinates, wherein the plurality of groups of effective gaze coordinates correspond to different interaction positions in a display region of the display interaction apparatus; and according to the plurality of groups of effective gaze coordinates and the corresponding plurality of groups of effective gaze pupil position coordinates, calculate a gaze mapping model to perform gaze demarcating;
the eye movement tracking apparatus is further configured to acquire gaze point coordinates;
the gesture recognition apparatus is further configured to acquire gesture pausing point coordinates;
the display interaction apparatus is further configured to send data of current display interaction region to the gaze behavior determining apparatus;
the gaze behavior determining apparatus is further configured to acquire the data of the current display interaction region, determine whether eyes are in an effective gaze state according to the acquired gaze point coordinates and the acquired data of the display interaction region, determine whether the gesture is in an effective interaction state according to the acquired gesture pausing point coordinates and the acquired data of the display interaction region, and determining that the eyes are in the effective gaze state and the gesture is in the effective interaction state, calculate an average value of a plurality of gaze point coordinates in effective gaze time, and calculate an average value of a plurality of gesture pausing points in effective interaction time; calibrate the gaze point coordinates according to the data of the display interaction region and the average value of the plurality of gaze point coordinates; and compensate the gesture pausing point coordinates according to the average value of the plurality of gesture pausing points; and
the gaze behavior determining apparatus is configured to:
calculate center coordinates of the data of the current display interaction region by taking data of the display interaction region closest to the average value of the plurality of gaze point coordinates and the average value of the plurality of gesture pausing point coordinates as the data of the current display interaction region;
calculate a difference between the average value of the plurality of gaze points coordinates and the center coordinates to obtain a first difference; calculate a difference between the average value of the plurality of gestures pausing point coordinates and the central coordinates to obtain a second difference; and
compensate the gaze point coordinates according to the first difference, and compensate the gesture pausing point coordinates according to the second difference.