US 12,277,264 B2
Eye tracking based video transmission and compression
Praveen Babu J D, Plantation, FL (US)
Assigned to Magic Leap, Inc., Plantation, FL (US)
Appl. No. 18/037,709
Filed by Magic Leap, Inc., Plantation, FL (US)
PCT Filed Sep. 16, 2021, PCT No. PCT/US2021/050665
§ 371(c)(1), (2) Date May 18, 2023,
PCT Pub. No. WO2022/108652, PCT Pub. Date May 27, 2022.
Claims priority of provisional application 63/115,287, filed on Nov. 18, 2020.
Prior Publication US 2023/0409111 A1, Dec. 21, 2023
Int. Cl. G06F 3/01 (2006.01); G06T 7/50 (2017.01); G06V 10/25 (2022.01); H04N 19/103 (2014.01); H04N 19/162 (2014.01); H04N 19/164 (2014.01); H04N 19/167 (2014.01); H04N 19/17 (2014.01); H04N 19/59 (2014.01)
CPC G06F 3/013 (2013.01) [G06T 7/50 (2017.01); G06V 10/25 (2022.01); H04N 19/59 (2014.11)] 15 Claims
OG exemplary drawing
 
1. A computer-implemented method comprising:
receiving gaze information about an observer of a video stream, wherein the gaze information includes information about instantaneous eye position and instantaneous eye velocity;
determining a video compression spatial map for the video stream based on the received gaze information and performance characteristics of a network connection with the observer, wherein the determining of the video compression spatial map includes:
selecting a first shape for a region of interest;
selecting a first video compression profile that includes higher compression outside the first shape;
selecting a second shape within the first shape of the region of interest and sharing a center of the region of interest;
scaling a size of the second shape in proportion to a network latency times a maximum eye velocity; and
selecting a second video compression profile that includes: (1) lower compression inside the second shape; (2) medium compression outside the second shape, but inside the first shape; and (3) higher compression outside the first shape, and
wherein the performance characteristics of the network connection include information about the network latency, and comprising:
identifying the center of the region of interest corresponding to a predicted eye position, wherein the center of the region of interest corresponds to the instantaneous eye position plus an offset proportional to the instantaneous eye velocity times the network latency;
compressing the video stream according to the video compression spatial map; and
sending the compressed video stream to the observer.