US 12,332,437 B2
System and method for reducing image data throughput to head mounted display
Christopher A. Peri, Mountain View, CA (US); Yingen Xiong, Mountain View, CA (US); and Lu Luo, Sunnyvale, CA (US)
Assigned to Samsung Electronics Co., Ltd., Suwon-si (KR)
Filed by Samsung Electronics Co., Ltd., Suwon-si (KR)
Filed on Aug. 11, 2020, as Appl. No. 16/990,779.
Claims priority of provisional application 63/006,490, filed on Apr. 7, 2020.
Prior Publication US 2021/0311307 A1, Oct. 7, 2021
Int. Cl. G02B 27/01 (2006.01); G06F 3/01 (2006.01); G06T 11/00 (2006.01); G06T 19/20 (2011.01)
CPC G02B 27/017 (2013.01) [G02B 27/0101 (2013.01); G06F 3/012 (2013.01); G06T 11/006 (2013.01); G06T 19/20 (2013.01); G02B 2027/014 (2013.01)] 27 Claims
OG exemplary drawing
 
1. A method of reducing a transmission load between components providing an extended reality (XR) display, the method comprising:
obtaining, from at least one memory of an electronic device communicatively connected over a wireless link to a head mounted display (HMD), a first reference frame, wherein the first reference frame comprises a first set of rendered pixels associated with an XR display to be provided by the HMD, wherein the wireless link comprises a wireless data path between a transmitter of the electronic device and a receiver of the HMD, and wherein the first set of rendered pixels is associated with a first time;
receiving, from the HMD via the wireless link, head pose data;
receiving, from the HMD via the wireless link, gaze tracking data showing a direction of a user's gaze;
generating, by the electronic device based on the head pose data, a predicted head pose of the HMD at a second time subsequent to the first time;
rendering, at the electronic device, a source image as a new frame, wherein the new frame comprises a second set of rendered pixels associated with the XR display to be provided by the HMD at the second time, wherein the new frame is rendered based on the predicted head pose of the HMD at the second time and at decreasing resolution at points away from the user's gaze based on the gaze tracking data;
generating, by the electronic device, a differential frame, wherein the differential frame is based on a difference operation between (i) the rendered pixels of the new frame and (ii) the rendered pixels of the first reference frame, wherein the differential frame includes pixels unique to the new frame and does not include pixels subtracted from the new frame during the difference operation, and wherein the subtracted pixels are not unique to the new frame;
selecting one of a plurality of encoding modes based on a size of the differential frame;
encoding the differential frame using the selected encoding mode, wherein encoding the differential frame comprises applying a video compression algorithm to the differential frame;
sending the encoded differential frame to the HMD via the wireless link; and
storing the new frame in the at least one memory of the electronic device as a second reference frame.