US 12,278,938 B2
Mixed reality display method, mixed reality device, and storage medium
Qianwen Jiang, Beijing (CN); Hao Zhang, Beijing (CN); Lili Chen, Beijing (CN); Peng Han, Beijing (CN); Huidong He, Beijing (CN); Juanjuan Shi, Beijing (CN); and Weihua Du, Beijing (CN)
Assigned to Beijing BOE Optoelectronics Technology Co., Ltd., Beijing (CN); and BOE Technology Group Co., Ltd., Beijing (CN)
Appl. No. 18/036,379
Filed by Beijing BOE Optoelectronics Technology Co., Ltd., Beijing (CN); and BOE Technology Group Co., Ltd., Beijing (CN)
PCT Filed Dec. 21, 2020, PCT No. PCT/CN2020/138106
§ 371(c)(1), (2) Date May 10, 2023,
PCT Pub. No. WO2022/133683, PCT Pub. Date Jun. 30, 2022.
Prior Publication US 2023/0403382 A1, Dec. 14, 2023
Int. Cl. H04N 13/156 (2018.01); H04N 13/15 (2018.01); H04N 13/383 (2018.01)
CPC H04N 13/156 (2018.05) [H04N 13/15 (2018.05); H04N 13/383 (2018.05)] 18 Claims
OG exemplary drawing
 
1. A method for displaying a mixed reality (MR), applied to an MR apparatus, wherein the MR apparatus comprises a display screen, an eye tracker on a display surface of the display screen, and an image sensor array on a non-display surface of the display screen, a corresponding relation is between image sensors in the image sensor array and a plurality of display sub-regions in the display screen, and the method comprises:
after receiving a mixed display signal, determining a gaze point of a user on the display screen by tracking an eyeball of the user through the eye tracker;
determining a display sub-region at which the user gazes according to a corresponding relation between gaze points and display sub-regions in the display screen;
determining at least one first image sensor according to the corresponding relation between display sub-regions and image sensors in the image sensor array, and adjusting the at least one first image sensor to increase a resolution of an image generated by the at least one first image sensor, wherein a distance between a center point of a display sub-region corresponding to the first image sensor and the gaze point is less than a set distance; and
obtaining and displaying an MR image by superimposing and rendering an environment image output by the image sensor array and a virtual image;
wherein the adjusting the at least one first image sensor comprises:
determining whether a gaze duration for which the gaze point remains in a display sub-region corresponding to the at least one first image sensor is greater than a preset threshold;
in response to determining that the gaze duration is less than or equal to the preset threshold, increasing the resolution of the image generated by the at least one first image sensor, so as to increase a resolution of a first image; and
in response to determining that the gaze duration is greater than the preset threshold, keeping the at least one first image sensor in a working state, and displaying an image output by the at least one first sensor as the environment image in a full-screen mode.