US 12,242,064 B2
Head-mounted display with low light operation
Killian J. Poore, Santa Clara, CA (US); Trevor J. Ness, Santa Cruz, CA (US); Stephen E. Dey, San Francisco, CA (US); and Fletcher R. Rothkopf, Los Altos, CA (US)
Assigned to APPLE INC., Cupertino, CA (US)
Filed by APPLE INC., Cupertino, CA (US)
Filed on Jul. 6, 2023, as Appl. No. 18/218,787.
Application 18/218,787 is a continuation of application No. 16/856,080, filed on Apr. 23, 2020, granted, now 11,733,518.
Claims priority of provisional application 62/838,995, filed on Apr. 26, 2019.
Prior Publication US 2023/0359037 A1, Nov. 9, 2023
Int. Cl. G02B 27/01 (2006.01); G06F 3/0346 (2013.01); G06V 10/80 (2022.01); G06V 20/20 (2022.01)
CPC G02B 27/0172 (2013.01) [G06F 3/0346 (2013.01); G06V 10/803 (2022.01); G06V 20/20 (2022.01); G02B 27/017 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A head-mounted display system, comprising:
a display device configured to display graphical content;
a sensor configured to obtain visible light images of an environment at a first time when the head-mounted display system is moved through the environment, wherein the visible light images are associated with first location information that describes a position and orientation of the head-mounted display system relative to the environment when the visible light images are obtained by the sensor; and
a controller configured to:
generate a rendering of the environment, wherein the rendering is generated from the visible light images of the environment obtained at the first time, and the rendering is generated based on the position and orientation of the head-mounted display system when the visible light images were obtained by the sensor, and
output the rendering of the environment to the display as part of the graphical content at a second time when the head-mounted display is in the environment, wherein the rendering of the environment is displayed according to the position and orientation of the head-mounted display system relative to the environment at the second time.
 
7. A head-mounted display system, comprising: a controller; a display device configured to display graphical content; a camera configured to obtain visible light images of an environment from the head-mounted display system; a movement sensor configured to determine a position and orientation of the head-mounted display system; and a controller, wherein: the controller is configured to obtain the visible light images using the camera at a first time when the head-mounted display is in the environment and store the visible light images in a storage, and the controller is configured to recall the visible light images from the storage at a second time when the head-mounted display is in the environment, generate the graphical content such that it includes a rendering of the environment generated from the visible light images that are recalled from the storage, and output the graphical content using the display according to the position and orientation of the head-mounted display system at the second time to enhance visibility of the environment in low light conditions.
 
14. A head-mounted display system, comprising: a display device configured to display graphical content; a sensor configured to output a sensor signal that represents an observation of an environment; a movement sensor configured to determine a position and orientation of the head-mounted display system; and a controller, wherein: the controller is configured to generate a rendering of the environment based on the sensor signal when the head-mounted display is in the environment, wherein the rendering of the environment includes computer-generated graphical reproductions of objects from the environment that are generated using the sensor output signal, and the controller is configured to output the graphical content to the display according to the position and orientation of the head-mounted display system when the head-mounted display is in the environment to enhance visibility of the environment in low light conditions, wherein the graphical content includes the rendering of the environment.