US 12,231,615 B2
Display system with machine learning (ML) based stereoscopic view synthesis over a wide field of view
Lei Xiao, Redmond, WA (US); Salah Eddine Nouri, Zurich (CH); Joel Hegland, Snohomish, WA (US); and Douglas Robert Lanman, Bellevue, WA (US)
Assigned to Meta Platforms Technologies, LLC, Menlo Park, CA (US)
Filed by Meta Platforms Technologies, LLC, Menlo Park, CA (US)
Filed on Nov. 3, 2022, as Appl. No. 17/980,342.
Claims priority of provisional application 63/303,371, filed on Jan. 26, 2022.
Prior Publication US 2023/0239456 A1, Jul. 27, 2023
Int. Cl. H04N 13/344 (2018.01); G02B 30/22 (2020.01); G06F 1/16 (2006.01); G06G 1/16 (2006.01); H04N 13/156 (2018.01); H04N 13/257 (2018.01); H04N 13/00 (2018.01)
CPC H04N 13/344 (2018.05) [G02B 30/22 (2020.01); G06F 1/163 (2013.01); H04N 13/156 (2018.05); H04N 13/257 (2018.05); H04N 2013/0081 (2013.01)] 19 Claims
OG exemplary drawing
 
1. A display system, comprising:
a head-mounted display (HMD) having a pass-through configuration, comprising:
at least two exterior-facing color stereo cameras mounted on a front face of the HMD and on a visual plane of both of a user's eyes, wherein the at least two exterior-facing color stereo cameras collects images;
a processor to receive the collected images to perform stereo view synthesis; and
a memory storing instructions, which when executed by the processor, cause the processor to perform stereo view synthesis by performing:
depth estimation;
sharpening;
splatting;
disocclusion filtering to minimize a disocclusion region, wherein the disocclusion region is due to a viewpoint difference caused by a lateral offset in the visual plane between each of the at least two exterior-facing color stereo cameras and a matching one of both of the user's eyes; and
fusion.