US 12,069,408 B2
Systems and methods for interpolative three-dimensional imaging within the viewing zone of a display
Michael Naimark, Berkeley, CA (US); Cameron Ballard, Shanghai (CN); Bruce Luo, San Mateo, CA (US); David Santiano, San Francisco, CA (US); Grace Huang, Shanghai (CN); Mateo Juvera Molina, Shanghai (CN); and Ada Zhao, Boulder, CA (US)
Assigned to NYU SHANGHAI, (CN)
Appl. No. 17/604,333
Filed by SHANGHAI NEW YORK UNIVERSITY, Shanghai (CN)
PCT Filed Apr. 15, 2019, PCT No. PCT/CN2019/082642
§ 371(c)(1), (2) Date Oct. 15, 2021,
PCT Pub. No. WO2020/210937, PCT Pub. Date Oct. 22, 2020.
Prior Publication US 2022/0217301 A1, Jul. 7, 2022
Int. Cl. H04N 7/14 (2006.01); G06T 7/521 (2017.01); G06T 7/55 (2017.01); H04N 23/661 (2023.01); H04N 23/90 (2023.01)
CPC H04N 7/144 (2013.01) [G06T 7/521 (2017.01); G06T 7/55 (2017.01); H04N 23/661 (2023.01); H04N 23/90 (2023.01); G06T 2207/10028 (2013.01); G06T 2207/10048 (2013.01)] 14 Claims
OG exemplary drawing
 
1. A system for three-dimensional video capture within a viewing zone of a display, comprising:
a frame configured to be positioned surrounding a display;
a plurality of cameras attached to the frame, each camera configured to generate a two or three-dimensional array of pixels; and
a processor connected to the plurality of cameras executing an interpolator configured to combine a plurality of two or three-dimensional arrays of pixels from the plurality of cameras into an output three-dimensional array of pixels, the interpolator being further configured to apply an offset or viewpoint shift to each pixel of the two or three-dimensional array of pixels generated by at least one camera of the plurality of cameras prior to combining the plurality of two or three-dimensional arrays of pixels, and wherein the interpolator is further configured to identify the offset or viewpoint shift according to a transformation matrix specific to a corresponding camera of the plurality of cameras, and further wherein the interpolator is further configured to generate the transformation matrix for each camera of the plurality of cameras responsive to comparison of a predetermined image to an output of the camera.