US 11,722,655 B2
Low latency networking of plenoptic data
Neal Weinstock, Brooklyn, NY (US)
Assigned to SOLIDDD CORP., Brooklyn, NY (US)
Filed by Soliddd Corp., Brooklyn, NY (US)
Filed on Nov. 30, 2021, as Appl. No. 17/538,588.
Prior Publication US 2023/0171396 A1, Jun. 1, 2023
Int. Cl. H04N 13/383 (2018.01); H04N 13/282 (2018.01); H04N 13/167 (2018.01); H04N 13/194 (2018.01); H04N 23/63 (2023.01); H04N 23/957 (2023.01)
CPC H04N 13/383 (2018.05) [H04N 13/167 (2018.05); H04N 13/194 (2018.05); H04N 13/282 (2018.05); H04N 23/63 (2023.01); H04N 23/957 (2023.01)] 16 Claims
OG exemplary drawing
 
1. A method, comprising;
obtaining, using at least one image obtaining sensor of an image capture device, image data, wherein the image obtaining sensor comprises a camera having a plenoptic lens arrangement;
tracking, using a gaze detection sensor, a user's gaze on a display of the image capture device, wherein a plurality of lens tiles are disposed on the display and wherein each of the plurality of lens tiles cover a portion of the display, wherein the portion of the plurality of lens tiles that the user's gaze is directed toward provides a full focus image of the whole image obtained by the image obtaining sensor;
predicting, using a processor and based on the tracking, a subsequent gaze location on the display that the user's gaze is expected to be directed toward, wherein the predicting the subsequent gaze location comprises:
identifying another portion of the display associated with the predicted subsequent gaze location; and
identifying a new set of the plurality of lens tiles that are disposed over the another portion;
transmitting, using a processor and based upon an amount of bandwidth present, additional image data to the display associated with the portion of the plurality of lens tiles the user is currently viewing and the another portion of the display associated with the new set of the plurality of lens tiles, wherein the additional image data comprises a portion of redundant image data providing the full focus image of the whole image obtained by the camera having the plenoptic lens arrangement; and
updating, responsive to identifying that the user's gaze has transitioned to the subsequent gaze location and using the additional image data, the image data associated with the another portion of the display corresponding to the new set of the plurality of lens tiles, wherein the updating the image data associated with the another portion of the display comprises utilizing foveated rendering, wherein the foveated rendering comprises maintaining a high resolution for the additional image data associated with the another portion of the display corresponding to the new set of the plurality of lens tiles.