US 12,137,199 B2
Immersive video streaming using view-adaptive prefetching and buffer control
Fanyi Duanmu, San Jose, CA (US); Jun Xin, San Jose, CA (US); Hsi-Jung Wu, San Jose, CA (US); and Xiaosong Zhou, Campbell, CA (US)
Assigned to Apple Inc., Cupertino, CA (US)
Filed by Apple Inc., Cupertino, CA (US)
Filed on Jan. 8, 2024, as Appl. No. 18/407,198.
Application 18/407,198 is a continuation of application No. 18/083,468, filed on Dec. 16, 2022, granted, now 11,924,391.
Application 18/083,468 is a continuation of application No. 17/326,204, filed on May 20, 2021, granted, now 11,570,417, issued on Jan. 31, 2023.
Prior Publication US 2024/0146892 A1, May 2, 2024
Int. Cl. H04N 13/117 (2018.01); H04N 13/178 (2018.01); H04N 21/218 (2011.01); H04N 21/24 (2011.01)
CPC H04N 13/117 (2018.05) [H04N 13/178 (2018.05); H04N 21/21805 (2013.01); H04N 21/2401 (2013.01); H04N 21/2402 (2013.01)] 21 Claims
OG exemplary drawing
 
1. A method comprising:
obtaining, by one or more processors, a data set representing immersive video content for display at a display time, wherein the data set comprises:
first data representing the immersive video content according to a first level of detail, and
second data representing the immersive video content according to a second level of detail, wherein the second level of detail is higher than the first level of detail;
during one or more first times prior to the display time, causing, by the one or more processors, at least a portion of the first data to be stored in a video buffer;
during one or more second times subsequent to the one or more first times and prior to the display time:
generating, by the one or more processors, a prediction of a viewport for displaying the immersive video content to a user at the display time,
identifying, by the one or more processors, a portion of the second data corresponding to the prediction of the viewport, and
causing, by the one or more processors, the identified portion of the second data to be stored in the video buffer; and
at the display time, causing, by the one or processors, the immersive video content to be displayed to the user using the video buffer.
 
11. A system comprising:
one or more processors; and
memory storing instructions that when executed by the one or more processors, cause the one or more processors to perform operations comprising:
obtaining a data set representing immersive video content for display at a display time, wherein the data set comprises:
first data representing the immersive video content according to a first level of detail, and
second data representing the immersive video content according to a second level of detail, wherein the second level of detail is higher than the first level of detail;
during one or more first times prior to the display time, causing at least a portion of the first data to be stored in a video buffer;
during one or more second times subsequent to the one or more first times and prior to the display time:
generating a prediction of a viewport for displaying the immersive video content to a user at the display time,
identifying a portion of the second data corresponding to the prediction of the viewport, and
causing the identified portion of the second data to be stored in the video buffer; and
at the display time, causing the immersive video content to be displayed to the user using the video buffer.
 
21. One or more non-transitory, computer-readable storage media having instructions stored thereon, that when executed by one or more processors, cause the one or more processors to perform operations comprising:
obtaining a data set representing immersive video content for display at a display time, wherein the data set comprises:
first data representing the immersive video content according to a first level of detail, and
second data representing the immersive video content according to a second level of detail, wherein the second level of detail is higher than the first level of detail;
during one or more first times prior to the display time, causing at least a portion of the first data to be stored in a video buffer;
during one or more second times subsequent to the one or more first times and prior to the display time:
generating a prediction of a viewport for displaying the immersive video content to a user at the display time,
identifying a portion of the second data corresponding to the prediction of the viewport, and
causing the identified portion of the second data to be stored in the video buffer; and
at the display time, causing the immersive video content to be displayed to the user using the video buffer.