US 12,081,769 B2
Rate control for low latency video encoding and transmission
Hao Pan, Sunnyvale, CA (US); Jim C. Chou, San Jose, CA (US); and Felix C. Fernandes, Saratoga, CA (US)
Assigned to Apple Inc., Cupertino, CA (US)
Filed by Apple Inc., Cupertino, CA (US)
Filed on Sep. 16, 2022, as Appl. No. 17/933,057.
Application 17/933,057 is a continuation of application No. 17/252,701, filed on Dec. 15, 2020, granted, now 11,451,788.
Application 17/252,701 is a continuation of application No. PCT/US2019/039586, filed on Jun. 27, 2019.
Claims priority of provisional application 62/691,427, filed on Jun. 28, 2018.
Prior Publication US 2023/0009707 A1, Jan. 12, 2023
Int. Cl. H04N 19/146 (2014.01); H04L 65/70 (2022.01); H04L 65/75 (2022.01); H04N 19/14 (2014.01); H04N 19/169 (2014.01); H04N 19/176 (2014.01); H04N 19/63 (2014.01)
CPC H04N 19/146 (2014.11) [H04L 65/70 (2022.05); H04L 65/75 (2022.05); H04N 19/14 (2014.11); H04N 19/176 (2014.11); H04N 19/188 (2014.11); H04N 19/63 (2014.11)] 20 Claims
OG exemplary drawing
 
1. A video encoding system, comprising:
a wavelet transform component comprising one or more processors configured to:
apply a wavelet transform to pixel blocks from a current video frame to decompose the pixel blocks into a plurality of frequency bands; and
output frequency blocks comprising frequency data for the plurality of frequency bands;
a complexity analysis component comprising one or more processors configured to:
analyze the frequency data in the frequency blocks to calculate complexity statistics across the plurality of frequency bands of the current video frame; and
output the complexity statistics for the plurality of frequency bands of the current video frame;
a block-based encoder configured to:
access a foveated region from a previous video frame;
determine a number of bits that were used to encode co-sited pixel blocks from the previous video frame;
determine quantization parameters for each of the frequency bands of the plurality of frequency bands of the current video frame based, at least in part, on:
(a) the complexity statistics for the plurality of frequency bands of the current video frame;
(b) the other complexity statistics from the previous video frame;
(c) the foveated region of the previous video frame; and
(d) the determined number of bits that were used to encode the co-sited pixel blocks from the previous video frame; and
apply an encoding technique to the frequency blocks of the current video frame, wherein the encoding technique uses the determined quantization parameters for the plurality of frequency bands of the current video frame to compress the frequency data in the frequency blocks; and
a wireless interface configured to transmit the compressed frequency data over a wireless connection to a receiving device.