US 11,998,837 B2
Video compression system and method for compensating for bandwidth limitations of a communication channel
Roger van der Laan, San Mateo, CA (US); and Stephen G. Perlman, Palo Alto, CA (US)
Assigned to Sony Interactive Entertainment LLC, San Mateo, CA (US)
Filed by Sony Interactive Entertainment LLC, San Mateo, CA (US)
Filed on Nov. 20, 2018, as Appl. No. 16/197,323.
Application 16/197,323 is a continuation of application No. 12/359,127, filed on Jan. 23, 2009, granted, now 10,130,891, issued on Nov. 20, 2018.
Application 12/359,127 is a continuation of application No. 11/999,457, filed on Dec. 5, 2007, granted, now 9,003,461, issued on Apr. 7, 2015.
Prior Publication US 2019/0083890 A1, Mar. 21, 2019
This patent is subject to a terminal disclaimer.
Int. Cl. A63F 13/335 (2014.01); A63F 13/352 (2014.01); A63F 13/533 (2014.01); A63F 13/87 (2014.01); H04L 65/611 (2022.01); H04L 65/65 (2022.01); H04L 65/70 (2022.01); H04L 65/75 (2022.01); H04L 67/131 (2022.01); H04N 7/10 (2006.01); H04N 21/21 (2011.01); H04N 21/478 (2011.01); H04N 21/61 (2011.01); H04W 84/12 (2009.01); H04W 88/14 (2009.01)
CPC A63F 13/335 (2014.09) [A63F 13/352 (2014.09); A63F 13/533 (2014.09); A63F 13/87 (2014.09); H04L 65/611 (2022.05); H04L 65/65 (2022.05); H04L 65/70 (2022.05); H04L 65/765 (2022.05); H04L 67/131 (2022.05); H04N 21/21 (2013.01); H04N 21/4781 (2013.01); H04N 21/6125 (2013.01); A63F 2300/402 (2013.01); A63F 2300/513 (2013.01); A63F 2300/538 (2013.01); A63F 2300/552 (2013.01); H04N 7/106 (2013.01); H04W 84/12 (2013.01); H04W 88/14 (2013.01)] 16 Claims
OG exemplary drawing
 
1. A computer-implemented method for streaming online video games over a network, comprising:
receiving control signals by a server, the control signals received from a client device over the network, the control signals are responsive to a user input from the client device during play of a video game;
executing program code of the video game on the server;
generating a plurality of video frames resulting from execution of the video game;
encoding, by compression resources the plurality of video frames to generate compressed video frames, wherein said compression resources are part of a pool of resources used to process content of a plurality of interactive applications including the video game and are shared among a plurality of servers including the server, wherein the encoding includes,
(a) determining by the server, a maximum data rate of a communication channel established between the server and the client device, wherein the maximum data rate is determined by sending a test stream to the client device and is used initially to encode a first set of video frames of the plurality of video frames, and
(b) updating a compression process state maintained for said video game executing on the server for adjusting an amount of compression used for generating said compressed video frames for subsequent sets of video frames following previous compressed video frames, wherein the compression process state is updated dynamically to include information about compression used in the previous compressed video frames and changes detected in communication channel state information during transmission of said previous compressed video frames of the video game, the adjusted amount of compression used to define a data rate for the communication channel to transmit the compressed video frames for the subsequent sets of video frames, after the maximum data rate is identified;
transmitting the compressed video frames from the server to the client device, wherein the compressed video frames are compressed in accordance to the data rate defined by the compression process state defined for the video game; and
while transmitting the compressed video frames for the video game, continuing periodic monitoring using a data stream of said compressed video frames of the video game to determine any changes in the data rate for the communication channel based on measurements of at least one of an increase in the packet loss rate or said latency for the video game, and responsively updating the compression process state for the video game to adjust the data rate by adjusting said amount of compression when encoding the subsequent sets of video frames to generate the compressed video frames until at least one of the packet loss rate or said latency have reached the acceptable level, wherein the data stream of said compressed video frames is from the video game produced during play of the video game instead of the data rate test stream.