US 11,717,748 B2
Latency compensation using machine-learned prediction of user input
Iestyn Bleasdale-Shepherd, Bellevue, WA (US); Scott Dalton, Seattle, WA (US); and Richard Kaethler, Redmond, WA (US)
Assigned to Valve Corporation, Bellevue, WA (US)
Filed by Valve Corporation, Bellevue, WA (US)
Filed on Nov. 19, 2019, as Appl. No. 16/688,833.
Prior Publication US 2021/0146241 A1, May 20, 2021
Int. Cl. A63F 13/358 (2014.01); A63F 13/355 (2014.01); A63F 13/86 (2014.01); H04N 21/478 (2011.01); A63F 13/332 (2014.01); H04L 67/131 (2022.01); A63F 13/335 (2014.01)
CPC A63F 13/355 (2014.09) [A63F 13/332 (2014.09); A63F 13/358 (2014.09); A63F 13/86 (2014.09); H04L 67/131 (2022.05); H04N 21/4781 (2013.01); A63F 13/335 (2014.09)] 23 Claims
OG exemplary drawing
 
1. A method comprising:
receiving, by a computing system, sensor data associated with a player of a video game, the sensor data having been generated by at least one of a gyroscope of a game controller associated with the player, an accelerometer of the game controller, a pressure sensor of the game controller, or a biofeedback sensor on or in proximity to the player;
predicting, by the computing system, user input to the game controller that will cause corresponding game control data to be received by the computing system within a time period since the receiving of the sensor data by:
providing the sensor data as input to a trained machine learning model; and
generating a score as output from the trained machine learning model, the score indicative of a probability that a type of user input will be provided to the game controller;
after the generating of the score and prior to proactively generating game control data that corresponds to the type of user input, accessing data indicating types of user input for which respective game control data is to be generated on behalf of players to compensate for latency;
determining that the type of user input is one of the types of user input;
in response to the determining that the type of user input is one of the types of user input, proactively generating, by the computing system, in advance of receiving actual game control data from a client machine associated with the player, the game control data based at least in part on the score;
providing the game control data as input to the video game;
receiving video game data as output from the video game based at least in part on the game control data; and
causing, by the computing system, presentation of the video game data on a display without waiting for the actual game control data to be received by the computing system from the client machine.