US 12,390,724 B2
Synchronizing audio streams in cloud-based gaming environment
Krishna Kant Chintalapudi, Bellevue, WA (US); Pouya Hamadanian, Cambridge, MA (US); Doug Gallatin, Kirkland, WA (US); and Thomas Pouget-Abadie, Seattle, WA (US)
Assigned to Microsoft Technology Licensing, LLC, Redmond, WA (US)
Filed by Microsoft Technology Licensing, LLC, Redmond, WA (US)
Filed on Feb. 21, 2023, as Appl. No. 18/171,926.
Prior Publication US 2024/0278118 A1, Aug. 22, 2024
Int. Cl. A63F 13/358 (2014.01); A63F 13/215 (2014.01); A63F 13/23 (2014.01); A63F 13/355 (2014.01); A63F 13/54 (2014.01); G10L 19/018 (2013.01); G10L 19/16 (2013.01)
CPC A63F 13/358 (2014.09) [A63F 13/215 (2014.09); A63F 13/23 (2014.09); A63F 13/355 (2014.09); A63F 13/54 (2014.09); G10L 19/018 (2013.01); G10L 19/167 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A data processing system comprising:
a processor; and
a machine-readable medium storing executable instructions that, when executed, cause the processor to perform operations comprising:
generating a first audio stream comprising audio content associated with a game being played on a cloud-based gaming platform, the cloud-based gaming platform being configured to execute the game on a server associated with the cloud-based gaming platform and to output audio and video streams comprising game content;
generating a second audio stream comprising the audio content associated with the game being played and an acoustic marker inaudible to a human user;
sending the first audio stream to a controller associated with the cloud-based gaming platform, the controller being associated with a first user, the controller providing a user interface for receiving control commands for the first user, the controller further configured to generate control signals based on the control commands and to send the control signals to the cloud-based gaming platform;
sending the second audio stream to a display device associated with the first user, the display device comprising a display for presenting video content from video streams received from the cloud-based gaming platform, and a first speaker for presenting audio content from the second audio stream;
receiving a third audio stream from the controller, the third audio stream being generated based on signals output by a second microphone associated with the controller, the third audio stream including at least the acoustic marker of the second audio stream output by the first speaker of the display device captured by the second microphone;
receiving timestamp information from the controller indicating when the controller outputs audio frames of the first audio stream via a second speaker associated with the controller;
analyzing the third audio stream and the timestamp information to determine an inter-stream latency between the first audio stream and the second audio stream, the inter-stream latency representing a difference between latencies experienced by the first audio stream and the second audio stream; and
adjusting one or more parameters of the first audio stream, the second audio stream, or both to reduce or eliminate the inter-stream latency to synchronize playback of the first audio stream at the controller with the playback of the second audio stream at the display device.