US 12,267,541 B2
Interactive video streaming for 3D applications
Christian Häring, Bavaria (DE); Stefan Koch, Bavaria (DE); and Andreas Adam, Bavaria (DE)
Assigned to MONKEYWAY GmbH, Munich (DE)
Filed by MONKEYWAY GmbH, Munich (DE)
Filed on Sep. 29, 2023, as Appl. No. 18/478,238.
Claims priority of provisional application 63/377,986, filed on Sep. 30, 2022.
Prior Publication US 2024/0114190 A1, Apr. 4, 2024
Int. Cl. H04N 21/262 (2011.01); G06T 9/00 (2006.01); G06T 15/00 (2011.01); H04N 21/218 (2011.01); H04N 21/442 (2011.01); H04N 21/472 (2011.01); H04N 21/81 (2011.01); H04L 65/65 (2022.01)
CPC H04N 21/26208 (2013.01) [G06T 9/00 (2013.01); G06T 15/00 (2013.01); H04N 21/21805 (2013.01); H04N 21/44227 (2013.01); H04N 21/472 (2013.01); H04N 21/8146 (2013.01); H04L 65/65 (2022.05)] 28 Claims
OG exemplary drawing
 
1. A video processing system comprising:
a plurality of servers operating in a peer-to-peer communication network via a WebRTC application programming interface (API) to generate 3D streaming data;
the plurality of servers responding to a session handler receiving a request for a 3D streaming session at a first client device by:
(i) receiving from the session handler in communication with the first client device, context verification about the first client device including an internet protocol (IP) location of the first client device;
(ii) responding to the context verification about the first client device by assigning a first server from the plurality to direct a 3D streaming session at the first client device based on the internet protocol (IP) location of the first client device;
(iii) directing a first instance of an encapsulated 3D streaming engine at the first server to initiate the 3D streaming session via the WebRTC application programming interface (API) at the first client device;
(iv) injecting a library file of the WebRTC application programming interface (API) into an executable file of the encapsulated 3D streaming engine to generate a digital representation of an interactive 3D environment for the 3D streaming session at the first client device;
(v) rendering the digital representation of the interactive 3D environment within a buffer of a graphics processing unit (GPU);
(vi) an encoder encoding the rendered representation for transmission of the 3D streaming session to the first client device;
(vii) streaming the encoded representation of the interactive 3D environment at the first client device;
(viii) dynamically adjusting the 3D streaming session of the interactive 3D environment in response to input received from the first client device; and
(ix) sustaining the digital representation of the interactive 3D environment of the 3D streaming session by iteratively processing (v) through (viii) until the 3D streaming session is terminated at the first client device.