US 12,032,735 B2
Method and apparatus for real-time data communication in full-presence immersive platforms
Amir Rubin, Los Gatos, CA (US); Joel Breton, Santa Rosa, CA (US); Ariel Z. Rubin, Los Angeles, CA (US); Scott C. Duensing, Smithton, IL (US); Alejandro S. Diaz, Ross, CA (US); William Herrera, Moreno Valley, CA (US); Christian Laursen, Phoenix, AZ (US); Andrew Yumul Huertas, Carson, CA (US); and Danilo Moura Silva, Thousand Oaks, CA (US)
Assigned to Penumbra, Inc., Alameda, CA (US)
Filed by Penumbra, Inc., Alameda, CA (US)
Filed on Jun. 20, 2022, as Appl. No. 17/844,416.
Claims priority of provisional application 63/213,187, filed on Jun. 21, 2021.
Prior Publication US 2022/0407902 A1, Dec. 22, 2022
Int. Cl. G06F 3/01 (2006.01); G06F 3/14 (2006.01); G06T 19/00 (2011.01); H04L 65/1069 (2022.01); H04L 65/1089 (2022.01); H04L 65/1093 (2022.01); H04L 65/403 (2022.01); H04N 7/15 (2006.01)
CPC G06F 3/012 (2013.01) [G06F 3/14 (2013.01); G06T 19/00 (2013.01); H04L 65/1069 (2013.01); H04L 65/1089 (2013.01); H04L 65/1093 (2013.01); H04L 65/403 (2013.01); H04N 7/152 (2013.01); G06T 2219/024 (2013.01)] 15 Claims
OG exemplary drawing
 
1. A method of providing real-time, two-way communication between a virtual reality user, viewing on a virtual reality display device a displayed sequence of video frames comprising the virtual reality user's field of view of a virtual reality environment, and a guest user operating a computing device having a camera and microphone, comprising:
creating, by a processor, a connection between the virtual reality display device and the computing device of the guest user;
capturing, by the processor, the sequence of video frames displayed to the virtual reality user on the virtual reality display device;
generating, by a microphone, an audio signal from the virtual reality user;
obtaining, by the processor, permission from the virtual reality user before creating a connection between the virtual reality display device and the computing device of the guest user;
sending, by the processor, the captured sequence of video frames displayed to the virtual reality user and the generated audio signal from the virtual reality user to the computing device of the guest user;
displaying, by the computing device of the guest user, the captured sequence of video frames;
playing, by the computing device of the guest user, the generated audio signal from the virtual reality user;
capturing, by the camera of the computing device of the guest user, a video stream of the guest user;
generating, by the microphone of the computing device of the guest user, an audio signal from the guest user;
sending, by the computing device of the guest user, the captured video stream of the guest user and the generated audio signal from the guest user to the processor;
displaying, by the processor, the captured video stream of the guest user in a window in the virtual reality environment; and
playing, by the processor, the generated audio signal from the guest user.