US 11,914,146 B2
Methods and systems for adding real-world sounds to virtual reality scenes
Victoria Dorn, San Mateo, CA (US); Celeste Bean, San Francisco, CA (US); Tatianna Manzon-Gutzman, San Mateo, CA (US); and Sarah Karp, Foster City, CA (US)
Assigned to Sony Interactive Entertainment LLC, San Mateo, CA (US); and Sony Interactive Entertainment Inc., Tokyo (JP)
Filed by Sony Interactive Entertainment LLC, San Mateo, CA (US); and Sony Interactive Entertainment Inc., Tokyo (JP)
Filed on May 27, 2022, as Appl. No. 17/827,462.
Prior Publication US 2023/0384592 A1, Nov. 30, 2023
Int. Cl. G06F 3/16 (2006.01); G02B 27/01 (2006.01); G06F 3/01 (2006.01); H04S 7/00 (2006.01)
CPC G02B 27/017 (2013.01) [G06F 3/011 (2013.01); G06F 3/16 (2013.01); H04S 7/304 (2013.01)] 13 Claims
OG exemplary drawing
 
1. A method for integrating media cues into virtual reality scenes presented on a head mounted display (HMD), comprising:
presenting a virtual reality scene on a display of an HMD, the HMD includes headphones for outputting audio related to the virtual reality space and the headphones use a noise canceling process for reducing the sound from a real-world space;
receiving sensor data from one or more sensors in a real-world space in which the HMD is located;
identifying an object location of an object in the real-world space that produces a sound using the sensor data; and
generating a media cue in the virtual reality scene presented in the HMD, the media cue is presented in a virtual location that is correlated to the object location of the object in the real-world space, the media cue is rendered as audio data associated with the sound that is captured from the real-world space and is mixed with audio related to the virtual reality space when output via said headphones;
wherein interactivity and audio related to the virtual reality scene is active when the media cue is presented.