US 12,010,493 B1
Visualizing spatial audio
Willy Wai-Lik Wong, Tokyo (JP); Marielle Venita Jakobsons, Oakland, CA (US); and Nikhil Ratnesh Javeri, Santa Clara, CA (US)
Assigned to EMBODYVR, INC., San Mateo, CA (US)
Filed by EmbodyVR, Inc., San Mateo, CA (US)
Filed on Sep. 3, 2020, as Appl. No. 17/011,710.
Claims priority of provisional application 62/934,940, filed on Nov. 13, 2019.
Int. Cl. H04R 5/027 (2006.01); G06F 16/68 (2019.01); H04R 5/04 (2006.01)
CPC H04R 5/027 (2013.01) [G06F 16/68 (2019.01); H04R 5/04 (2013.01)] 21 Claims
OG exemplary drawing
 
1. A method comprising:
for a first auditory spatial cue decoded from a spatialized audio signal,
determining which of a plurality of positions indicated in a map of audio source positions corresponds to the first auditory spatial cue;
calculating a value of a first visual feature based on an audio intensity value indicated in the first auditory spatial cue;
generating a first visual spatial cue based on the calculated value for the first visual feature and the position of the plurality of positions determined as corresponding to the first auditory spatial cue;
communicating the first visual spatial cue for rendering the first visual spatial cue around a listener displayed in a graphical user interface in coordination with rendering of the first auditory spatial cue and corresponding audio with an audio output device of the listener; and
synchronizing communicating of the first visual spatial cue for rendering with communicating the first auditory spatial cue for rendering, wherein the communicating of the first visual spatial cue for rendering and the communicating the first auditory spatial cue for rendering are synchronized using buffering;
wherein the spatialized audio signal comprises object-based audio and determining which of the plurality of positions indicated in the map of audio source positions corresponds to the first auditory spatial cue comprises determining spatial information from metadata decoded from the spatialized audio signal and determining the corresponding position based on the metadata indicated in the first auditory spatial cue.