US 12,470,889 B2
Future visual frame prediction for autonomous vehicles using long-range acoustic beamforming and synthetic aperture expansion
Felix Heide, Blackburg, VA (US); and Jim Aldon D'Souza, Blacksburg, VA (US)
Assigned to Torc Robotics, Inc., Blacksburg, VA (US)
Filed by Torc Robotics, Inc., Blacksburg, VA (US)
Filed on Dec. 20, 2023, as Appl. No. 18/391,527.
Claims priority of provisional application 63/508,784, filed on Jun. 16, 2023.
Prior Publication US 2024/0416953 A1, Dec. 19, 2024
Int. Cl. H04S 7/00 (2006.01); B60W 60/00 (2020.01); G01S 5/18 (2006.01); H04R 1/40 (2006.01); H04R 3/00 (2006.01)
CPC H04S 7/40 (2013.01) [B60W 60/00 (2020.02); B60W 60/001 (2020.02); G01S 5/18 (2013.01); H04R 1/406 (2013.01); H04S 7/302 (2013.01); B60W 2420/40 (2013.01); B60W 2420/403 (2013.01); B60W 2420/54 (2013.01); B60W 2556/35 (2020.02); B60W 2556/40 (2020.02); H04R 3/005 (2013.01); H04R 2201/401 (2013.01); H04R 2499/13 (2013.01)] 20 Claims
OG exemplary drawing
 
1. An autonomous vehicle, comprising:
a microphone array of a plurality of microphones;
a visual sensor network configured to receive visual signals;
at least one processor; and
at least one memory storing instructions, which, when executed by the at least one processor, cause the at least one processor to:
generate spatial beamforming maps locating a sound source using a beamforming model corresponding to acoustic signals received at the plurality of microphones of the microphone array;
apply a synthetic aperture expansion to the acoustic signals to increase resolution of the spatial beamforming maps; and
generate a future visual frame based at least partially upon temporal information extracted from the spatial beamforming maps and visualization maps generated based on the visual signals received by the visual sensor network.