US 11,714,157 B2
System to determine direction toward user
Carlos Renato Nakagawa, San Jose, CA (US); Mohamed Mansour, Cupertino, CA (US); Borham Lee, Sunnyvale, CA (US); and Shobha Devi Kuruba Buchannagari, Fremont, CA (US)
Assigned to AMAZON TECHNOLOGIES, INC., Seattle, WA (US)
Filed by AMAZON TECHNOLOGIES, INC., Seattle, WA (US)
Filed on Feb. 12, 2021, as Appl. No. 17/174,941.
Claims priority of provisional application 63/111,930, filed on Nov. 10, 2020.
Prior Publication US 2022/0146617 A1, May 12, 2022
Int. Cl. G01S 3/80 (2006.01); G01S 3/786 (2006.01); H04R 1/40 (2006.01); H04R 3/00 (2006.01)
CPC G01S 3/8003 (2013.01) [G01S 3/7864 (2013.01); H04R 1/406 (2013.01); H04R 3/005 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A device comprising:
a camera;
a microphone array;
one or more memories storing first computer-executable instructions; and
one or more processors to execute the first computer-executable instructions to:
acquire sound data using the microphone array;
determine, based on the sound data, utterance of a wakeword;
determine, based on the sound data, first data comprising:
a plurality of sound direction values, wherein each sound direction value is an estimated direction of the utterance with respect to the device, and
a plurality of first variance values, wherein each first variance value is associated with each of the sound direction values;
acquire image data using the camera;
determine second data based at least in part on the image data;
determine third data comprising:
the plurality of sound direction values, and
a plurality of second variance values that are based on the plurality of first variance values as modified based on the second data;
determine direction data based on the third data; and
operate the device based on the direction data.