US 11,721,342 B2
Multi-modal interaction with intelligent assistants in voice command devices
Jeffrey C. Olson, Woodside, CA (US); Henry N. Holtzman, San Francisco, CA (US); Jean-David Hsu, San Francisco, CA (US); and Jeffrey A. Morgan, Napa, CA (US)
Assigned to SAMSUNG ELECTRONICS CO., LTD., Suwon-si (KR)
Filed by SAMSUNG ELECTRONICS CO., LTD., Suwon-si (KR)
Filed on Sep. 22, 2022, as Appl. No. 17/950,668.
Application 17/950,668 is a continuation of application No. 16/367,063, filed on Mar. 27, 2019, granted, now 11,482,215.
Prior Publication US 2023/0027433 A1, Jan. 26, 2023
This patent is subject to a terminal disclaimer.
Int. Cl. G10L 15/22 (2006.01); G10L 15/08 (2006.01)
CPC G10L 15/22 (2013.01) [G10L 15/08 (2013.01); G10L 2015/088 (2013.01); G10L 2015/223 (2013.01); G10L 2015/228 (2013.01)] 18 Claims
OG exemplary drawing
 
1. An electronic device, comprising:
a display;
a microphone;
a speaker;
a sensor; and
at least one processor coupled to the display, the microphone, the speaker, and the sensor, and configured to:
based on an occurrence of a first event that activates a speech recognition function of the electronic device, provide a first feedback via at least one of the display or the speaker, the first event comprising identifying a wake word based on a first speech signal obtained via the microphone;
based on a voice command not being identified in a second speech signal obtained via the microphone, after providing the first feedback, provide a speech response via the speaker;
based on an occurrence of a second event that activates the speech recognition function of the electronic device, provide a second feedback via at least one of the display or the speaker, the second event comprising detecting an external object within a proximity of the electronic device based on a second signal obtained via the sensor, and the second feedback being different from the first feedback; and
based on a voice command not being identified in a third speech signal obtained via the microphone, after providing the second feedback, provide a visual response via the display.