US 12,300,236 B2
Multi-modal interaction with intelligent assistants in voice command devices
Jeffrey C. Olson, Woodside, CA (US); Henry N. Holtzman, San Francisco, CA (US); Jean-David Hsu, San Francisco, CA (US); and Jeffrey A. Morgan, Napa, CA (US)
Assigned to SAMSUNG ELECTRONICS CO., LTD., Suwon-si (KR)
Filed by SAMSUNG ELECTRONICS CO., LTD., Suwon-si (KR)
Filed on Jun. 9, 2023, as Appl. No. 18/332,479.
Application 18/332,479 is a continuation of application No. 17/950,668, filed on Sep. 22, 2022, granted, now 11,721,342.
Application 17/950,668 is a continuation of application No. 16/367,063, filed on Mar. 27, 2019, granted, now 11,482,215, issued on Oct. 25, 2022.
Prior Publication US 2023/0335133 A1, Oct. 19, 2023
This patent is subject to a terminal disclaimer.
Int. Cl. G10L 15/22 (2006.01); G10L 15/08 (2006.01)
CPC G10L 15/22 (2013.01) [G10L 15/08 (2013.01); G10L 2015/088 (2013.01); G10L 2015/223 (2013.01); G10L 2015/228 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A method of operating an electronic device comprising a display, a microphone, a speaker, and at least one sensor, the method comprising:
based on a first occurrence of a first event, providing a first feedback via at least one of the display or the speaker, the first event comprising identifying a wake word from a first speech signal obtained via the microphone;
generating a first timeout based on no speech signal being obtained via the microphone within a first time duration after providing the first feedback;
based on a first occurrence of a second event, providing a second feedback via at least one of the display or the speaker, the second event comprising detecting, via the at least one sensor, an external object within a proximity of the electronic device, and the second feedback being different from the first feedback; and
generating a second timeout based on no speech signal being obtained via the microphone within a second time duration after providing the second feedback,
wherein the method further comprising:
generating a third timeout based on no voice command being identified from a second speech signal obtained via the microphone within a third time duration after a second occurrence of the first event; and
generating a fourth timeout based on no voice command being identified from a third speech signal obtained via the microphone within a fourth time duration after a second occurrence of the second event.