US 12,112,097 B2
Facilitating discovery of verbal commands using multimodal interfaces
Lubomira Dontcheva, Seattle, WA (US); Arjun Srinivasan, Atlanta, GA (US); Seth John Walker, Oakland, CA (US); and Eytan Adar, Ann Arbor, MI (US)
Assigned to Adobe Inc., San Francisco, CA (US)
Filed by Adobe Inc., San Jose, CA (US)
Filed on Jan. 9, 2023, as Appl. No. 18/152,055.
Application 18/152,055 is a continuation of application No. 17/469,519, filed on Sep. 8, 2021, granted, now 11,573,764.
Application 17/469,519 is a continuation of application No. 16/355,599, filed on Mar. 15, 2019, granted, now 11,132,174.
Prior Publication US 2023/0161553 A1, May 25, 2023
This patent is subject to a terminal disclaimer.
Int. Cl. G06F 3/16 (2006.01); G06F 3/0481 (2022.01); G06F 3/0482 (2013.01); G06F 3/0483 (2013.01); G06F 3/0487 (2013.01); G06F 9/451 (2018.01); G06F 16/68 (2019.01); G06F 16/9032 (2019.01); G10L 15/22 (2006.01)
CPC G06F 3/167 (2013.01) [G06F 3/0481 (2013.01); G06F 3/0482 (2013.01); G06F 3/0483 (2013.01); G06F 3/0487 (2013.01); G06F 9/451 (2018.02); G06F 9/453 (2018.02); G06F 16/686 (2019.01); G06F 16/90324 (2019.01); G10L 15/22 (2013.01); G06F 2203/0381 (2013.01); G10L 2015/223 (2013.01)] 18 Claims
OG exemplary drawing
 
1. A computing system comprising:
one or more processors; and
one or more computer storage media storing computer-useable instructions that, when used by the one or more processors, cause the one or more processors to:
select an operation relevant to a determined target associated with a direct-manipulation input of a multimodal interface;
generate, using a phrasing template selected based on a quantity of modifiable parameters in the phrasing template, a verbal command suggestion relevant to the selected operation for the determined target; and
cause display, via the multimodal interface, of the verbal command suggestion presented adjacent to the determined target associated with the direct-manipulation input of the multimodal interface.
 
9. A computer-implemented method, comprising:
identifying a direct-manipulation input of a multimodal interface indicating selection to present a list of available operations and corresponding verbal command suggestions; and
causing display, via the multimodal interface, of the list of available operations and a set of verbal command suggestions corresponding with each operation in the list of available operations, wherein, based on contextual information, operations and verbal command suggestions unapplicable to an active state of the multimodal interface are visually deemphasized while displayed.