US 12,093,462 B2
Virtual keyboard selections using multiple input modalities
Aaron Faucher, Seattle, WA (US); Pol Pla I Conesa, San Francisco, CA (US); Daniel Rosas, Auburn, WA (US); and Nathan Aschenbach, Seattle, WA (US)
Assigned to Meta Platforms Technologies, LLC, Menlo Park, CA (US)
Filed by Meta Platforms Technologies, LLC, Menlo Park, CA (US)
Filed on Apr. 11, 2022, as Appl. No. 17/717,585.
Prior Publication US 2023/0324997 A1, Oct. 12, 2023
Int. Cl. G06F 3/01 (2006.01); G06F 3/04886 (2022.01)
CPC G06F 3/017 (2013.01) [G06F 3/013 (2013.01); G06F 3/04886 (2013.01); G06F 3/011 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A method for triggering virtual keyboard selections using multiple input modalities, the method comprising:
displaying, to a user, a virtual keyboard in an augmented reality environment;
tracking a) user gaze input in relation to the virtual keyboard and b) user hand motion, wherein the virtual keyboard is displayed to appear at a distance greater than 0.5 meters from the user within the augmented reality environment;
detecting a first instance when the tracked user hand motion meets a velocity criteria;
in response to the detecting the first instance when the tracked user hand motion meets the velocity criteria, resolving a first character selection according to the user gaze input, in relation to the virtual keyboard, at a time that corresponds to the detected first instance when the user hand motion meets the velocity criteria,
wherein the resolving the first character selection is not based on a position of the user's hand in relation to the virtual keyboard;
detecting a second instance when the tracked user hand motion meets the velocity criteria;
in response to the detecting the second instance when the tracked user hand motion meets the velocity criteria, resolving a second character selection according to the user gaze input, in relation to the virtual keyboard, at a time that corresponds to the detected second instance when the user hand motion meets the velocity criteria; and
generating a character sequence comprising the resolved first and second character selections.