US 12,282,607 B2
Fingerspelling text entry
Austin Vaday, Laguna Nigel, CA (US); Rebecca Jean Lee, Los Angeles, CA (US); and Jennica Pounds, Cape Coral, FL (US)
Assigned to Snap Inc., Santa Monica, CA (US)
Filed by Snap Inc., Santa Monica, CA (US)
Filed on Apr. 27, 2022, as Appl. No. 17/660,965.
Prior Publication US 2023/0350495 A1, Nov. 2, 2023
Int. Cl. G06F 3/01 (2006.01); G02B 27/01 (2006.01); G06T 11/60 (2006.01); G06T 19/00 (2011.01); G06V 20/20 (2022.01); G06V 40/10 (2022.01); G06V 40/20 (2022.01)
CPC G06F 3/017 (2013.01) [G02B 27/0101 (2013.01); G02B 27/017 (2013.01); G06F 3/011 (2013.01); G06T 11/60 (2013.01); G06T 19/006 (2013.01); G06V 20/20 (2022.01); G06V 40/113 (2022.01); G06V 40/28 (2022.01); G02B 2027/0138 (2013.01); G02B 2027/014 (2013.01); G02B 2027/0178 (2013.01)] 21 Claims
OG exemplary drawing
 
1. A computer-implemented method of a text entry process, the method comprising:
providing, by an Augmented Reality (AR) system, to a user of the AR system, a text scene component in a user interface, the text scene component having an apparent location in a real-world scene environment, the text scene component associated with an AR application component;
determining, by one or more processors, a language model based on component registration data of the AR application component, the component registration data including a component ID field identifying the AR application component, a language field identifying the language model, and one or more symbol fields indicating respective one or more symbols to be routed to the AR application component;
detecting, by the one or more processors, using one or more cameras of the AR system and the language model, a start text entry gesture being made by a user of the AR system;
determining, by the one or more processors, that the apparent location of the text scene component is correlated to a location in the real-world scene environment of a hand of the user while the user is making the start text entry gesture;
detecting, by the one or more processors, using the one or more cameras and the language model, one or more input symbols corresponding to fingerspelling signs being made by the user;
routing, by the one or more processors, the one or more input symbols to the AR application component using the one or more input symbols and the one or more symbol fields of the component registration data;
generating, by the one or more processors, entered text data based on the one or more input symbols; and
providing, by the one or more processors, text in the text scene component based on the entered text data.