US 12,216,832 B2
Gesture recognition based on likelihood of interaction
Julia Schwarz, Bellevue, WA (US); Bugra Tekin, Zurich (CH); Sophie Stellmach, Kirkland, WA (US); Erian Vazquez, Redmond, WA (US); Casey Leon Meekhof, Redmond, WA (US); and Fabian Gobel, Zurich (CH)
Assigned to Microsoft Technology Licensing, LLC, Redmond, WA (US)
Filed by Microsoft Technology Licensing, LLC, Redmond, WA (US)
Filed on Sep. 8, 2023, as Appl. No. 18/463,906.
Application 18/463,906 is a continuation of application No. 17/649,659, filed on Feb. 1, 2022, granted, now 11,768,544.
Prior Publication US 2023/0418390 A1, Dec. 28, 2023
This patent is subject to a terminal disclaimer.
Int. Cl. G06F 3/01 (2006.01); G06N 3/08 (2023.01); G06V 10/82 (2022.01); G06V 40/20 (2022.01)
CPC G06F 3/017 (2013.01) [G06F 3/013 (2013.01); G06N 3/08 (2013.01); G06V 10/82 (2022.01); G06V 40/28 (2022.01)] 20 Claims
OG exemplary drawing
 
1. A method for evaluating gesture input, comprising:
receiving input data for sequential data frames, the input data including at least hand tracking data for one or more hands of a user;
at a first neural network trained to recognize features indicative of a plurality of subsequent gesture interactions:
evaluating input data for a sequence of two or more data frames; and
outputting an indication of a likelihood of the user performing one or more subsequent gesture interactions with a user interface during a predetermined window of one or more data frames; and
at a second neural network trained to recognize features indicative of whether the user is currently performing one or more of the plurality of subsequent gesture interactions:
adjusting parameters for gesture interaction recognition during the predetermined window based on the indicated likelihood of the user performing one or more of the plurality of subsequent gesture interactions and based at least on one or more heuristic evaluations of the received input data for the predetermined window;
evaluating at least the predetermined window for performed gesture interactions based on the adjusted parameters; and
outputting a signal as to whether the user is performing one or more gesture interactions with the user interface during the predetermined window.