US 12,474,783 B1
Face-tracking and gesture recognition for hands-free computer control
Chetan Jaiswal, Hamden, CT (US); Michael Ruocco, Hamden, CT (US); Jack Duggan, Hamden, CT (US); Brian O'Neill, Hamden, CT (US); and Karen Majeski, Hamden, CT (US)
Assigned to Quinnipiac University, Hamden, CT (US)
Filed by Quinnipiac University, Hamden, CT (US)
Filed on Oct. 22, 2024, as Appl. No. 18/922,718.
Int. Cl. G06F 3/01 (2006.01); G06F 3/04847 (2022.01); G06K 9/00 (2022.01); G06K 9/62 (2022.01); G06V 40/16 (2022.01); G06V 40/18 (2022.01); G06V 40/20 (2022.01)
CPC G06F 3/017 (2013.01) [G06F 3/012 (2013.01); G06F 3/013 (2013.01); G06F 3/04847 (2013.01); G06V 40/171 (2022.01); G06V 40/193 (2022.01); G06V 40/20 (2022.01)] 29 Claims
OG exemplary drawing
 
1. A method of computer operation, comprising:
accessing video of a computer user's face acquired by a camera input of a computer, wherein the video comprises a plurality of sequential frames;
identifying, from the plurality of sequential frames, facial landmarks of at least a first eye, a second eye, and a nose of the user;
monitoring, from at least two sequential frames, positional changes of facial landmarks of:
the first and second eyes to detect one or more of a closure of the first eye only, a closure of the second eye only, or a closure of both eyes, respectively defining independent input functions to the computer, and
the nose to detect head movement of the user, wherein head movement of the user includes one or more head articulations defining one or more input functions to the computer; and
operating the computer in at least two modes recognizing different inputs from the user, the at least two modes comprising:
a cursor-active mode enabling movement of a cursor on the screen of the computer in response to head movement of the user and disabling the one or more input functions associated with the one or more head articulations, and
a cursor-inactive mode disabling movement of the cursor on the screen of the computer in response to head movement of the user and enabling the one or more input functions associated with the one or more head articulations.