US 12,086,324 B2
Micro hand gestures for controlling virtual and graphical elements
Ilteris Canberk, Marina Del Rey, CA (US); Viktoria Hwang, Los Angeles, CA (US); Shin Hwun Kang, Los Angeles, CA (US); David Meisenholder, Los Angeles, CA (US); and Daniel Moreno, Santa Monica, CA (US)
Assigned to Snap Inc., Santa Monica, CA (US)
Filed by Snap Inc., Santa Monica, CA (US)
Filed on Dec. 14, 2021, as Appl. No. 17/550,679.
Claims priority of provisional application 63/131,403, filed on Dec. 29, 2020.
Prior Publication US 2022/0206588 A1, Jun. 30, 2022
Int. Cl. G06F 3/04815 (2022.01); G02B 27/01 (2006.01); G06F 3/01 (2006.01); G06F 3/04847 (2022.01); G06T 19/00 (2011.01); G06V 20/40 (2022.01); G06V 40/10 (2022.01); G06V 40/20 (2022.01)
CPC G06F 3/017 (2013.01) [G02B 27/0101 (2013.01); G02B 27/017 (2013.01); G06F 3/04815 (2013.01); G06F 3/04847 (2013.01); G06T 19/006 (2013.01); G06V 20/46 (2022.01); G06V 40/113 (2022.01); G06V 40/28 (2022.01); G02B 2027/0138 (2013.01); G02B 2027/0178 (2013.01)] 17 Claims
OG exemplary drawing
 
1. A method of controlling virtual elements in response to hand gestures detected with an eyewear device, the eyewear device comprising a camera system, an image processing system, and a display, the method comprising:
capturing frames of video data within a field of view associated with the camera system;
detecting a series of hand shapes in the captured frames of video data with the image processing system, wherein the detected series of hand shapes includes a thumb near an extended finger;
determining whether the detected series of hand shapes matches a predefined series of hand gestures associated with an action;
establishing a finger scale relative to the extended finger, the finger scale extending lengthwise from a palmar end through a midpoint to a tip end;
calibrating a graphical scale relative to the display with the finger scale, wherein graphical scale is associated with both a virtual element and an interactive graphical element;
identifying, in the frames of video data with the image processing system, a current thumb position relative to the finger scale;
measuring a gap distance between the identified current thumb position and the midpoint of the finger scale;
presenting the interactive graphical element on the display at a series of locations in accordance with the identified current thumb position, the calibrated graphical scale, and the gap distance, such that the interactive graphical element appears to move through the series of locations at a speed correlated with the gap distance; and
controlling movement of the virtual element relative to the display in accordance with the associated action and according to the identified current thumb position relative to the calibrated graphical scale.