US 12,135,841 B2
Gesture interaction method and system based on artificial reality
Fei Sun, Beijing (CN); Haitao Yu, Beijing (CN); Tao Wu, Beijing (CN); and Achuan Li, Beijing (CN)
Assigned to QINGDAO PICO TECHNOLOGY CO., LTD., Qingdao (CN)
Filed by Qingdao Pico Technology Co., Ltd., Shandong (CN)
Filed on Dec. 29, 2023, as Appl. No. 18/400,624.
Application 18/400,624 is a continuation of application No. PCT/CN2022/081494, filed on Mar. 17, 2022.
Claims priority of application No. 202110926659.X (CN), filed on Aug. 12, 2021.
Prior Publication US 2024/0134461 A1, Apr. 25, 2024
Prior Publication US 2024/0231503 A9, Jul. 11, 2024
Int. Cl. G06F 3/01 (2006.01); G06F 3/00 (2006.01); G06F 3/04815 (2022.01); G06F 3/04842 (2022.01); G06F 3/0486 (2013.01)
CPC G06F 3/017 (2013.01) [G06F 3/005 (2013.01); G06F 3/011 (2013.01); G06F 3/04815 (2013.01); G06F 3/04842 (2013.01); G06F 3/0486 (2013.01)] 10 Claims
OG exemplary drawing
 
1. A gesture interaction method based on artificial reality, comprising:
determining a ray formed from a predetermined position of a hand to a finger joint of the hand in a state in which four fingers of the hand form a first as an operation indication line, an end point of the ray being the predetermined position of the hand;
determining a point of intersection between the operation indication line and a display surface in the artificial reality as a position of an operation icon in the artificial reality;
controlling the operation icon to move correspondingly in response to detecting that the four fingers of the hand form the first and move, to move the operation icon in the display surface to a target key; and
determining, in response to detecting an action of a thumb of the hand touching an index finger of the hand, that the operation icon clicks on the target key to complete an interaction,
wherein said determining, in response to detecting the action of the thumb of the hand touching the index finger of the hand, that the operation icon clicks on the target key to complete the interaction comprises:
determining an index finger joint plane based on joint positions of the index finger;
determining a distance between a distal finger pulp of the thumb and the index finger joint plane as a thumb-index-finger spacing;
controlling the operation icon to click on the target key, in response to detecting that the distal finger pulp of the thumb approaches the index finger joint plane and the thumb-index-finger spacing smaller than a predetermined clicking distance threshold; and
in response to detecting that the thumb is lifted, the distal finger pulp gets away from the index finger joint plane, and the thumb-index-finger spacing is greater than a predetermined effective distance threshold, controlling to complete the action of the operation icon clicking on the target key, and enabling the action of the operation icon clicking on the target key to become effective, to complete one interactive action,
wherein the index finger joint plane comprises all points covered by a plane formed between a distal finger tip of the index finger and an interphalangeal joint of the index finger, between two interphalangeal joints of the index finger, and between an interphalangeal joint and a metacarpal joint of the index finger.