| CPC G06F 3/04886 (2013.01) | 16 Claims |
|
1. An electronic apparatus comprising:
a camera;
a display;
a memory configured to store at least one instruction; and
a processor configured to execute the at least one instruction to:
detect a plurality of fingers in a plurality of first image frames obtained through the camera, obtain depth information regarding the plurality of detected fingers through the camera, identify a plurality of feature points corresponding to respective fingers in a predetermined three-dimensional coordinate space based on the depth information regarding the plurality of detected fingers,
identify whether a pose of the plurality of detected fingers corresponds to a trigger pose which is set as a pose in which at least one finger of the plurality of fingers are apart by a preset distance from a first plane of the predetermined three-dimensional coordinate space, based on the identified the plurality of feature points corresponding to respectively fingers,
in response to an identification that the pose of the plurality of detected fingers corresponds to the trigger pose, set one of the plurality of feature points as a reference point corresponding to a respective finger, and enter a character input mode,
detect a first motion of a finger among the plurality of fingers in a plurality of second image frames obtained through the camera in the character input mode,
identify a distance between a position of the finger by the first motion and a position of the reference point set to the finger in the predetermined three-dimensional coordinate space,
identify a key corresponding to the first motion, from among a plurality of keys mapped to the finger according to a predetermined mapping between the plurality of keys and the finger, based on the identified distance, the position of the finger by the first motion and the reference point set to the finger,
control the display to display information corresponding to the identified key, and
detect the plurality of fingers in a plurality of third image frames obtained through the camera in the character input mode and, in response to an identification that the pose of the plurality of detected fingers in the plurality of third image frames corresponds to a predetermined pose, re-set a reference point for respective fingers based on a position of the plurality of detected fingers in the plurality of third image frames,
wherein the plurality of keys mapped to the finger comprises a reference key, determined based on a type of a hand including the finger and a type of the finger, among keys included in a virtual keyboard and at least two keys adjacent to the reference key among the keys included in the virtual keyboard,
wherein the processor is further configured to execute the at least one instruction to:
identify the reference key by mapping the reference key to a position of the reference point corresponding to the each finger in the predetermined three-dimensional coordinate space,
based on the identified distance being less than a predetermined value, identify a reference key, among the plurality of keys mapped to the finger, as the key corresponding to the first motion, and
based on the identified distance being greater than or equal to the predetermined value, identify a key positioned in a direction in which the finger moves from the reference key, from among the plurality of keys mapped to the finger, as the key corresponding to the first motion based on a moving direction of the finger by the first motion,
wherein the processor is further configured to execute the at least one instruction to, in response to the identification that the pose of the plurality of fingers, in the predetermined three-dimensional coordinate space distinct from the display, detected in the plurality of first image frames corresponds to the trigger pose, control to display a virtual keyboard on the display.
|