US 12,449,908 B2
Augmented reality device, and method for controlling augmented reality device
Insun Han, Suwon-si (KR); Doil Kwon, Suwon-si (KR); Jiin Kim, Suwon-si (KR); Jaemin Chun, Suwon-si (KR); and Minseok Han, Suwon-si (KR)
Assigned to SAMSUNG ELECTRONICS CO., LTD., Suwon-si (KR)
Filed by SAMSUNG ELECTRONICS CO., LTD., Suwon-si (KR)
Filed on May 31, 2023, as Appl. No. 18/204,116.
Application 18/204,116 is a continuation of application No. PCT/KR2021/019813, filed on Dec. 24, 2021.
Claims priority of application No. 10-2020-0189713 (KR), filed on Dec. 31, 2020.
Prior Publication US 2023/0305635 A1, Sep. 28, 2023
Int. Cl. G06F 3/01 (2006.01); G06F 3/04847 (2022.01); G06T 11/00 (2006.01); G06V 20/20 (2022.01); G06V 40/20 (2022.01)
CPC G06F 3/017 (2013.01) [G06F 3/04847 (2013.01); G06T 11/00 (2013.01); G06V 20/20 (2022.01); G06V 40/28 (2022.01); G06F 3/013 (2013.01); G06T 2200/24 (2013.01)] 16 Claims
OG exemplary drawing
 
1. An augmented reality (AR) device comprising:
an eye tracking sensor configured to obtain gaze information of a user;
a communication interface;
a camera;
a display engine;
a display;
a memory storing one or more instructions; and
a processor configured to execute the one or more instructions to:
transmit, through the communication interface, gaze coordinates corresponding to an object based on the gaze information to an image processing device that is external to the AR device and displays the object on a screen of the image processing device, wherein the gaze coordinates are correlated with screen coordinates of the object displayed on the screen, wherein a visual effect is applied to the screen coordinates of the object displayed on the screen,
recognize, based on an image captured through the camera in a state in which the visual effect is applied to the screen coordinates of the object displayed on the screen, a first gesture to call a virtual image of the object displayed on the screen of the image processing device,
receive, through the communication interface, object information of the object from the image processing device in response to an event notification transmitted to the image processing device, the event notification corresponding to the recognized first gesture, and
based on the object being a setting user interface,
control the display engine to provide on the display of the AR device, while the object is displayed on the screen of the image processing device, a virtual image of the setting user interface corresponding to the object information among prepared virtual images, the virtual image of the setting user interface provided at a first depth layer of the display,
recognize a second gesture that manipulates a pop-up of the virtual image of the setting user interface to change one or more settings of the setting user interface, and
transmit, to the image processing device, information that causes the image processing device to change the one or more settings in accordance with the second gesture.