US 12,073,017 B2
Wearable electronic device for displaying virtual object on a surface of a real object and method of controlling the same
Eunbin Lee, Suwon-si (KR); Eunyoung Park, Suwon-si (KR); Shinjae Jung, Suwon-si (KR); Yoonho Lee, Suwon-si (KR); and Eunkyung Lee, Suwon-si (KR)
Assigned to SAMSUNG ELECTRONICS CO., LTD., Suwon-si (KR)
Filed by SAMSUNG ELECTRONICS CO., LTD., Suwon-si (KR)
Filed on Nov. 2, 2022, as Appl. No. 17/979,306.
Application 17/979,306 is a continuation of application No. PCT/KR2022/095128, filed on Oct. 19, 2022.
Claims priority of application No. 10-2021-0148523 (KR), filed on Nov. 2, 2021; and application No. 10-2022-0004440 (KR), filed on Jan. 12, 2022.
Prior Publication US 2023/0135420 A1, May 4, 2023
Int. Cl. G06F 3/01 (2006.01); G02B 27/00 (2006.01); G02B 27/01 (2006.01); G06F 3/04817 (2022.01); G06N 20/00 (2019.01); G06T 19/00 (2011.01)
CPC G06F 3/013 (2013.01) [G02B 27/0093 (2013.01); G02B 27/0101 (2013.01); G02B 27/017 (2013.01); G02B 27/0179 (2013.01); G06F 3/04817 (2013.01); G06N 20/00 (2019.01); G06T 19/006 (2013.01); G02B 2027/0138 (2013.01); G02B 2027/014 (2013.01); G02B 2027/0178 (2013.01); G02B 2027/0187 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A wearable electronic device comprising:
a display;
a camera;
at least one first sensor;
a processor; and
memory storing instructions that, when executed by the processor, cause the wearable electronic device to:
control to display an execution screen of an application on the display, wherein the execution screen comprises a plurality of contents,
based at least on receiving a first user input, through the camera, for selecting a content from among the plurality of contents, control to display a first virtual object related to the selected content on the display,
based at least on receiving a second user input, through the camera, for moving the first virtual object to an object included in an image of a real space obtained via the camera and mapping the first virtual object to the object included in the image of the real space, identify a type of the object in the image of the real space through the at least one first sensor, and
in response to at least the type of the object in the image of the real space being related to the selected content and/or the first virtual object, change the first virtual object to a second virtual object related to the selected content based at least on at least one of a shape, a position, or a size of the object in the image of the real space, and
in response to at least the type of the object in the image of the real space being not related to the selected content and/or the first virtual object, control to display on the display a third virtual object in a form of a three-dimensional (3D) object related to the selected content, wherein the wearable electronic device changes the first virtual object to the third virtual object based on the third virtual object being different from the first virtual object and displays the first virtual object as the third virtual object based on the third virtual object being a same as the first virtual object,
wherein a type of the second virtual object and a type of the third virtual object are different from each other.