US 12,475,912 B2
Augmented reality device performing audio recognition and control method therefor
Ilyoung Jeong, Seoul (KR); Hyungui Lim, Seoul (KR); Yoonchang Han, Seoul (KR); Subin Lee, Seoul (KR); Jeongsoo Park, Suwon-si (KR); and Donmoon Lee, Seoul (KR)
Assigned to Cochl Inc, Dover, DE (US)
Appl. No. 17/911,637
Filed by Cochl Inc, Dover, DE (US)
PCT Filed Feb. 26, 2021, PCT No. PCT/KR2021/002497
§ 371(c)(1), (2) Date Sep. 14, 2022,
PCT Pub. No. WO2021/187771, PCT Pub. Date Sep. 23, 2021.
Claims priority of application No. 10-2020-0034136 (KR), filed on Mar. 20, 2020.
Prior Publication US 2023/0145966 A1, May 11, 2023
Int. Cl. G10L 25/48 (2013.01); G10L 15/06 (2013.01)
CPC G10L 25/48 (2013.01) [G10L 15/06 (2013.01)] 5 Claims
OG exemplary drawing
 
1. An augmented reality device comprising:
a see-through display configured to allow a user's eyes to see a virtual object therethrough and output image information of the virtual object;
an audio input device configured to receive an audio signal generated at a preset distance or less from the see-through display;
a camera installed on a part of the see-through display and configured to perform photographing in one direction; and
a controller configured to identify event information corresponding to the audio signal through an artificial neural network and control an operation of the see-through display to output image information of the virtual object corresponding to the identified event information,
wherein the controller identifies at least one object included in image information taken by the camera through the artificial neural network corresponding to a type or situation of the audio signal,
wherein, when the audio signal is identified as a certain type of event information and the virtual object of the certain type corresponding to that of the event information is identified from the image information, the controller matches the audio signal and an image signal to each other such that the image information of the virtual object overlaps the image signal corresponding to a position on a screen of the see-through display on which the audio signal is generated, and
wherein, when a position of a point at which the audio signal is generated is outside a visible area that is visible to a user through the see-through display, the controller controls an operation of the see-through display to output image information associated with the position and a direction of the point.