US 12,436,736 B2
Voice controlled UIs for AR wearable devices
Sharon Moll, Lachen (CH); and Piotr Gurgul, Hergiswil (CH)
Assigned to Snap Inc., Santa Monica, CA (US)
Filed by Snap Inc., Santa Monica, CA (US)
Filed on Dec. 27, 2023, as Appl. No. 18/397,786.
Application 18/397,786 is a continuation of application No. 17/823,169, filed on Aug. 30, 2022, granted, now 11,922,096.
Prior Publication US 2024/0126502 A1, Apr. 18, 2024
This patent is subject to a terminal disclaimer.
Int. Cl. G06F 3/16 (2006.01); G06F 3/04817 (2022.01); G06F 3/0484 (2022.01); G10L 15/16 (2006.01); G10L 15/22 (2006.01)
CPC G06F 3/167 (2013.01) [G06F 3/04817 (2013.01); G06F 3/0484 (2013.01); G10L 15/16 (2013.01); G10L 15/22 (2013.01); G10L 2015/223 (2013.01)] 20 Claims
OG exemplary drawing
 
1. An augmented reality (AR), mixed reality (MR), or virtual reality (VR) wearable device comprising:
one or more processors; and
a memory storing instructions that, when executed by the one or more processors, configure the one or more processors to perform operations comprising:
sending a notification to an application that voice user interface (UI) mode is enabled;
receiving, from the application, an indication of a UI element, the UI element indicating a tag and a UI element type;
retrieving an action based on the UI element type from a data structure associating actions with UI element types;
causing to be displayed on a display of the AR, MR, or VR wearable device, the UI element and a voice UI label for the UI element, the voice UI label comprising the action and the tag, the action indicating how to activate the UI element with the tag;
accessing audio data captured from a microphone of the AR, MR, or VR wearable device, the audio data generated by utterances of a user;
processing the audio data to determine whether the audio data comprises the displayed action and the displayed tag; and
in response to the audio data comprising the displayed action and the displayed tag, sending, to the application, an indication that the UI element is activated.