US 11,995,774 B2
Augmented reality experiences using speech and text captions
Ilteris Canberk, Marina Del Rey, CA (US); Shin Hwun Kang, Los Angeles, CA (US); and Daniel Moreno, Santa Monica, CA (US)
Assigned to Snap Inc., Santa Monica, CA (US)
Filed by Ilteris Canberk, Marina Del Rey, CA (US); Shin Hwun Kang, Los Angeles, CA (US); and Daniel Moreno, Santa Monica, CA (US)
Filed on Dec. 18, 2020, as Appl. No. 17/126,207.
Claims priority of provisional application 63/045,537, filed on Jun. 29, 2020.
Prior Publication US 2021/0407203 A1, Dec. 30, 2021
Int. Cl. G06T 19/00 (2011.01); G02B 27/01 (2006.01); G06F 3/01 (2006.01); G10L 15/26 (2006.01)
CPC G06T 19/006 (2013.01) [G02B 27/0176 (2013.01); G06F 3/013 (2013.01); G06F 3/017 (2013.01); G10L 15/26 (2013.01); G02B 2027/0178 (2013.01)] 12 Claims
OG exemplary drawing
 
1. An augmented reality system comprising:
an image capture system;
a display system;
a speech recognition system;
an eyewear device comprising the image capture system, the display system, a processor, and a memory; and
programming in the memory, wherein execution of the programming by the processor configures the eyewear device to perform functions, including functions to:
capture an image of a physical environment using the image capture system;
acquire text from a user input, a data input, or the speech recognition system;
generate, using the display system, visual text graphics from the acquired text;
set, using the display system, the visual text graphics at a predefined location relative to one or more items in the physical environment in response to a first gaze direction of the user;
move, using the display system, the visual text graphics in the image from the predefined location relative to the one or more items in the physical environment in response to a second gaze direction or a hand gesture of the user; and
transmit, by the eyewear device, the image of the physical environment including the set visual text graphics,
wherein execution of the programming by the processor configures the eyewear device to enable manipulation of font style or color of the visual text graphics by arranging attributes of the visual text graphic as a menu of options that are displayed on the display system based on characteristics of the physical environment.