US 12,423,917 B2
Extended reality based digital assistant interactions
Lynn I. Streja, San Francisco, CA (US); Keith P. Avery, Seattle, WA (US); Karan M. Daryanani, San Francisco, CA (US); Stephen O. Lemay, Palo Alto, CA (US); Myra C. Lukens, San Francisco, CA (US); Sreeneel K. Maddika, San Ramon, CA (US); Chaitanya Mannemala, San Ramon, CA (US); Aswath Manoharan, Tucson, AZ (US); Pedro Mari, Santa Cruz, CA (US); Jay Moon, San Francisco, CA (US); and Abhishek Rawat, Fremont, CA (US)
Assigned to Apple Inc., Cupertino, CA (US)
Filed by Apple Inc., Cupertino, CA (US)
Filed on May 26, 2023, as Appl. No. 18/202,849.
Claims priority of provisional application 63/351,195, filed on Jun. 10, 2022.
Prior Publication US 2023/0401795 A1, Dec. 14, 2023
Int. Cl. G06T 19/00 (2011.01); G06F 3/01 (2006.01); G06T 13/40 (2011.01); G10L 15/22 (2006.01)
CPC G06T 19/003 (2013.01) [G06F 3/013 (2013.01); G06F 3/017 (2013.01); G06T 13/40 (2013.01); G10L 15/22 (2013.01); G06T 2219/2012 (2013.01); G06T 2219/2016 (2013.01); G06T 2219/2021 (2013.01); G10L 2015/223 (2013.01)] 58 Claims
OG exemplary drawing
 
1. A non-transitory computer-readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by one or more processors of an electronic device with one or more sensors, cause the electronic device to:
while displaying a portion of an extended reality (XR) environment representing a current field of view of a user of the electronic device:
detect, with the one or more sensors, a user gaze at a first object displayed in the XR environment, wherein the first object is persistent in the current field of view of the XR environment;
in response to detecting the user gaze at the first object, expand the first object into a list of objects, wherein the list of objects includes a second object representing a digital assistant;
detect, with the one or more sensors, a user gaze at the second object;
in accordance with detecting the user gaze at the second object, display a first animation of the second object indicating that a first digital assistant session is initiated;
receive a first audio input from the user of the electronic device; and
display a second animation of the second object indicating that the digital assistant is actively listening to the user in response to receiving the first audio input, wherein the first animation is different from the second animation of the second object.