US 12,293,759 B2
Method and device for presenting a CGR environment based on audio data and lyric data
Ian M. Richter, Los Angeles, CA (US)
Assigned to APPLE INC., Cupertino, CA (US)
Filed by Apple Inc., Cupertino, CA (US)
Filed on Oct. 31, 2023, as Appl. No. 18/385,540.
Application 18/385,540 is a continuation of application No. 16/867,947, filed on May 6, 2020, granted, now 11,842,729.
Claims priority of provisional application 62/844,867, filed on May 8, 2019.
Prior Publication US 2024/0071377 A1, Feb. 29, 2024
Int. Cl. G10L 13/00 (2006.01); G06F 3/01 (2006.01); G06F 16/683 (2019.01); G06N 20/00 (2019.01); G10L 15/18 (2013.01); G10L 21/10 (2013.01); G10L 25/63 (2013.01)
CPC G10L 15/1815 (2013.01) [G06F 3/011 (2013.01); G06F 16/685 (2019.01); G06N 20/00 (2019.01); G10L 21/10 (2013.01); G10L 25/63 (2013.01)] 27 Claims
OG exemplary drawing
 
14. A device comprising:
one or more processors;
a non-transitory memory;
a speaker;
a display; and
one or more programs stored in the non-transitory memory, which, when executed by the one or more processors, cause the device to:
obtain a first audio file and a second audio file;
parse the first audio file into a plurality of first segments;
parse the second audio file into a plurality of second segments;
generate, for each of the plurality of first segments and each of the plurality of second segments, segment metadata;
determine a relationship between first segment metadata of one of the plurality of first segments and second segment metadata of one of the plurality of second segments;
generate computer-generated reality (CGR) content associated with the one of the plurality of first segments and the one of the plurality of second segments based on the relationship, the first segment metadata, and the second segment metadata; and
display the CGR content on the display by overlaying a virtual object onto a pass-through representation of a physical environment of the device when the device is concurrently playing the one of the plurality of first segments and the one of the plurality of second segments via the speaker.