US 12,452,390 B2
Word flow annotation
Jeffrey Scott Sommers, Mountain View, CA (US); Jennifer M. R. Devine, Plantation, FL (US); Joseph Wayne Seuck, Tamarac, FL (US); and Adrian Kaehler, Los Angeles, CA (US)
Assigned to MAGIC LEAP, INC., Plantation, FL (US)
Filed by Magic Leap, Inc., Plantation, FL (US)
Filed on Jun. 27, 2022, as Appl. No. 17/850,860.
Application 17/850,860 is a continuation of application No. 16/791,855, filed on Feb. 14, 2020, granted, now 11,423,909.
Application 16/791,855 is a continuation of application No. 15/673,894, filed on Aug. 10, 2017, granted, now 10,607,609, issued on Mar. 31, 2020.
Claims priority of provisional application 62/374,183, filed on Aug. 12, 2016.
Prior Publication US 2022/0327281 A1, Oct. 13, 2022
Int. Cl. G10L 15/26 (2006.01); G06F 40/169 (2020.01); G06F 40/242 (2020.01); G06F 40/58 (2020.01); G06V 20/20 (2022.01); G10L 15/18 (2013.01); G10L 25/84 (2013.01); H04N 7/15 (2006.01); G10L 15/08 (2006.01); H04N 7/14 (2006.01)
CPC H04N 7/157 (2013.01) [G06F 40/169 (2020.01); G06F 40/242 (2020.01); G06F 40/58 (2020.01); G06V 20/20 (2022.01); G10L 15/1815 (2013.01); G10L 15/26 (2013.01); G10L 25/84 (2013.01); G10L 2015/088 (2013.01); H04N 7/142 (2013.01)] 19 Claims
OG exemplary drawing
 
1. A method of identifying a thread in a text stream, the method comprising:
identifying a first audio stream at an augmented reality device associated with a first user;
receiving a second audio stream at the augmented reality device;
generating a first text stream from the first audio stream and a second text stream from the second audio stream;
identifying one or more keywords associated with both the first and second text streams, wherein the identifying the one or more keywords associated with the both the first and second text streams includes:
detecting a verbal repeating of the first word by the first user from the first or second text stream and designating the first word as one of the one or more keywords;
identifying an identity of a speaker of the second audio stream;
in response to identifying a plurality of keywords associated with both the first and second text streams, identifying a topic associated with each of the one or more keywords;
in response to identifying more than one unique topic, determining one or more groups of topics based at least in part on whether any of the more than one unique topics are related;
in response to determining more than one group of topics, generating a thread associated with each of the group of topics;
causing at least one of the generated threads to be rendered by the augmented reality device;
determining a context of each of the plurality of keywords based at least in part on the identity of the speaker of the second audio stream;
retrieving auxiliary information associated with the keywords based on the respective context; and
rendering the auxiliary information with the at least one of the generated threads for the keyword associated with topic of the at least one of the generated threads.