US 12,456,466 B2
Personalized and curated transcription of auditory experiences to improve user engagement
Antje Ihlefeld, Redmond, WA (US); Alexander Reich, Seattle, WA (US); and Morteza Khaleghimeybodi, Bothell, WA (US)
Assigned to Meta Platforms Technologies, LLC, Menlo Park, CA (US)
Filed by META PLATFORMS TECHNOLOGIES, LLC, Menlo Park, CA (US)
Filed on Jan. 26, 2023, as Appl. No. 18/160,249.
Prior Publication US 2024/0257812 A1, Aug. 1, 2024
Int. Cl. G10L 15/26 (2006.01); G02B 27/01 (2006.01); G10L 25/51 (2013.01)
CPC G10L 15/26 (2013.01) [G02B 27/017 (2013.01); G10L 25/51 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A computer-implemented method, comprising:
computing a first metric for a word, the first metric indicating an attentional state of a user perceiving the word during an auditory experience, wherein the metric is computed based on biomarkers associated with the user that are computed using signals from one or more sensors on a near-eye display system operating in an artificial reality environment;
computing a second metric corresponding to an intelligibility of the word by estimating an audibility of the word using a signal-to-noise (SNR) ratio associated with the word during the auditory experience;
computing a third metric corresponding to an importance of the word by using a machine learning model to determine the importance of the word in a context of a conversation during the auditory experience; and
based on a weighting assigned to each of the first metric, second metric and third metric, computing a fourth metric; and
determining whether to transcribe the word for display based on the fourth metric.