US 11,941,342 B2
Text layout interpretation using eye gaze data
Alexander James Faaborg, Mountain View, CA (US); and Brett Barros, San Mateo, CA (US)
Assigned to Google LLC, Mountain View, CA (US)
Filed by Google LLC, Mountain View, CA (US)
Filed on May 26, 2022, as Appl. No. 17/804,242.
Application 17/804,242 is a continuation of application No. 17/310,977, granted, now 11,347,927, previously published as PCT/US2020/070650, filed on Oct. 9, 2020.
Prior Publication US 2022/0284168 A1, Sep. 8, 2022
Int. Cl. G06K 9/00 (2022.01); G06F 3/01 (2006.01); G06F 18/21 (2023.01); G06F 18/214 (2023.01); G06F 40/106 (2020.01); G06F 40/30 (2020.01); G06V 30/412 (2022.01); G10L 13/02 (2013.01); G06V 30/10 (2022.01)
CPC G06F 40/106 (2020.01) [G06F 3/013 (2013.01); G06F 18/214 (2023.01); G06F 18/217 (2023.01); G06F 40/30 (2020.01); G06V 30/412 (2022.01); G10L 13/02 (2013.01); G06V 30/10 (2022.01)] 19 Claims
OG exemplary drawing
 
1. A computer program product, the computer program product being tangibly embodied on a non-transitory computer-readable storage medium and comprising instructions that, when executed by at least one computing device, are configured to cause the at least one computing device to:
determine, from a head-mounted device (HMD), text that includes words arranged according to a layout having a displayed order of the words, the layout being arranged to visually indicate location-based information with respect to at least one item that is physically proximate to the text;
determine, using a layout interpretation model, an output order of at least a subset of the words of the text based on the text, the layout, and a physical location of the text, the output order being different from the displayed order; and
output, using the HMD and based on the physical location, the at least the subset of the words of the text in the output order and based on the layout, and conveying the location-based information with respect to the at least one item.