US 12,147,652 B1
Annotating screen content in a mobile environment
Matthew Sharifi, Kilchberg (CH); and David Petrou, Brooklyn, NY (US)
Assigned to GOOGLE LLC, Mountain View, CA (US)
Filed by GOOGLE LLC, Mountain View, CA (US)
Filed on Dec. 18, 2023, as Appl. No. 18/543,747.
Application 17/192,248 is a division of application No. 16/666,976, filed on Oct. 29, 2019, granted, now 10,963,630, issued on Mar. 30, 2021.
Application 16/666,976 is a division of application No. 15/680,690, filed on Aug. 18, 2017, granted, now 10,491,660, issued on Nov. 26, 2019.
Application 15/680,690 is a division of application No. 14/465,279, filed on Aug. 21, 2014, granted, now 9,762,651, issued on Sep. 12, 2017.
Application 18/543,747 is a continuation of application No. 18/106,218, filed on Feb. 6, 2023, granted, now 11,907,739.
Application 18/106,218 is a continuation of application No. 17/827,196, filed on May 27, 2022, granted, now 11,573,810, issued on Feb. 7, 2023.
Application 17/827,196 is a continuation of application No. 17/192,248, filed on Mar. 4, 2021, granted, now 11,347,385, issued on May 31, 2022.
Claims priority of provisional application 62/023,736, filed on Jul. 11, 2014.
Int. Cl. G06F 3/04842 (2022.01); G06F 40/134 (2020.01); G06F 40/169 (2020.01); G06F 40/295 (2020.01); G06V 30/416 (2022.01)
CPC G06F 3/04842 (2013.01) [G06F 40/169 (2020.01); G06F 40/295 (2020.01); G06F 40/134 (2020.01); G06V 30/416 (2022.01)] 15 Claims
OG exemplary drawing
 
1. A method implemented by one or more processors, the method comprising:
identifying, at a client device, content that is displayed at the client device via a display of the client device, wherein the content includes at least a body of text;
transmitting, to a server, the content that is displayed at the client device to obtain annotation data for the body of text, wherein transmitting the content to the server causes the server to:
process the content that is displayed at the client device to generate annotation data for the body of text, wherein the annotation data includes at least a first visual cue associated with a first entity that is referenced in the body of text and a second visual cue associated with a second entity that is also referenced in the body of text;
receiving, from the server, the annotation data for the body of text;
causing the body of text to be annotated with the first visual cue associated with the first entity that is referenced in the body of text and the second visual cue associated with the second entity that is also referenced in the body of text;
receiving, from a user of the client device, a selection of the first visual cue associated with the first entity and/or the second visual cue associated with the second entity; and
in response to receiving the selection of the first visual cue and/or the second visual cue:
causing a description of a relationship between the first entity and the second entity to be rendered via the display of the client device.