US 12,260,029 B2
Universal highlighter for contextual notetaking
Fnu Primadona, Mill Creek, WA (US); Sivaramakrishna Mopati, Issaquah, WA (US); and Jason Glenn Silvis, Snohomish, WA (US)
Assigned to Microsoft Technology Licensing, LLC, Redmond, WA (US)
Filed by Microsoft Technology Licensing, LLC, Redmond, WA (US)
Filed on Sep. 22, 2022, as Appl. No. 17/950,488.
Prior Publication US 2024/0103646 A1, Mar. 28, 2024
Int. Cl. G06F 3/038 (2013.01); G06F 3/0354 (2013.01); G06F 3/0482 (2013.01); G06V 30/14 (2022.01); G06V 30/413 (2022.01)
CPC G06F 3/03545 (2013.01) [G06F 3/038 (2013.01); G06F 3/0482 (2013.01); G06V 30/1456 (2022.01); G06V 30/413 (2022.01); G06V 2201/02 (2022.01)] 20 Claims
OG exemplary drawing
 
1. A computer-implemented method for interactive notetaking, comprising:
receiving, in a notetaking mode of a display screen, a selection of one or more regions displayed on the display screen, wherein the one or more selected regions include pixel data of content, and the pixel data include a copy of raster image data in pixels of the selection of one or more regions displayed on the display screen for object recognition of the content;
displaying an indication of the selection over the content of the one or more selected regions;
evaluating the pixel data of the content displayed on the display screen using the object recognition to detect at least one of text data or image data associated with the content;
evaluating at least one of a user profile, a user browsing history, the one or more selected regions, or the display screen to determine context data of the content;
automatically generating notes data comprising both the detected text data or image data associated with the content and the determined context data of the content; and
displaying the notes data to a user for recall of the content based on the context data.