US 12,333,831 B2
Image based command classification and task engine for a computing system
Bharath Cheluvaraju, Bangalore (IN); Saheel Ram Godhane, Warud (IN); Nishan Hassan, Delhi (IN); and Sheetal Shamsher Sethi, Hyderabad (IN)
Assigned to Microsoft Technology Licensing, LLC, Redmond, WA (US)
Filed by MICROSOFT TECHNOLOGY LICENSING, LLC, Redmond, WA (US)
Filed on Dec. 31, 2021, as Appl. No. 17/566,995.
Prior Publication US 2023/0215198 A1, Jul. 6, 2023
Int. Cl. G06V 30/10 (2022.01); G06F 3/048 (2013.01); G06F 3/0481 (2022.01); G06F 3/0482 (2013.01); G06N 3/0455 (2023.01); G06N 3/048 (2023.01); G06N 3/09 (2023.01); G06N 7/01 (2023.01); G06Q 10/10 (2023.01); G06V 10/40 (2022.01); G06V 10/44 (2022.01); G06V 10/82 (2022.01); G06V 20/20 (2022.01); G06V 20/70 (2022.01); G06V 30/413 (2022.01)
CPC G06V 30/10 (2022.01) [G06F 3/048 (2013.01); G06F 3/0481 (2013.01); G06F 3/0482 (2013.01); G06N 3/0455 (2023.01); G06V 10/40 (2022.01); G06V 10/82 (2022.01); G06V 30/413 (2022.01); G06N 3/048 (2023.01); G06N 3/09 (2023.01); G06N 7/01 (2023.01); G06Q 10/10 (2013.01); G06V 10/454 (2022.01); G06V 20/20 (2022.01); G06V 20/70 (2022.01)] 19 Claims
OG exemplary drawing
 
1. A computerized system, the computerized system comprising:
at least one computer processor; and
computer memory storing computer-readable instructions that, when used by the at least one computer processor, cause the at least one computer processor to perform operations comprising:
receiving an image comprising at least an indication of non-alphanumeric-character objects;
extracting image data for the image based on at least the non-alphanumeric-character objects to produce a plurality of image data features;
determining, by a machine learning model, an image context based on a first image data feature of the plurality of image data features, wherein the image context defines a relationship between alphanumeric characters depicted in the image and the non-alphanumeric-character objects;
determining a command associated with the image based on a second image data feature in the plurality of image data features;
determining at least one task corresponding to the command based on at least one of: the command, the first image data feature, the image context, and the second image data feature;
based on the at least one task and the plurality of image data features, determining at least one task entity; and
generating the at least one task, comprising the at least one task entity and configured for use in a computer productivity application.