US 12,333,776 B2
System and method for generating an optimized image with scribble-based annotation of images using a machine learning model
C. V. Jawahar, Hyderabad (IN); Bhavani Sambaturu, Hyderabad (IN); Ashutosh Gupta, New Delhi (IN); and Chetan Arora, Hari Nagar (IN)
Filed by International Institute of Information Technology, Hyderabad, Hyderabad (IN); and Indian Institute of Technology, Delhi, New Delhi (IN)
Filed on Mar. 1, 2022, as Appl. No. 17/684,242.
Claims priority of application No. 202141008605 (IN), filed on Mar. 1, 2021.
Prior Publication US 2022/0277540 A1, Sep. 1, 2022
Int. Cl. G06K 9/00 (2022.01); G06N 20/00 (2019.01); G06V 10/26 (2022.01); G06V 10/764 (2022.01); G06V 20/70 (2022.01)
CPC G06V 10/26 (2022.01) [G06N 20/00 (2019.01); G06V 10/764 (2022.01); G06V 20/70 (2022.01)] 17 Claims
OG exemplary drawing
 
1. A processor-implemented method for generating an optimized image with scribble-based annotation using a machine learning model, comprising:
segmenting, using the machine learning model, a received image from at least one of a cloud, or a user device to obtain a classified image using a plurality of classes, wherein each class is represented with a label, wherein the plurality of classes are obtained based on pre-defined weights of the classified image;
displaying, using a graphical user interface, the classified image which specifies the plurality of classes on the classified image with outlines, wherein the outlines on the classified image are generated by the machine learning model;
enabling a user to mark or scribble on the classified image to annotate the plurality of classes if an area on the classified image is not classified into at least one of the plurality of classes;
assigning, using the machine learning model, a color mask for each scribbled area after receiving the classified image that is marked or scribbled by the user;
computing, using the machine learning model, a loss function for a location of pixels based on the color mask assigned on each scribbled area of the classified image, wherein the loss function identifies whether the classified image matches with the received mark or scribble at the scribbled scribble area on the classified image;
modifying, using the machine learning model, the pre-defined weights for each scribbled area to match the classified image and a determined class on the classified image using the loss function by a learning rate, wherein the learning rate determines a speed of change of the weight;
determining, using the machine learning model, whether the classified image is matched with the determined class on the classified image; and
generating the optimized image if the classified image is matched with the determined class on the classified image.