US 12,277,757 B2
Systems and methods for color-based outfit classification using semantic embeddings
Sergey Ulasen, Saint-Petersburg (RU); Alexander Snorkin, Moscow (RU); Andrey Adaschik, Saint-Petersburg (RU); Artem Shapiro, Dnipro (UA); Vasyl Shandyba, Dnipro (UA); Serg Bell, Costa del Sol (SG); and Stanislav Protasov, Singapore (SG)
Assigned to Acronis International GmbH, Schaffhausen (CH)
Filed by Acronis International GmbH, Schaffhausen (CH)
Filed on Apr. 28, 2022, as Appl. No. 17/731,875.
Claims priority of provisional application 63/212,181, filed on Jun. 18, 2021.
Prior Publication US 2022/0406045 A1, Dec. 22, 2022
Int. Cl. G06V 10/82 (2022.01); G06V 10/56 (2022.01); G06V 10/764 (2022.01)
CPC G06V 10/82 (2022.01) [G06V 10/56 (2022.01); G06V 10/764 (2022.01)] 18 Claims
OG exemplary drawing
 
1. A method for classifying objects in an image using a color-based neural network, the method comprising:
training, with a dataset comprising a plurality of images, a neural network to classify an object in a given image into a color class from a set of color classes each representing a distinct color, wherein the color class represents a predominant color of the object;
wherein the training further comprises:
for each anchor image from the plurality of images, identifying a positive image that shares a color class with the anchor image and a negative image that does not share a color class with the anchor image;
determining a respective color mask input for each of the anchor image, the positive image, and the negative image;
calculating semantic embeddings for each of the anchor image, the positive image, and the negative image with their respective color mask input; and
minimizing a triplet loss cost function comprising the semantic embeddings by updating weights used to generate the semantic embeddings;
receiving an input image depicting at least one object belonging to the set of color classes;
determining, from the set of color classes, a subset of color classes that are anticipated to be in the input image based on metadata of the input image;
generating a matched mask input indicating the subset set of color classes in the input image;
inputting both the input image and the matched mask input into the neural network, wherein the neural network is configured to:
determine a first semantic embedding of the input image and the matched mask input;
compare the first semantic embedding to a plurality of semantic embeddings of the plurality of images; and
identify, based on the comparison, a second semantic embedding with a least amount of distance to the first semantic embedding; and
outputting a color class associated with the second semantic embedding.