US 11,941,681 B2
System, method, and computer program product for determining compatibility between items in images
Maryam Moosaei, San Jose, CA (US); and Hao Yang, San Jose, CA (US)
Assigned to Visa International Service Association, San Francisco, CA (US)
Appl. No. 17/271,257
Filed by Visa International Service Association, San Francisco, CA (US)
PCT Filed Aug. 26, 2019, PCT No. PCT/US2019/048102
§ 371(c)(1), (2) Date Feb. 25, 2021,
PCT Pub. No. WO2020/046795, PCT Pub. Date Mar. 5, 2020.
Claims priority of provisional application 62/725,454, filed on Aug. 31, 2018.
Prior Publication US 2021/0256588 A1, Aug. 19, 2021
Int. Cl. G06K 9/00 (2022.01); G06N 3/084 (2023.01); G06Q 30/0601 (2023.01); G06T 7/00 (2017.01)
CPC G06Q 30/0631 (2013.01) [G06N 3/084 (2013.01); G06T 7/0002 (2013.01); G06T 2207/20081 (2013.01); G06T 2207/20084 (2013.01)] 18 Claims
OG exemplary drawing
 
1. A computer-implemented method for determining compatibility between items in images, comprising:
obtaining, with at least one processor, image data associated with a plurality of images, wherein each image is associated with a single item of a plurality of items, wherein each image includes a plurality of elements of a matrix, and wherein the plurality of elements of the matrix of each image comprises the image data associated with that image;
for each image, inputting, with at least one processor, the image data associated with that image as a single object in a set of objects to a relation network; and
generating, with at least one processor, a compatibility score for the plurality of items from the relation network based on the set of objects, wherein the compatibility score includes a prediction of whether the plurality of items is compatible, and
wherein generating the compatibility score for the plurality of items from the relation network based on the set of objects further comprises:
for each pair of objects in the set of objects, processing, with at least one processor, that pair of objects with at least one multi-layer perceptron to produce a relation output;
determining, with at least one processor, an average of the relation outputs of the each pair of objects in the set of objects to produce a single averaged output; and
processing, with at least one processor, the single averaged output with at least one other multi-layer perceptron to produce the compatibility score for the plurality of items.