US 12,450,933 B2
Method, device, computer equipment and storage medium for identifying illegal commodity
Yao Qi, Hangzhou (CN); Hongyang Chen, Hangzhou (CN); Jingsong Lv, Hangzhou (CN); and Wentao Yang, Hangzhou (CN)
Assigned to ZHEJIANG LAB, Hangzhou (CN)
Filed by ZHEJIANG LAB, Zhejiang (CN)
Filed on Sep. 4, 2023, as Appl. No. 18/460,680.
Application 18/460,680 is a continuation of application No. PCT/CN2023/086438, filed on Apr. 6, 2023.
Claims priority of application No. 202310342924.9 (CN), filed on Apr. 3, 2023.
Prior Publication US 2024/0331425 A1, Oct. 3, 2024
Int. Cl. G06V 30/19 (2022.01); G06N 5/02 (2023.01)
CPC G06V 30/19187 (2022.01) [G06N 5/02 (2013.01); G06V 30/19173 (2022.01)] 10 Claims
OG exemplary drawing
 
1. A method for identifying an illegal commodity for monitoring e-commerce platforms, comprising:
step (1) constructing a multi-modal knowledge graph based on a multi-modal knowledge graph data set, and extracting visual features of all visual modality entities and text features of a text modality entities in the multi-modal knowledge graph;
step (2) acquiring commodity information based on a database, wherein the commodity information comprises a commodity image and a commodity text, and the commodity text comprises at least one of a commodity title and a commodity description;
step (3) generating a commodity visual feature based on the commodity image;
step (4) generating a commodity text feature based on the commodity text;
step (5) linking the commodity image and the commodity text to the multi-modal knowledge graph constructed in the step (1) using an entity linking method, based on the visual features and the text features extracted in the step (1) as well as the commodity visual feature generated in the step (3) and the commodity text feature generated in the step (4); and
step (6) acquiring a correlation between the commodity image and the commodity text based on the linked multi-modal knowledge graph obtained in the step (5) to determine illegality of a commodity by a discriminator.