US 12,430,676 B2
Method and electronic device for recognizing product
Kyungsu Kim, Suwon-si (KR); Ohjoon Kwon, Suwon-si (KR); Younguk Kim, Suwon-si (KR); Hyunsoo Choi, Suwon-si (KR); Yehoon Kim, Suwon-si (KR); Hyunhan Kim, Suwon-si (KR); Hyosang Kim, Suwon-si (KR); and Hyungmin Lee, Suwon-si (KR)
Assigned to Samsung Electronics Co., Ltd., Suwon-si (KR)
Filed by Samsung Electronics Co., Ltd., Suwon-si (KR)
Filed on Dec. 30, 2022, as Appl. No. 18/148,629.
Application 18/148,629 is a continuation of application No. PCT/KR2022/020670, filed on Dec. 19, 2022.
Claims priority of application No. 10-2021-0191652 (KR), filed on Dec. 29, 2021; and application No. 10-2022-0077805 (KR), filed on Jun. 24, 2022.
Prior Publication US 2023/0206298 A1, Jun. 29, 2023
Int. Cl. G06V 30/14 (2022.01); G06Q 30/0601 (2023.01); G06V 10/80 (2022.01)
CPC G06Q 30/0627 (2013.01) [G06V 10/806 (2022.01); G06V 30/1444 (2022.01)] 19 Claims
OG exemplary drawing
 
1. A method of performed by an electronic device for recognizing a product, the method comprising:
obtaining first feature information and second feature information from an image related to a product;
obtaining fusion feature information based on the first feature information and the second feature information by using a main encoder model that reflects a correlation between feature information of different modalities;
comparing the fusion feature information against a product database for a match to the fusion feature information in the product database; and
providing information about the product, based on a result of the comparing of the fusion feature information against the product database being that there is the match to the fusion feature information in the product database;
comparing the first feature information and the second feature information against the product database for a match to the first feature information or the second feature information in the product database, based on the result of the comparing of the fusion feature information against the product database being that there is not the match to the fusion feature information in the product database; and
updating the main encoder model in such a manner that there is a match to the non-matching fusion feature information in the product database, based on a result of the comparing of the first feature information and the second feature information against the product database being that there is the match to the first feature information or the second feature information in the product database.