US 12,147,909 B2
Cross-media corresponding knowledge generation method and apparatus
Feng Lin, Hangzhou (CN); and Yunhe Pan, Hangzhou (CN)
Assigned to ZHEJIANG LAB, Hangzhou (CN)
Filed by ZHEJIANG LAB, Zhejiang (CN)
Filed on Oct. 23, 2023, as Appl. No. 18/491,817.
Application 18/491,817 is a continuation of application No. PCT/CN2022/099376, filed on Jun. 17, 2022.
Prior Publication US 2024/0046122 A1, Feb. 8, 2024
Int. Cl. G06N 5/022 (2023.01); G06F 40/30 (2020.01); G06V 10/77 (2022.01); G06V 10/80 (2022.01); G06V 40/20 (2022.01)
CPC G06N 5/022 (2013.01) [G06F 40/30 (2020.01); G06V 10/7715 (2022.01); G06V 10/806 (2022.01); G06V 40/25 (2022.01)] 15 Claims
OG exemplary drawing
 
1. A cross-media corresponding knowledge generation method, comprising the following steps:
Generating a second knowledge unit of a second medium according to a first knowledge unit of a predefined first medium, wherein types of the first medium and the second medium are different from each other, and the first knowledge unit and the second knowledge unit are configured to represent semantics of feature information of a same target object in a media corresponding to the types of the first medium and the second medium, respectively;
Generating a first feature parameter vector corresponding to the first knowledge unit and a second feature parameter vector corresponding to the second knowledge unit;
mapping the first feature parameter vector and the second feature parameter vector to two-dimensional spherical feature surfaces corresponding to the first feature parameter vector and the second feature parameter, respectively, according to a preset mapping relationship, to obtain a first feature point of the first feature parameter vector on a two-dimensional spherical feature surface corresponding to the first feature parameter vector and a second feature point of the second feature parameter vector on a two-dimensional spherical feature surface corresponding to the second feature parameter vector;
indexing the first feature point and the second feature point, respectively, to obtain a first index corresponding to the first feature point and a second index corresponding to the second feature point; and
generating a bidirectional index corresponding relationship between the first knowledge unit and the second knowledge unit according to the first index and the second index;
acquiring a third feature parameter vector of a to-be-converted medium;
mapping the third feature parameter vector to the two-dimensional spherical feature surface according to the mapping relationship to obtain a third feature point;
mapping cross-media corresponding knowledge according to the third feature point and the bidirectional index corresponding relationship, and mapping the to-be-converted medium to a target medium to obtain a target knowledge unit, expressing cross-media corresponding knowledge between the first knowledge unit and the second knowledge unit, so that the cross-media corresponding knowledge is capable of being directly configured to map between subsequent cross-media knowledge units;
wherein the to-be-converted medium is one of the first medium and the second medium, and the target medium is the other one of the first medium and the second medium; and
wherein when the to-be-converted medium is the first medium, and the target medium is the second medium, the target knowledge unit is the second knowledge unit; and when the to-be-converted medium is the second medium, and the target medium is the first medium, the target knowledge unit is the first knowledge unit.