US 11,875,220 B2
Method, apparatus, and storage medium for generating network representation for neural network
Zhaopeng Tu, Shenzhen (CN); Baosong Yang, Shenzhen (CN); and Tong Zhang, Shenzhen (CN)
Assigned to Tencent Technology (Shenzhen) Company Limited, Shenzhen (CN)
Filed by Tencent Technology (Shenzhen) Company Limited, Shenzhen (CN)
Filed on Oct. 13, 2020, as Appl. No. 17/069,609.
Application 17/069,609 is a continuation of application No. PCT/CN2019/100212, filed on Aug. 12, 2019.
Claims priority of application No. 201811027795.X (CN), filed on Sep. 4, 2018.
Prior Publication US 2021/0042603 A1, Feb. 11, 2021
Int. Cl. G06N 3/04 (2023.01)
CPC G06N 3/04 (2013.01) 20 Claims
OG exemplary drawing
 
1. A method for generating a network representation for a neural network, the method comprising:
obtaining, by a device comprising a memory storing instructions and a processor in communication with the memory, a source-side vector sequence corresponding to an input sequence;
performing, by the device, linear transformation on the source-side vector sequence, to obtain a request vector sequence, a key vector sequence, and a value vector sequence corresponding to the source-side vector sequence;
calculating, by the device, a logical similarity between the request vector sequence and the key vector sequence;
constructing, by the device, a local strength matrix according to the request vector sequence;
performing, by the device, nonlinear transformation based on the logical similarity and the local strength matrix, to obtain a local strength attention weight distribution corresponding to elements in the input sequence; and
fusing, by the device, value vectors in the value vector sequence according to the local strength attention weight distribution, to obtain a network representation sequence corresponding to the input sequence.