US 12,436,984 B2
Pre-training language model-based summarization generation method
Shubin Cai, Shenzhen (CN); Huaifeng Zhou, Shenzhen (CN); Zhong Ming, Shenzhen (CN); and Xiaogang Feng, Shenzhen (CN)
Assigned to SHENZHEN UNIVERSITY, Shenzhen (CN)
Appl. No. 18/249,863
Filed by SHENZHEN UNIVERSITY, Guangdong (CN)
PCT Filed Dec. 14, 2020, PCT No. PCT/CN2020/136138
§ 371(c)(1), (2) Date Apr. 20, 2023,
PCT Pub. No. WO2022/104967, PCT Pub. Date May 27, 2022.
Claims priority of application No. 202011301462.9 (CN), filed on Nov. 19, 2020.
Prior Publication US 2023/0418856 A1, Dec. 28, 2023
Int. Cl. G06F 17/00 (2019.01); G06F 16/34 (2019.01); G06F 40/126 (2020.01); G06F 40/30 (2020.01)
CPC G06F 16/345 (2019.01) [G06F 40/126 (2020.01); G06F 40/30 (2020.01)] 9 Claims
OG exemplary drawing
 
1. A pre-training language model-based summary generation method, the method comprising:
acquiring text information of a summary to be generated, and performing a language pre-training process having a multi-feature weight on the text information to obtain a candidate summary; wherein the multi-feature weight comprises a plurality of dimension weighted feature data, and the plurality of dimension weighted feature data comprises: a corresponding sentence similarity calculation value, a title similarity weighting value, a keyword weighting value, a subject term weighting value, a position information weighting value, and a KNN smoothing strategy value;
inputting the candidate summary into a pre-training language model to obtain a pre-training language model output data, wherein the pre-training language model is generated according to a first modeling model, and a parameter setting of the first modeling model comprises: setting a size of a training batch, a text maximum length, a maximum length of a target summary, and a bundling size; and
inputting the pre-training language model output data into a decoder model and obtaining a target summary, wherein a number of a plurality of layers in a decoder of the decoder model is a preset value.