US 11,941,346 B2
Systems and methods for long document summarization
Bo Pang, Los Angeles, CA (US); Erik Nijkamp, Palo Alto, CA (US); Yingbo Zhou, Palo Alto, CA (US); and Caiming Xiong, Menlo Park, CA (US)
Assigned to Salesforce, Inc., San Francisco, CA (US)
Filed by Salesforce, Inc., San Francisco, CA (US)
Filed on Jan. 31, 2022, as Appl. No. 17/589,650.
Claims priority of provisional application 63/252,526, filed on Oct. 5, 2021.
Prior Publication US 2023/0107640 A1, Apr. 6, 2023
Int. Cl. G06F 40/166 (2020.01); G06F 40/284 (2020.01)
CPC G06F 40/166 (2020.01) [G06F 40/284 (2020.01)] 20 Claims
OG exemplary drawing
 
1. A method for document summarization, the method comprising:
receiving, via a communication interface, a text document comprising a plurality of tokens;
computing, at an attention layer, a first set of token representations by attending the plurality of tokens to respective nearby tokens within a pre-defined encoding window;
generating, by a pooling layer, a set of segment representations from the computed first set of token representations from the attention layer;
updating the set of segment representations with a full self-attention layer;
generating a second set of token representations by applying cross attention upon the first set of token representations and the updated set of segment representations after the full self-attention layer; and
sending the generated second set of token representations of the text document to a decoder for generating a summary output based on the second set of token representations.