CPC G06F 40/166 (2020.01) [G06F 40/284 (2020.01)] | 20 Claims |
1. A method for document summarization, the method comprising:
receiving, via a communication interface, a text document comprising a plurality of tokens;
computing, at an attention layer, a first set of token representations by attending the plurality of tokens to respective nearby tokens within a pre-defined encoding window;
generating, by a pooling layer, a set of segment representations from the computed first set of token representations from the attention layer;
updating the set of segment representations with a full self-attention layer;
generating a second set of token representations by applying cross attention upon the first set of token representations and the updated set of segment representations after the full self-attention layer; and
sending the generated second set of token representations of the text document to a decoder for generating a summary output based on the second set of token representations.
|