CPC G06N 3/082 (2013.01) [G06F 18/213 (2023.01); G06N 3/04 (2013.01); G06N 3/048 (2023.01); G06N 3/08 (2013.01); G06F 18/24 (2023.01); G06F 2218/00 (2023.01)] | 12 Claims |
1. A computer-based method for analyzing and modeling a multivariate time series data based on an attention computation, the method comprising:
capturing dependencies across different variables through input embedding;
mapping an order of a sample appearance to a randomized lookup table via positional encoding;
capturing dependencies within a plurality of self-attention mechanisms, each self-attention mechanism of the plurality of self-attention mechanisms capturing dependencies within a single sequence of each self-attention mechanism;
determining a range of dependency to consider for each position being analyzed within the single sequence of each self-attention mechanism;
obtaining a plurality of attention weightings to other positions within the single sequence through computation of an inner product, each of the plurality of attention weightings obtained within the single sequence of each self-attention mechanism;
utilizing the plurality of attention weightings to acquire a plurality of vector representations for a position;
masking the single sequence of each self-attention mechanism to enable causality;
employing a dense interpolation technique for encoding partial temporal ordering to obtain a single vector representation from the plurality of vector representations;
applying a linear layer to obtain logits from the single vector representation; and
applying a final prediction layer whose type depends on a specific task.
|