US 11,699,065 B2
Ensemble of clustered dual-stage attention-based recurrent neural networks for multivariate time series prediction
Dongjin Song, Princeton, NJ (US); Yuncong Chen, Plainsboro, NJ (US); and Haifeng Chen, West Windsor, NJ (US)
Assigned to NEC Corporation
Filed by NEC Laboratories America, Inc., Princeton, NJ (US)
Filed on Aug. 4, 2020, as Appl. No. 16/984,359.
Claims priority of provisional application 62/884,338, filed on Aug. 8, 2019.
Prior Publication US 2021/0042605 A1, Feb. 11, 2021
Int. Cl. G06N 3/02 (2006.01); G06N 3/044 (2023.01); G06N 3/049 (2023.01); G06N 20/20 (2019.01); G06F 18/23 (2023.01); G06F 18/214 (2023.01)
CPC G06N 3/044 (2023.01) [G06F 18/214 (2023.01); G06F 18/23 (2023.01); G06N 3/049 (2013.01); G06N 20/20 (2019.01)] 20 Claims
OG exemplary drawing
 
1. A computer-implemented method for multivariate time series prediction, comprising:
decomposing each time series from among a batch of multiple driving time series and a target time series into a raw component, a shape component, and a trend component, the shape component including an original time series segment minus values of each time series at T time step, and the trend component including a value difference of two consecutive time steps;
for each of the decomposed components, selecting a driving time series relevant thereto from the batch and obtaining hidden features of the selected driving time series, by applying the batch to an input attention-based encoder of an Ensemble of Clustered dual-stage attention-based Recurrent Neural Networks (EC-DARNNS);
automatically clustering, by a hardware processor, the hidden features in a hidden space using a temporal attention-based decoder of the EC-DARNNS, each of the Clustered dual-stage attention-based Recurrent Neural Networks (RNNs) in the Ensemble being dedicated and applied to a respective one of the decomposed components; and
predicting a respective value of one or more future time steps for the target time series based on respective prediction outputs for each of the decomposed components by the EC-DARNNS.