US 12,346,810 B2
Method and system for self supervised training of deep learning based time series models
Pradeep Rathore, Pune (IN); Arghya Basak, Pune (IN); Sri Harsha Nistala, Pune (IN); and Venkataramana Runkana, Pune (IN)
Assigned to TATA CONSULTANCY SERVICES LIMITED, Mumbai (IN)
Filed by Tata Consultancy Services Limited, Mumbai (IN)
Filed on Dec. 20, 2021, as Appl. No. 17/555,554.
Claims priority of application No. 202121029887 (IN), filed on Jul. 2, 2021.
Prior Publication US 2023/0033835 A1, Feb. 2, 2023
Int. Cl. G06N 3/08 (2023.01); G06F 18/214 (2023.01); G06F 18/2433 (2023.01)
CPC G06N 3/08 (2013.01) [G06F 18/2148 (2023.01); G06F 18/2155 (2023.01); G06F 18/2433 (2023.01)] 7 Claims
OG exemplary drawing
 
1. A processor-implemented method for self-supervised training of a deep learning based model with un-labelled time-series data comprising:
receiving, via one or more hardware processors, a plurality of input data from one or more predefined data sources, wherein the plurality of input data is the un-labelled time-series data;
preprocessing, via the one or more hardware processors, the received plurality of input data for verification of availability of the received plurality of input data to:
remove noise and outliers;
achieve uniform sampling frequency of the received plurality of input data; and
synchronization by incorporating lags and integration of a plurality of variables from one or more databases;
masking, via the one or more hardware processors, the preprocessed plurality of input data for one or more missing values of the plurality of input data and applying missingness mask corresponding to the one or more missing values, wherein the method handles missing data without need of imputation as the method uses the missingness mask;
distorting, via the one or more hardware processors, the masked plurality of input data of a flame detector voltage of an industrial gas turbine combustor using one or more distortion techniques, wherein the one or more distortion techniques include quantization determined using minimum and maximum of each sensor data and the quantization is different for different sensors, insertion, deletion, and combination of the one or more such distortion techniques with random subsequence shuffling, wherein calculating loss considering the missingness mask appropriately and update weights using back propagation, wherein, in case of the deletion, add padding towards an end or in starting of the input data to maintain a length of the input data same before and after the deletion, and the length of the padding in case of the deletion is same as number of deleted instances, wherein, in case of the insertion, deleting the input data from last or starting to maintain length of the input data after the insertion same as before the insertion, and the length of a cropped portion of the input data is same as number of newly inserted instances; and
training, via the one or more hardware processors, the deep learning based model with the distorted plurality of input data using self-supervised learning to reconstruct the masked plurality of input data, wherein, after training the deep learning based model using self-supervised learning, the deep learning based model is finetuned either by replacing final layers of the deep learning based model or training the weights of the final layer of the deep learning based model, wherein the trained deep learning based model is used to identify abnormal trends of the flame detector voltage of the industrial gas turbine combustor.