US 12,456,057 B2
Methods for building a deep latent feature extractor for industrial sensor data
Denis Krompaß, Munich (DE); Hans-Georg Köpken, Erlangen (DE); and Tomislav Tomov, Munich (DE)
Assigned to Siemens Aktiengesellschaft, Munich (DE)
Appl. No. 17/426,366
Filed by Siemens Aktiengesellschaft, Munich (DE)
PCT Filed Jan. 17, 2020, PCT No. PCT/EP2020/051127
§ 371(c)(1), (2) Date Jul. 28, 2021,
PCT Pub. No. WO2020/156835, PCT Pub. Date Jun. 8, 2020.
Claims priority of application No. 19154667 (EP), filed on Jan. 31, 2019.
Prior Publication US 2022/0101125 A1, Mar. 31, 2022
Int. Cl. G06N 3/09 (2023.01); G06N 3/08 (2023.01); G06N 3/096 (2023.01)
CPC G06N 3/096 (2023.01) [G06N 3/08 (2013.01)] 17 Claims
OG exemplary drawing
 
1. A method of pre-training a latent feature extractor that is one single sub-neural network for an industrial neural network by pre-initializing parameters of the latent feature extractor for industrial sensor data that automatically extracts latent features from general industrial sensor data such that an amount of labelled training data needed to train the industrial neural network for a specific task is reduced, the method comprising:
receiving, by the industrial neural network, non-uniform training data comprising training examples for each single task of a multitude of tasks covering different problems in an industrial field; and
optimizing the parameters of the latent feature extractor of the industrial neural network, by the latent feature extractor of the industrial neural network, based on the multitude of tasks, wherein the following acts are executed iteratively until the parameters of the latent feature extractor have converged:
randomly selecting a single task from the multitude of tasks for optimizing the parameters of the latent feature extractor; and
sampling the randomly selected single task for iterations with the respective training examples of the non-uniform training data for optimizing the parameters of the latent feature extractor based on the randomly selected single task, wherein each iteration of the iterations comprises:
extracting latent features from a training signal of a current training example of the respective training examples with the latent feature extractor; and
deriving an output signal for the randomly selected single task from the extracted latent features with a respective task module of the randomly selected single task,
wherein the parameters of the latent feature extractor have converged when, after the randomly selecting of the single task in an nth iteration, the parameters of the latent feature extractor have not changed by more than a predefined threshold; and
providing the pre-trained latent feature extractor to the industrial neural network when training the industrial neural network for the specific task.