| CPC G06N 3/08 (2013.01) | 20 Claims |
|
1. A method of training a neural network, the method comprising:
providing a neural network comprising an input layer of input neurons, a plurality of hidden layers of neurons in successive layers of neurons above said input layer and at least one concept-identifying layer of neurons above said intermediate hidden layers of neurons;
wherein said hidden layers of neurons comprise a first hidden layer of neurons connected to said input layer of neurons, and one or more intermediate hidden layers of neurons above said first hidden layer of neurons, and wherein each said layer of neurons is connected to the layer of neurons below;
training said neural network on a set of example inputs provided successively to said input layer of neurons to determine weights of connections between said first hidden layers of neurons and said one or more intermediate hidden layers of neurons of a feature-identifying subset of said neural network;
presenting, successively, a set of further example inputs to said input neurons of said trained neural network;
capturing a pattern of activation from a said intermediate hidden layer of neurons for each of said further example inputs;
storing said captured patterns of activation from said intermediate hidden layer for the further example inputs in an intermediate feature memory;
identifying a plurality of overlap patterns of activation stored in said intermediate feature memory, wherein a said overlap pattern of activation is a pattern of activation within a said group of stored patterns of activation defining a subset of neurons in said intermediate hidden layer common to all the patterns of said group in which the neurons of said subset have greater than a threshold level of activation;
activating neurons in said intermediate layer with successive said overlap patterns of activation; and
training said neural network on said overlap patterns of activation to determine weights of connections between said intermediate hidden layer and said concept-identifying layer of neurons to train a concept-identifying subset of said neural network;
wherein said at least one concept-identifying layer of neurons comprises groups of neurons activatable by generalised concepts, wherein a said generalised concept comprises a set of features common to at least some of said example inputs but not represented as a distinct pattern of activation in said intermediate hidden layer separate from one or more other features of said example inputs represented in said intermediate hidden layer.
|