US 10,891,541 B2
Devices, systems, and methods for feature encoding
Jie Yu, Santa Clara, CA (US); and Francisco Imai, San Jose, CA (US)
Assigned to Canon Kabushiki Kaisha, Tokyo (JP)
Filed by CANON KABUSHIKI KAISHA, Tokyo (JP)
Filed on Apr. 20, 2017, as Appl. No. 15/492,944.
Claims priority of provisional application 62/337,040, filed on May 16, 2016.
Prior Publication US 2017/0330068 A1, Nov. 16, 2017
Int. Cl. G06N 3/04 (2006.01); G06N 3/08 (2006.01); G06K 9/62 (2006.01)
CPC G06N 3/0454 (2013.01) [G06K 9/629 (2013.01); G06K 9/6274 (2013.01); G06N 3/084 (2013.01)] 18 Claims
OG exemplary drawing
 
1. A system comprising:
one or more computer-readable media; and
one or more processors that are coupled to the one or more computer-readable media and that are configured to cause the device to
obtain data in a first modality;
propagate the data in the first modality through a first neural network, thereby generating first network outputs,
wherein the first neural network includes a first-stage neural network and a second-stage neural network,
wherein the first-stage neural network includes two or more layers,
wherein each layer of the two or more layers of the first-stage neural network includes a plurality of respective nodes,
wherein the second-stage neural network includes two or more layers, one of which is an input layer and one of which is an output layer,
wherein each node in each layer of the first-stage neural network is connected to the input layer of the second-stage neural network, and
wherein the output layer of the second-stage neural network produces the first network outputs;
calculate a gradient of a loss function based on the first network outputs and on the loss function;
backpropagate the gradient of the loss function through the first neural network including backpropagating the gradient through the first neural network in the following order: from the output layer of the second-stage neural network to the input layer of the second-stage neural network, then from the input layer of the second-stage neural network to each layer of the first-stage neural network, and then from a deepest layer of the first-stage neural network to an input layer of the first-stage neural network; and
update the first neural network based on the backpropagation of the gradient.