US 11,741,355 B2
Training of student neural network with teacher neural networks
Takashi Fukuda, Yokohama (JP); Masayuki Suzuki, Tokyo (JP); Osamu Ichikawa, Yokohama (JP); Gakuto Kurata, Tokyo (JP); Samuel Thomas, Elmsford, NY (US); and Bhuvana Ramabhadran, Mount Kisco, NY (US)
Assigned to INTERNATIONAL BUSINESS MACHINES CORPORATION, Armonk, NY (US)
Filed by INTERNATIONAL BUSINESS MACHINES CORPORATION, Armonk, NY (US)
Filed on Jul. 27, 2018, as Appl. No. 16/47,526.
Prior Publication US 2020/0034703 A1, Jan. 30, 2020
Int. Cl. G06N 3/08 (2023.01); G06N 3/045 (2023.01); G10L 25/51 (2013.01); G10L 15/02 (2006.01)
CPC G06N 3/08 (2013.01) [G06N 3/045 (2023.01); G10L 15/02 (2013.01); G10L 25/51 (2013.01); G10L 2015/025 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A computer-implemented method, comprising:
inputting input data to each teacher neural network among a plurality of teacher neural networks to obtain a soft label output among a plurality of soft label outputs from each teacher neural network among the plurality of teacher neural networks, wherein the plurality of teacher neural networks includes two or more different types of teacher neural networks;
evaluating an accuracy of each of the plurality of teacher neural networks using test data;
iteratively training a student neural network with the input data and the plurality of soft label outputs by selecting two or more of the plurality of teacher neural networks in a predetermined order for each of two or more training iterations;
iteratively increasing a frequency of selecting a particular teacher neural network among the plurality of teacher neural networks based on a comparison of an accuracy of the soft label output from the particular teacher neural network with corresponding correct training data; and
adjusting one or more of a plurality of weights between nodes in the selected teacher neural network based on the comparison as a number of iterations increases.