US 11,934,956 B2
Regularizing machine learning models
Sergey Ioffe, Mountain View, CA (US)
Assigned to Google LLC, Mountain View, CA (US)
Filed by Google LLC, Mountain View, CA (US)
Filed on Nov. 30, 2022, as Appl. No. 18/071,806.
Application 18/071,806 is a continuation of application No. 15/343,458, filed on Nov. 4, 2016, granted, now 11,531,874.
Claims priority of provisional application 62/252,374, filed on Nov. 6, 2015.
Prior Publication US 2023/0093469 A1, Mar. 23, 2023
This patent is subject to a terminal disclaimer.
Int. Cl. G06N 3/08 (2023.01); G06F 18/214 (2023.01); G06F 18/2413 (2023.01); G06N 3/04 (2023.01); G06V 10/44 (2022.01); G06V 10/764 (2022.01); G06V 10/774 (2022.01)
CPC G06N 3/08 (2013.01) [G06F 18/214 (2023.01); G06F 18/24137 (2023.01); G06N 3/04 (2013.01); G06V 10/454 (2022.01); G06V 10/764 (2022.01); G06V 10/774 (2022.01)] 20 Claims
OG exemplary drawing
 
1. A method of training a neural network, wherein the neural network is configured to receive an input data item and to process the input data item to generate a respective score for each label in a predetermined set of multiple labels, the method comprising:
obtaining a plurality of training items, wherein each training item is associated with an initial target label distribution that assigns a respective target score to each label in the predetermined set of multiple labels;
for each training item, modifying the initial target label distribution using a smoothing label distribution to obtain a modified initial target label distribution, wherein the smoothing label distribution includes a respective smoothing score for each label in the predetermined set of multiple labels; and
training the neural network to minimize a loss function based on (1) a respective network output generated by the neural network for each training item in the plurality of training items, and (2) the respective modified initial target label distribution for each training item in the plurality of training items.