US 12,190,235 B2
System for training an artificial neural network
Maximilian Golub, Seattle, WA (US); Ritchie Zhao, Bellevue, WA (US); Eric Chung, Woodinville, WA (US); Douglas Burger, Bellevue, WA (US); Bita Darvish Rouhani, Bellevue, WA (US); Ge Yang, Bellevue, WA (US); and Nicolo Fusi, Watertown, MA (US)
Assigned to Microsoft Technology Licensing, LLC, Redmond, WA (US)
Filed by Microsoft Technology Licensing, LLC, Redmond, WA (US)
Filed on Jan. 29, 2021, as Appl. No. 17/163,299.
Prior Publication US 2022/0245444 A1, Aug. 4, 2022
Int. Cl. G06N 3/08 (2023.01); G06F 17/18 (2006.01)
CPC G06N 3/08 (2013.01) [G06F 17/18 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A system comprising:
one or more artificial intelligence (AI) processors;
one or more control processors; and
a non-transitory computer readable storage medium having stored thereon program code executable by the one or more control processors, the program code causing the one or more control processors to:
receive an artificial neural network model and configure the model, based on a plurality of training parameters, to execute a training process on the one or more AI processors;
monitor a plurality of statistics produced upon execution of the training process; and
adjust one or more training parameters of the plurality of training parameters, based on one or more statistics of the plurality of statistics, to maintain at least one of the plurality of statistics within a predetermined range, wherein the plurality of training parameters includes a parameter configuring a precision associated with the artificial neural network model, and wherein the artificial neural network model is configured with a first precision associated with a first layer of the artificial neural network model and a second precision associated with a second layer of the artificial neural network model, the first precision being higher than the second precision, and the first layer being a layer closer to an input of the artificial neural network model and the second layer being a layer closer to an output of the artificial neural network model.