| CPC G06N 3/084 (2013.01) [G06N 3/04 (2013.01)] | 17 Claims |
|
1. A computing system for performing gating-based regularization of a neural network, the computing system comprising:
one or more processors; and
one or more non-transitory computer-readable media that collectively store:
a neural network, the neural network comprising:
a gated network unit, the gated network unit comprising one or more network parameters; and
a gating path associated with the gated network unit, wherein the gating path comprises one or more gating units, wherein each of the one or more gating units comprises one or more gating parameters, wherein the gating path is configured to produce a gating value that represents an overall magnitude of an effect that the gated network unit has on predictions of the neural network without an adjustment by the gating unit;
wherein a gated output of the gated network unit comprises an intermediate output of the gated network unit multiplied by the gating value; and
instructions that, when executed by the one or more processors, cause the computing system to perform operations to train the neural network based on one or more training examples, wherein the operations comprise, for each of the one or more training examples:
determining a gradient of a loss function with respect to at least one of the one or more network parameters and one or more gating parameters; and
updating a respective value of at least one of the one or more network parameters and the one or more gating parameters based on the gradient of the loss function.
|