US 11,893,485 B2
Batch normalization layers
Sergey Ioffe, Mountain View, CA (US); and Corinna Cortes, New York, NY (US)
Assigned to Google LLC, Mountain View, CA (US)
Filed by Google LLC, Mountain View, CA (US)
Filed on Jan. 22, 2021, as Appl. No. 17/156,453.
Application 17/156,453 is a continuation of application No. 16/572,454, filed on Sep. 16, 2019, granted, now 10,902,319.
Application 16/572,454 is a continuation of application No. 15/009,647, filed on Jan. 28, 2016, granted, now 10,417,562, issued on Sep. 17, 2019.
Claims priority of provisional application 62/108,984, filed on Jan. 28, 2015.
Prior Publication US 2021/0224653 A1, Jul. 22, 2021
This patent is subject to a terminal disclaimer.
Int. Cl. G06N 3/08 (2023.01); G06N 3/04 (2023.01); G06N 3/084 (2023.01); G06F 18/10 (2023.01); G06F 18/2415 (2023.01); G06V 10/70 (2022.01); G06V 10/82 (2022.01)
CPC G06N 3/08 (2013.01) [G06F 18/10 (2023.01); G06F 18/2415 (2023.01); G06N 3/04 (2013.01); G06N 3/084 (2013.01); G06V 10/70 (2022.01); G06V 10/82 (2022.01); G06T 2207/20081 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A method performed by one or more computers, the method comprising:
receiving a neural network input; and
generating a neural network output for the neural network input by processing the neural network input using a neural network system, wherein:
the neural network system includes a sequence of neural network layers, wherein the sequence includes a third layer between a first neural network layer and a second neural network layer in the sequence of neural network layers of the neural network system,
the neural network system generates the neural network output from the neural network input by processing the neural network input through each of the layers in the sequence, and
the third layer is configured to:
receive a first layer output generated for the neural network input by the first neural network layer, the first layer output comprising a plurality of components;
normalize each component of the first layer output to generate a normalized output comprising a plurality of normalized components;
generate a transformed version of the normalized output by transforming each normalized component of the normalized output in accordance with a set of normalization parameters for the third layer; and
provide the transformed version of the normalized output as input to the second neural network layer.