US 12,265,911 B2
Neural network layers with a controlled degree of spatial invariance
Gamaleldin Elsayed, Milpitas, CA (US); Prajit Ramachandran, Santa Clara, CA (US); Jon Shlens, San Francisco, CA (US); and Simon Kornblith, Toronto (CA)
Assigned to GOOGLE LLC, Mountain View, CA (US)
Filed by Google LLC, Mountain View, CA (US)
Filed on Dec. 14, 2020, as Appl. No. 17/121,161.
Claims priority of provisional application 62/970,826, filed on Feb. 6, 2020.
Prior Publication US 2021/0248472 A1, Aug. 12, 2021
Int. Cl. G06N 3/08 (2023.01); G06N 3/04 (2023.01); G06N 3/082 (2023.01)
CPC G06N 3/082 (2013.01) [G06N 3/04 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A computing system for relaxing spatial invariance in a neural network, comprising:
one or more processors; and
one or more non-transitory computer-readable media that collectively store:
a neural network comprising one or more layers with relaxed spatial invariance, wherein each of the one or more layers is configured to:
receive a respective layer input;
convolve a plurality of different kernels against the respective layer input to generate a plurality of intermediate outputs, each of the plurality of intermediate outputs having a plurality of portions;
relax spatial invariance of the plurality of different kernels by applying, for each of the plurality of intermediate outputs, a respective plurality of weights respectively associated with the plurality of portions to generate a respective weighted output; and
generate a respective layer output based on the weighted outputs; and
instructions that, when executed by the one or more processors, cause the computing system to perform operations, the operations including using the neural network to process a network input to generate a network output.