US 11,710,042 B2
Shaping a neural network architecture utilizing learnable sampling layers
Shikun Liu, London (GB); Zhe Lin, Fremont, CA (US); Yilin Wang, San Jose, CA (US); Jianming Zhang, Campbell, CA (US); and Federico Perazzi, San Jose, CA (US)
Assigned to Adobe Inc., San Jose, CA (US)
Filed by Adobe Inc., San Jose, CA (US)
Filed on Feb. 5, 2020, as Appl. No. 16/782,793.
Prior Publication US 2021/0241111 A1, Aug. 5, 2021
This patent is subject to a terminal disclaimer.
Int. Cl. G06N 3/04 (2023.01); G06N 3/082 (2023.01)
CPC G06N 3/082 (2013.01) [G06N 3/04 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A non-transitory computer-readable medium storing instructions thereon that, when executed by at least one processor, cause the at least one processor to perform operations comprising:
initializing a neural network comprising a plurality of sampling layers and a plurality of network weights;
providing a neural network shaping mechanism for at least one sampling layer of the neural network, the neural network shaping mechanism comprising a first sampling branch that includes the at least one sampling layer, a second sampling branch that includes an additional sampling function, and a learnable scaling factor between a sampling rate of the at least one sampling layer and the additional sampling function based on input to the at least one sampling layer;
jointly learning the scaling factor and the plurality of network weights using neural network output generated based on a dataset by:
generating a first feature map comprising a first size using the sampling rate of the at least one sampling layer based on the input to the at least one sampling layer, wherein the input to the at least one sampling layer corresponds to the dataset;
generating a second feature map comprising a second size using the additional sampling function based on the input to the at least one sampling layer, wherein the second size is different from the first size; and
performing linear interpolation between the first feature map and the second feature map based on the scaling factor; and
shaping an architecture of the neural network by combining the first sampling branch and the second sampling branch of the neural network shaping mechanism to modify the sampling rate of the at least one sampling layer of the neural network based on the learned scaling factor.