US 12,217,172 B2
Adaptive off-ramp training and inference for early exits in a deep neural network
Siva Kalyana Pavan Kumar Mallapragada Naga Surya, Redmond, WA (US); Joseph John Pfeiffer, III, Bothell, WA (US); and Davis Leland Gilton, Chicago, IL (US)
Assigned to Microsoft Technology Licensing, LLC, Redmond, WA (US)
Filed by Microsoft Technology Licensing, LLC, Redmond, WA (US)
Filed on Jun. 15, 2021, as Appl. No. 17/348,299.
Prior Publication US 2022/0398451 A1, Dec. 15, 2022
Int. Cl. G06N 3/08 (2023.01); G06F 16/953 (2019.01); G06F 18/214 (2023.01); G06F 18/25 (2023.01)
CPC G06N 3/08 (2013.01) [G06F 16/953 (2019.01); G06F 18/214 (2023.01); G06F 18/25 (2023.01)] 20 Claims
OG exemplary drawing
 
1. A computer-implemented method of training a deep neural network with an adaptive off-ramp for exiting the deep neural network, the method comprising:
receiving training data;
predicting, based on the training data, a label using a prediction layer in a sequence of prediction layers;
determining a combination of a weighted entropy value associated with the label and a confidence value of the label;
training, based on the combination and a predetermined threshold, the prediction layer and an off-ramp associated with the prediction layer to exit from the deep neural network;
removing, based at least on the trained prediction layer, a portion of the training data to create updated training data;
resampling the updated training data; and
training, based on resampled training data, a subsequent production layer of the sequence of prediction layers.