US 12,423,981 B2
Storage and processing of intermediate features in neural networks
Shaomin Xiong, Fremont, CA (US); Toshiki Hirano, San Jose, CA (US); Ramy Ayad, San Jose, CA (US); and Damien Kah, San Jose, CA (US)
Assigned to Western Digital Technologies, Inc., San Jose, CA (US)
Filed by Western Digital Technologies, Inc., San Jose, CA (US)
Filed on Jun. 30, 2021, as Appl. No. 17/363,490.
Prior Publication US 2023/0004742 A1, Jan. 5, 2023
Int. Cl. G06V 20/52 (2022.01); G06N 3/045 (2023.01); G06T 7/246 (2017.01); G06T 7/292 (2017.01); G06V 20/40 (2022.01)
CPC G06V 20/52 (2022.01) [G06N 3/045 (2023.01); G06T 7/246 (2017.01); G06T 7/292 (2017.01); G06V 20/41 (2022.01); G06T 2207/20084 (2013.01); G06T 2207/30232 (2013.01); G06V 2201/10 (2022.01)] 28 Claims
OG exemplary drawing
 
1. A device comprising:
at least one processor configured to, alone or in combination, process video data wherein the processing includes:
inputting the video data into a primary artificial neural network, wherein:
the primary artificial neural network comprises at least an input layer, a plurality of intermediate layers, and an output layer;
the input layer is configured to receive the video data;
the plurality of intermediate layers comprises a first group of intermediate layers and a second group of intermediate layers;
the first group of intermediate layers and the second group of intermediate layers include at least one shared intermediate layer;
a first intermediate layer of the plurality of intermediate layers is configured to pass first intermediate layer data from the first group of intermediate layers to another intermediate layer of the plurality of intermediate layers;
a second intermediate layer of the plurality of intermediate layers is configured to pass second intermediate layer data from the second group of intermediate layers to another intermediate layer of the plurality of intermediate layers or the output layer; and
a primary output of the output layer is provided to a metadata generator;
providing the first intermediate layer data as an input of a first secondary artificial neural network that does not include any layer from the primary artificial neural network, wherein the first secondary artificial neural network is configured to provide a first secondary output to the metadata generator;
providing the second intermediate layer data as an input of a second secondary artificial neural network that does not include any layer from the primary artificial neural network, wherein the second secondary artificial neural network is configured to provide a second secondary output to the metadata generator; and
generating, using the metadata generator, a plurality of metadata associated with the video data based on the primary output, the first secondary output, and the second secondary output.