US 12,462,350 B2
Circuit for executing stateful neural network
Andrew C. Mihal, San Jose, CA (US); Steven L. Teig, Menlo Park, CA (US); and Eric A. Sather, Palo Alto, CA (US)
Assigned to Amazon Technologies, Inc., Seattle, WA (US)
Filed by Amazon Technologies, Inc., Seattle, WA (US)
Filed on Jan. 5, 2024, as Appl. No. 18/405,440.
Application 18/405,440 is a continuation of application No. 16/584,891, filed on Sep. 26, 2019, granted, now 11,868,871.
Claims priority of provisional application 62/901,740, filed on Sep. 17, 2019.
Claims priority of provisional application 62/888,413, filed on Aug. 16, 2019.
Prior Publication US 2024/0153044 A1, May 9, 2024
Int. Cl. G06T 5/70 (2024.01); G06N 3/04 (2023.01); G06N 3/044 (2023.01); G06N 3/049 (2023.01); G06N 3/063 (2023.01); G06N 3/084 (2023.01); G06N 5/046 (2023.01); G06N 20/10 (2019.01)
CPC G06T 5/70 (2024.01) [G06N 3/04 (2013.01); G06N 3/044 (2023.01); G06N 3/049 (2013.01); G06N 3/063 (2013.01); G06N 3/084 (2013.01); G06N 5/046 (2013.01); G06N 20/10 (2019.01); G06T 2207/10016 (2013.01); G06T 2207/20081 (2013.01); G06T 2207/20084 (2013.01)] 20 Claims
OG exemplary drawing
 
20. A method for using a neural network inference circuit to execute a neural network comprising a plurality of layers, the method comprising:
executing, using a set of computation circuits, the plurality of layers of the neural network, wherein executing the plurality of layers of the neural network comprises:
generating, for a first input by a first set of layers of the neural network, state data;
storing the state data in a set of non-transitory machine-readable memories for retrieval and use by a second set of layers of the neural network while executing the neural network for each of a plurality of inputs after the first input,
generating pointer data that indicates that the state data was generated for the first input, and
modifying, during subsequent executions of the neural network, the pointer data to indicate a number of executions that have occurred since the state data was generated.