US 12,265,913 B1
Machine learning architectures with sub-quadratic iterator modules
Cole Louis Kissane, Palo Alto, CA (US); and Niveditha Subramanyam Iyer, Mountain View, CA (US)
Assigned to Crystal Computing Corp., Menlo Park, CA (US)
Filed by Crystal Computing Corp., Menlo Par, CA (US)
Filed on Jul. 23, 2024, as Appl. No. 18/781,938.
Claims priority of provisional application 63/674,721, filed on Jul. 23, 2024.
Int. Cl. G06N 3/084 (2023.01); G06N 3/04 (2023.01)
CPC G06N 3/084 (2013.01) [G06N 3/04 (2013.01)] 21 Claims
OG exemplary drawing
 
1. A method of using an artificial neural network (ANN), the method comprising:
receiving, at an input layer of the ANN, one or more inputs representative of an input sequence;
passing the one or more inputs sequentially through a plurality of layers of the ANN,
wherein each layer of the plurality of layers comprises weights used for processing the one or more inputs to generate one or more outputs,
wherein at least one of the plurality of layers is an iterator layer that feeds back intermediate outputs of the iterator layer as inputs to itself one or more times before passing an iterated output to a subsequent layer of the plurality of layers, and
wherein the iterator layer is configured to execute each iteration at a computational complexity that is sub-quadratic in relation to a length of the input sequence; and
outputting the one or more outputs generated by a final layer of the plurality of layers,
wherein passing the one or more inputs sequentially through the plurality of layers of the ANN comprises:
determining whether each intermediate output of the iterator layer satisfies a condition, and
in response to determining that the condition is satisfied by a specific intermediate output, passing the specific intermediate output to the subsequent layer of the plurality of layers as the iterated output, and
wherein determining whether each intermediate output of the iterator layer satisfies the condition comprises determining whether a similarity metric computed based on the specific intermediate output and one or more previously generated intermediate outputs is within a threshold.