US 12,293,290 B2
Sparse local connected artificial neural network architectures involving hybrid local/nonlocal structure
Benjamin Forrest Dribus, Hattiesburg, MS (US)
Filed by Benjamin Forrest Dribus, Hattiesburg, MS (US)
Filed on Jul. 14, 2020, as Appl. No. 16/928,219.
Claims priority of provisional application 62/890,615, filed on Aug. 23, 2019.
Prior Publication US 2021/0073642 A1, Mar. 11, 2021
Int. Cl. G06N 3/082 (2023.01); G06N 3/04 (2023.01); G06N 3/063 (2023.01)
CPC G06N 3/082 (2013.01) [G06N 3/04 (2013.01); G06N 3/063 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A method of constructing a geometry-induced sparse hybrid-local-nonlocal highly-connected artificial neural network architecture, the method comprising:
at least one processing unit, a computer readable memory and a computer non-transitory readable storage medium associated with a computing device;
choosing a geometry defined in terms of a manifold;
choosing a direction of data flow in the geometry;
choosing a node set as a finite subset of the geometry, where nodes specify the locations of artificial neurons in the geometry-induced sparse hybrid-local-nonlocal highly-connected artificial neural network architecture;
partitioning the node set into layers with respect to the geometry and the direction of data flow, wherein each node belongs to a unique layer of the layers, and wherein the layers are ordered, wherein a first layer of the layers is called an input layer and a last layer of the layers is called an output layer;
choosing local edges between each node in each non-input layer of the layers and nearby nodes in one or more preceding layers of the layers with respect to the geometry and the direction of data flow, wherein a degree of locality of the local edges is restricted by limiting connections to the one or more preceding layers to be among the √N closest nodes as defined by the geometry, where N is the number of nodes in the one or more preceding layers, and wherein the local edges specify physical locations of local synaptic connections between pairs of artificial neurons in the geometry-induced sparse hybrid-local-nonlocal highly-connected artificial neural network architecture;
choosing sparse nonlocal edges between each node in the each non-input layer and nodes in one or more preceding layers with respect to the geometry and the direction of data flow, wherein the nodes in the one or more preceding layer connected to at this step are not among the nodes connected via the local edges chosen in the previous step, and wherein the nonlocal edges specify physical locations of nonlocal synaptic connections between pairs of artificial neurons in the geometry-induced sparse hybrid-local-nonlocal highly-connected artificial neural network architecture;
modifying a predetermined percentage of the local synaptic connections and the nonlocal synaptic connections to achieve a specific target connectivity or to accommodate boundary effects;
implementing the geometry-induced sparse hybrid-local-nonlocal highly-connected artificial neural network architecture concretely complete by assigning edge weights to each synaptic connection and activation functions to each artificial neuron; and
training the geometry-induced sparse hybrid-local-nonlocal highly-connected artificial neural network architecture using at least one physical dataset, wherein the training is accomplished by means of a gradient descent algorithm and wherein the trained network geometry-induced sparse hybrid-local-nonlocal highly-connected artificial neural network architecture performs operations that solve specified problems and/or carry out specified tasks.