US 12,346,824 B2
Embedding constrained and unconstrained optimization programs as neural network layers
Tarek Aziz Lahlou, McLean, VA (US); Christopher Larson, Washington, DC (US); and Oluwatobi Olabiyi, Arlington, VA (US)
Assigned to Capital One Services, LLC, McLean, VA (US)
Filed by Capital One Services, LLC, McLean, VA (US)
Filed on Apr. 22, 2024, as Appl. No. 18/642,271.
Application 18/642,271 is a continuation of application No. 18/079,588, filed on Dec. 12, 2022, granted, now 12,001,961.
Application 18/079,588 is a continuation of application No. 16/791,945, filed on Feb. 14, 2020, granted, now 11,544,572, issued on Jan. 3, 2023.
Claims priority of provisional application 62/806,341, filed on Feb. 15, 2019.
Prior Publication US 2024/0346332 A1, Oct. 17, 2024
This patent is subject to a terminal disclaimer.
Int. Cl. G06F 17/00 (2019.01); G06F 17/11 (2006.01); G06F 17/16 (2006.01); G06N 3/044 (2023.01); G06N 3/10 (2006.01)
CPC G06N 3/10 (2013.01) [G06F 17/11 (2013.01); G06F 17/16 (2013.01); G06N 3/044 (2023.01)] 20 Claims
OG exemplary drawing
 
1. A computer-implemented method for embedding a convex optimization program as an optimization layer in a neural network architecture comprising a plurality of layers, the method comprising:
determining a set of parameters associated with the convex optimization program comprising associated primal decision variables and dual decision variables, wherein a coefficient matrix corresponding to one or more constraints of the convex optimization program imposes linear feasibility constraints on the decision variables;
generating, based on the coefficient matrix, a linear component of the optimization layer;
generating, based on a second transformation to a set of intermediary functions, a non-linear component of the optimization layer, wherein the second transformation is based on an intermediate value of the optimization layer;
iteratively determining, based on the linear component and non-linear component of the optimization layer and input values of the optimization layer and intermediate values of the optimization layer, fixed point values for the decision variables;
providing, by the optimization layer and based on the determined fixed point values for the decision variables, a first output to a next layer of the neural network architecture;
providing, by the optimization layer, a second output, wherein the second output is based on the first output;
determining, based on the second output, an error;
backpropagating the determined error through the plurality of layers; and
generating, by the neural network architecture and based on a trained set of parameters associated with the convex optimization program, one or more predictions.