US 12,291,357 B1
Deep replacement: reinforcement learning-based constellation management and autonomous replacement
Joseph Ryan Kopacz, Lone Tree, CO (US)
Assigned to Lockheed Martin Corporation, Bethesda, MD (US)
Filed by LOCKHEED MARTIN CORPORATION, Bethesda, MD (US)
Filed on Feb. 25, 2021, as Appl. No. 17/185,806.
Claims priority of provisional application 62/981,458, filed on Feb. 25, 2020.
Int. Cl. G06N 3/04 (2023.01); B64G 3/00 (2006.01); B64G 7/00 (2006.01); B64G 99/00 (2009.01); G06N 3/08 (2023.01); G06Q 10/20 (2023.01); H04W 84/06 (2009.01)
CPC B64G 99/00 (2022.08) [B64G 3/00 (2013.01); B64G 7/00 (2013.01); G06N 3/04 (2013.01); G06N 3/08 (2013.01); G06Q 10/20 (2013.01); H04W 84/06 (2013.01)] 20 Claims
OG exemplary drawing
 
1. An artificial intelligence (AI) system, the AI system comprising:
an agent configured to learn a policy and provide an action, the agent being a neural network; and
a processor configured to process information associated with the action and provide a state and a reward to the agent,
wherein:
the state is based on a plurality of state variables,
the agent is further configured to update the policy based on multiple updates of the state variables to achieve a highest reward; and
in each of the multiple updates of the state variables, the processor is further configured to increment a simulation environment by a single time step and apply a respective one of a sequence of actions including a build piece parts action, a build components action, a build subsystems action, a build spacecraft action, and a launch spacecraft action, wherein the build components action requires a first resource, corresponding to a first one of the state variables, from the build piece parts action, the build subsystems action requires a second resource, corresponding to a second one of the state variables, from the build components action, the build spacecraft action requires a third resource, corresponding to a third one of the state variables, from the build subsystems action, and the launch spacecraft action requires a fourth resource, corresponding to a fourth one of the state variables, from the build spacecraft action.