US 12,346,795 B2
Apparatus and method for improved inspection and/or maintenance management
Mohammad Saifullah, University Park, PA (US); Konstantinos Papakonstantinou, University Park, PA (US); and Charalampos Andriotis, University Park, PA (US)
Assigned to The Penn State Research Foundation, University Park, PA (US)
Appl. No. 18/853,426
Filed by The Penn State Research Foundation, University Park, PA (US)
PCT Filed May 12, 2023, PCT No. PCT/US2023/021992
§ 371(c)(1), (2) Date Oct. 2, 2024,
PCT Pub. No. WO2023/235136, PCT Pub. Date Dec. 7, 2023.
Claims priority of provisional application 63/347,192, filed on May 31, 2022.
Prior Publication US 2025/0111207 A1, Apr. 3, 2025
Int. Cl. G06N 3/04 (2023.01); G06N 3/047 (2023.01); G06Q 10/20 (2023.01)
CPC G06N 3/047 (2023.01) [G06Q 10/20 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A communication apparatus comprising:
a computer device having a processor connected to a non-transitory computer readable medium, the computer device configured to:
(1) initialize actor neural network weights for a pre-selected number of actors for an episode;
(2) initialize critic neural network weights and Lagrange multipliers for the episode;
(3) determine current beliefs based on asset condition states, model parameters, and pre-defined deterministic constraint metrics;
(4) sample actions from actor neural network outputs or at random and sample an observation from a pre-defined observation probability model for determining a total cost, probabilistic constraints (“probabilistic constraint metrics”) and beliefs for a next step of the episode;
(5) repeat (1)-(4) until the episode ends;
(6) sample a batch of experiences based on a current belief, deterministic constraint metrics, actions, costs, and probabilistic constraint metrics, and the beliefs, deterministic constraint metrics, actions, costs, and probabilistic constraint metrics for the next step, at the end of the episode;
(7) calculate an advantage function;
(8) update the actor neural network weights, the critic neural network weights and the Lagrange multipliers for a next episode; and
(9) repeat (3)-(8) until a convergence condition is detected.