US 11,892,809 B2
Controlling operation of an electrical grid using reinforcement learning and multi-particle modeling
Wolf Kohn, Seattle, WA (US); Chad Edward Steelberg, Newport Beach, CA (US); Andrew Elvin Badgett, Boulder, CO (US); and Leslie Gene Engelbrecht, Niwot, CO (US)
Assigned to Veritone, Inc., Irvine, CA (US)
Filed by Veritone Alpha, Inc., Costa Mesa, CA (US)
Filed on Jul. 26, 2021, as Appl. No. 17/385,764.
Prior Publication US 2023/0041412 A1, Feb. 9, 2023
Int. Cl. G05B 13/02 (2006.01); G06N 20/00 (2019.01); G06N 3/00 (2023.01); G05B 13/04 (2006.01); H02J 3/38 (2006.01); G06N 3/006 (2023.01)
CPC G05B 13/0265 (2013.01) [G05B 13/04 (2013.01); G06N 3/006 (2013.01); G06N 20/00 (2019.01); H02J 3/381 (2013.01); H02J 2203/20 (2020.01)] 30 Claims
OG exemplary drawing
 
1. A computer-implemented method comprising:
generating, by one or more computing systems, a model of a physical system whose operations include providing electrical power for an electrical grid with multiple nodes each having at least one producer of electrical power and at least source of electrical power load, wherein the model describes operational characteristics of the physical system and includes multiple rules that each has one or more conditions to evaluate and that specify restrictions involving a plurality of elements of the physical system and includes state information from sensors for the physical system and includes information about multiple control actions available to affect the providing of the electrical power and includes an indicated goal to maximize for the operations; and
controlling, by the one or more computing systems and using the generated model, the operations of the physical system for each of multiple successive current time periods, including:
receiving, by the one or more computing systems, information that includes total projected electrical power production available from the physical system for the current time period and includes total projected electrical power load for the physical system for the current time period, wherein the total projected electrical power production includes respective projected electrical power production for each of the multiple nodes, wherein the total projected electrical power load includes respective projected electrical power load for each of the multiple nodes, and wherein the total projected electrical power load exceeds the total projected electrical power production by a difference having a determined amount;
generating, by the one or more computing systems, multiple particles that each represents a different set of state information for the physical system;
for each of a plurality of iterations during at least some of the current time period,
propagating, by the one or more computing systems, and separately for each of the multiple particles, the respective state information for the particle to attempt to determine projected future state information for the particle that reflects a decrease in the determined amount of the difference for the current time period in light of the indicated goal; and
combining, by the one or more computing systems and using reinforcement learning, information from at least some of the multiple particles having projected future state information;
determining, by the one or more computing systems and after the plurality of iterations, one or more of the multiple particles whose projected future state information provides a solution for the physical system to satisfy the total projected electrical power load for the current time period using the total projected electrical power production for the current time period in light of the indicated goal, including determining that the difference between the total projected electrical power load and the total projected electrical power production is eliminated for the determined one or more particles;
implementing, by the one or more computing systems, and using at least one of the determined one or more particles, at least one control action in the physical system to satisfy the total projected electrical power load for the current time period using the total projected electrical power production for the current time period; and
updating, by the one or more computing systems, the generated model to reflect the implementing of the at least one control action using the at least determined particle.