US 12,147,875 B2
Systems and methods for evaluating a behavior of a machine-learning algorithm
Aaron R. Ladurini, Palm Bay, CA (US); Andrew W. Kom, Melbourne, FL (US); and Rebecca A. Howard, Melbourne, FL (US)
Assigned to NORTHROP GRUMMAN SYSTEMS CORPORATION, Falls Church, VA (US)
Filed by Aaron R. Ladurini, Palm Bay, CA (US); Andrew W. Kom, Melbourne, FL (US); and Rebecca A. Howard, Melbourne, FL (US)
Filed on Jul. 7, 2020, as Appl. No. 16/922,775.
Prior Publication US 2022/0012624 A1, Jan. 13, 2022
Int. Cl. G06N 5/045 (2023.01); G05D 1/00 (2006.01); G06N 5/01 (2023.01); G06N 20/00 (2019.01)
CPC G06N 20/00 (2019.01) [G05D 1/0088 (2013.01); G05D 1/0221 (2013.01); G05D 1/101 (2013.01); G06N 5/01 (2023.01); G06N 5/045 (2013.01)] 19 Claims
OG exemplary drawing
 
1. A computer implemented method for post-execution evaluation of a machine-learning (ML) algorithm, the method comprising:
receiving ML behavior data comprising a post-execution version of the ML algorithm by a processing device of a system, the ML algorithm having a plurality of behavioral states, the plurality of behavioral states corresponding to different decision-making states of the ML algorithm;
evaluating the plurality of behavioral states by comparing each of the plurality of behavioral states to a library of approved behavioral states for the ML algorithm to identify a given behavioral state from the plurality of behavioral states of the ML algorithm, the given behavioral state corresponding to a decision-making state of the ML algorithm that is not part of the library of approved behavioral states for the ML algorithm;
generating a graphical user interface (GUI) that provides interactive time objects on an interactive timeline, the interactive time objects being controllable based on user input to define a start and end time specifying a user selectable time window to control a behavior of the ML algorithm;
in response to user selections of a first interactive time object from the interactive time objects and a second interactive time object from the interactive time objects to specify the user selectable time window, displaying in the GUI a plurality of graphical objects comprising:
a behavior object within the user selectable time window characterizing the given behavioral state of the ML algorithm;
a plurality of condition objects associated with a performance state of the system at an instance of time within the user selectable time window; and
a plurality of task objects that identify tasks executed by the processing device of the system within the user selectable time window, wherein each condition object is associated with at least one task object;
wherein the behavior object and a first condition object of the plurality of condition objects are visually indicated by the GUI to be associated with a first task object of the plurality of task objects and a second condition object of the plurality of condition objects is visually indicated by the GUI to be associated with a second task object of the plurality of task objects, wherein the first condition object is different from the second condition object, and the first task object is different from the second task object;
generating behavior evaluation data based on user input to the GUI, the behavior evaluation data indicating one of that the given behavioral state of the ML algorithm is an approved behavioral state for the ML algorithm, that the given behavioral state is not an approved behavioral state of the ML algorithm and an update to the learning process for the given behavioral state of the ML algorithm; and
updating the ML algorithm to alter a learning process of the ML algorithm by modifying the given behavioral state of the ML algorithm to provide an updated ML based on the behavior evaluation data.