US 11,854,102 B2
Evaluating software license usage using reinforcement learning agents
Amihai Savir, Sansana (IL); Assaf Natanzon, Tel Aviv (IL); and Shiri Gaber, Beer Sheva (IL)
Assigned to EMC IP Holding Company LLC, Hopkinton, MA (US)
Filed by EMC IP Holding Company LLC, Hopkinton, MA (US)
Filed on May 2, 2019, as Appl. No. 16/401,610.
Prior Publication US 2020/0349662 A1, Nov. 5, 2020
Int. Cl. G06Q 10/00 (2023.01); G06Q 50/18 (2012.01); G06N 20/00 (2019.01); G06Q 10/0639 (2023.01); G06F 11/34 (2006.01); H04L 67/143 (2022.01); H04L 67/50 (2022.01)
CPC G06Q 50/184 (2013.01) [G06F 11/34 (2013.01); G06F 11/3409 (2013.01); G06F 11/3423 (2013.01); G06F 11/3452 (2013.01); G06F 11/3466 (2013.01); G06N 20/00 (2019.01); G06Q 10/06393 (2013.01); H04L 67/143 (2013.01); H04L 67/535 (2022.05)] 20 Claims
OG exemplary drawing
 
1. A method, comprising:
obtaining a plurality of key performance indicators indicating usage of a software product by a user of an organization;
determining, by at least one software-based agent using at least one processing device, a set of metrics for a predefined time window, the set of metrics comprising:
(i) one or more of a mean and a median of one or more of the obtained key performance indicators;
(ii) an amount of time that the software product was active on a display of the user; and
(iii) an amount of interactions by the user with a user interface in the predefined time window; and
applying, by the at least one software-based agent using the at least one processing device, a reinforcement learning process comprising:
evaluating, using at least one neural network, a plurality of possible login states of the software product based at least in part on the set of metrics for the predefined time window, wherein the evaluating comprises observing the plurality of possible login states, including a current state comprising a current login state of the software product, and obtaining an expected utility score for changing from the current login state of the software product to a different login state of the software product based at least in part on a comparison of the plurality of key performance indicators to a corresponding set of predefined thresholds, wherein the set of predefined thresholds are defined for a plurality of users of the organization;
determining whether to change from the current login state of the software product to the different login state of the software product based on the expected utility score;
triggering the change to the different login state based on a result of said determining;
prompting the user to provide feedback corresponding to the change;
in response to obtaining negative feedback, from the user, corresponding to the change, adjusting at least one of the predefined thresholds in the set of predefined thresholds based at least in part on the negative feedback; and
applying the adjusted at least one predefined threshold for the user for determining an adjusted expected utility score, wherein the adjusted at least one predefined threshold is further applied to one or more additional users of the plurality of users of the organization for determining respective expected utility scores for the one or more additional users;
wherein the method is performed by at least one processing device comprising a processor coupled to a memory.