US 12,066,920 B2
Automated software testing with reinforcement learning
Xiaoyan Liu, Bothell, WA (US); Steve K. Lim, Redmond, WA (US); Taylor Paul Spangler, Kirkland, WA (US); Kashyap Maheshkumar Patel, Bellevue, WA (US); Marc Mas Mezquita, Redmond, WA (US); Levent Ozgur, Seattle, WA (US); and Timothy James Chapman, Bellevue, WA (US)
Assigned to MICROSOFT TECHNOLOGY LICENSING, LLC, Redmond, WA (US)
Filed by MICROSOFT TECHNOLOGY LICENSING, LLC, Redmond, WA (US)
Filed on Jun. 30, 2022, as Appl. No. 17/854,640.
Claims priority of provisional application 63/341,791, filed on May 13, 2022.
Prior Publication US 2023/0367703 A1, Nov. 16, 2023
Int. Cl. G06F 11/36 (2006.01); G06N 3/08 (2023.01); G06N 3/10 (2006.01)
CPC G06F 11/3664 (2013.01) [G06F 11/368 (2013.01); G06F 11/3688 (2013.01); G06N 3/08 (2013.01); G06N 3/10 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A method of automated software testing comprising:
selecting, by a reinforcement-learning model, an action to be taken in a first user interface of a software instance being tested,
wherein the first user interface includes a first plurality of interactive elements;
causing the action to be performed, wherein the action includes interacting with an interface element in the first user interface and performance of the action causes the software instance to change states from the first user interface to a second user interface,
wherein the second user interface includes an interactive element different from the first plurality of interactive elements;
determining a reward associated with the action;
inputting an embedding of a natural language description of a state associated with the first user interface into the reinforcement-learning model;
generating an updated reinforcement-learning model by training the reinforcement learning model using the action, the embedding, and the reward; and
storing the updated reinforcement learning model.