US 11,704,230 B2
Test script generation based on event data and video frames
Er-Xin Shang, Shanghai (CN); Bin Zhou, Shanghai (CN); and Chao Lin Jiang, Shanghai (CN)
Assigned to Micro Focus LLC, Santa Clara, CA (US)
Appl. No. 17/420,337
Filed by MICRO FOCUS LLC, Santa Clara, CA (US); Er-Xin Shang, Shanghai (CN); Bin Zhou, Shanghai (CN); and Chao Lin Jiang, Shanghai (CN)
PCT Filed Jan. 11, 2019, PCT No. PCT/CN2019/071377
§ 371(c)(1), (2) Date Jul. 1, 2021,
PCT Pub. No. WO2020/143030, PCT Pub. Date Jul. 16, 2020.
Prior Publication US 2022/0083458 A1, Mar. 17, 2022
Int. Cl. G06F 9/44 (2018.01); G06F 11/36 (2006.01); G06V 10/25 (2022.01); G06V 20/40 (2022.01); G06V 40/20 (2022.01)
CPC G06F 11/3684 (2013.01) [G06F 11/3672 (2013.01); G06V 10/25 (2022.01); G06V 20/44 (2022.01); G06V 40/28 (2022.01)] 20 Claims
OG exemplary drawing
 
1. A non-transitory machine-readable storage medium comprising instructions that upon execution cause a system to:
process event data and video frames produced by a program during execution of the program, the event data representing user actions with respect to a graphical user interface (GUI) of the program, and wherein the user actions are received as an input to an input device;
identify an area of the GUI that corresponds to a respective user action of the user actions, wherein identifying the area of the GUI uses a first video frame before an event corresponding to the respective user action, and a second video frame after the event corresponding to the respective user action;
identify, based on the identified area of the GUI, a test object representing a user interface (UI) element;
generate a test script for testing the program, the test script including the test object;
analyze video frames that occurred between events corresponding to user actions to identify regions of the GUI that change; and
mark the identified regions of the GUI as not related to user actions.