	US 12,190,080 B2
	Automatic generation of user experience mockups using adaptive gaze tracking
Mouleswara Reddy Chintakunta, Allagadda (IN); Omar Odibat, Cedar Park, TX (US); and Pritpal S. Arora, Bangalore (IN)
Assigned to Kyndryl, Inc., New York, NY (US)
Filed by Kyndryl, Inc., New York, NY (US)
Filed on Aug. 10, 2022, as Appl. No. 17/884,616.
Prior Publication US 2024/0053964 A1, Feb. 15, 2024
Int. Cl. G06F 3/01 (2006.01); G06F 8/20 (2018.01); G06F 40/279 (2020.01)

CPC G06F 8/20 (2013.01) [G06F 3/013 (2013.01); G06F 40/279 (2020.01)]

20 Claims

1. A computer-implemented method for automated user experience mockup generation, comprising:

obtaining a user experience theme description;

obtaining a new feature image set;

generating a plurality of user experience designs corresponding to the new feature image set by performing an iterative process using a generative adversarial network (GAN) based on the theme description and the new feature image set;

sequentially displaying, using an electronic display device, the plurality of user experience designs to a user and tracking an eye gaze of the user with respect to a display of the plurality of user experience designs and determining where on the electronic display the user is viewing the user experience designs of the plurality of user experience designs;

for the user experience designs of the plurality of user experience designs, identifying a corresponding set of user interface elements based on the eye gaze, wherein the corresponding set of user interface elements identified is a set of elements that are gazed at for a duration exceeding a predetermined threshold duration;

comparing the identified corresponding set of user interface elements to a designer created user experience design; and

returning the user interface elements of the identified corresponding set of user interface elements as feedback input to the GAN.