US 12,260,235 B2
Visual assist chatbot for improved accessibility of content by a visually impaired user
Marion Bonnet, Miami, FL (US); Ankurjyoti Bordoloi, Guwahati (IN); and Karan Paresh Trivedi, Mumbai (IN)
Assigned to MASTERCARD INTERNATIONAL INC., Purchase, NY (US)
Filed by MASTERCARD INTERNATIONAL INCORPORATED, Purchase, NY (US)
Filed on May 23, 2023, as Appl. No. 18/322,581.
Prior Publication US 2024/0394078 A1, Nov. 28, 2024
Int. Cl. G06F 9/451 (2018.01); G06F 3/16 (2006.01); G06F 40/106 (2020.01); H04L 51/02 (2022.01)
CPC G06F 9/453 (2018.02) [G06F 3/167 (2013.01); G06F 40/106 (2020.01); H04L 51/02 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A system for providing assistance to a visually impaired user navigating a website, the system comprising:
a processor; and
a computer-readable medium storing instructions that are operative upon execution by the processor to:
generate user interaction data describing user interaction with a webpage in real time as a user navigates the webpage via a user interface (UI) device, the user interaction data being generated without applying a visual impairment test or evaluation, the user interaction data comprising a number of navigation errors of the user with the webpage occurring within a period of time, the number of navigation errors being a count of user misses during attempted clicking of components of the webpage;
based on generating the user interaction data, predict a presence of a visual impairment of the user, by a trained machine learning (ML) model using the user interaction data and one or more visual impairment criteria, the visual impairment criteria comprising a minimum threshold number of navigation errors;
generate a predicted visual assist (VA) level using the number of navigation errors, wherein the number of navigation errors exceeding the minimum threshold number of navigation errors indicates the presence of the visual impairment, and wherein the trained ML model predicts an absence of the visual impairment in response to the number of navigation errors falling below the minimum threshold number of navigation errors;
modify a layout of the webpage in response to the VA level exceeding a threshold VA level, modifying the layout comprising increasing a font size of text on the webpage, increasing a size of text boxes on the webpage, and removing content of the webpage to decrease a difficulty of webpage navigation;
cause an audible activation prompt to be output to the user, the audible activation prompt requesting whether the user wants to activate a VA chatbot; and
activate the VA chatbot in response to the trained ML model predicting the presence of the visual impairment and in response to an affirmative response to the audible activation prompt, the VA chatbot configured to perform text-to-speech and to assist the user in performing a task during navigation of the webpage.