US 11,720,380 B1
System and method for updating augmented reality navigation instructions based on a detected error
Shailendra Singh, Maharashtra (IN)
Assigned to Bank of America Corporation, Charlotte, NC (US)
Filed by Bank of America Corporation, Charlotte, NC (US)
Filed on May 18, 2022, as Appl. No. 17/747,636.
Int. Cl. G06F 9/451 (2018.01); G06F 3/01 (2006.01); G06F 11/34 (2006.01); G06T 19/00 (2011.01)
CPC G06F 9/453 (2018.02) [G06F 3/013 (2013.01); G06F 11/3438 (2013.01); G06T 19/006 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A system for updating augmented reality navigation instructions based on errors, comprising:
a memory configured to store:
a web application comprising a plurality of elements that comprise at least one of a button, a text field, and a hyperlink;
a mapping table comprising a mapping between each of the plurality of elements mapped to a respective element attribute, such that:
a first element from among the plurality of elements is mapped to a first element attribute, wherein the first element attribute comprises at least one of a first function of the first element; and
a second element from among the plurality of elements is mapped to a second element attribute, wherein the second element attribute comprises a second function of the second element;
a processor, operably coupled with the memory, and configured to:
receive, from a first augmented reality device, a first image of the web application that shows the first element;
receive, from the first augmented reality device, eye tracking information that indicates eye movements of a first user as the first user is looking at different elements on the web application;
determine, based at least in part upon the eye tracking information, that the first user is looking at a first location coordinate on the first image of the web application;
determine that the first element is located at the first location coordinate;
determine that the first element is actuated;
receive, from the first augmented reality device, a second image of the web application that shows an error message, wherein the error message indicates that the web application has failed;
determine that the first element is faulty in response to the error message;
in response to determining that the first element is faulty:
record user events that led to failing the web application, wherein the user events comprise actuating the first element;
update an augmented reality message for the first element by indicating that the first element is faulty;
generate an augmented reality display in which the updated augmented reality message is presented as a virtual object;
anchor the updated augmented reality message to the first location coordinate on the augmented reality display; and
communicate the updated augmented reality message for the first element to a second augmented reality device in response to determining that a second user wearing the second augmented reality device is looking at the first element.