US 11,989,786 B1
System and method to enable augmented intelligence guided help
Ruthie D. Lyle, Durham, NC (US); Sean Carl Mitchem, San Antonio, TX (US); Ravi Durairaj, San Antonio, TX (US); Mark Anthony Lopez, Helotes, TX (US); and Nolan Serrao, Plano, TX (US)
Assigned to UNITED SERVICES AUTOMOBILE ASSOCIATION (USAA), San Antonio, TX (US)
Filed by UIPCO, LLC, San Antonio, TX (US)
Filed on Jun. 24, 2021, as Appl. No. 17/356,592.
Claims priority of provisional application 63/046,268, filed on Jun. 30, 2020.
Int. Cl. G06Q 40/00 (2023.01); G06N 3/08 (2023.01); G06N 5/01 (2023.01); G06Q 30/016 (2023.01); G06Q 40/08 (2012.01); G06T 19/00 (2011.01); G10L 15/22 (2006.01)
CPC G06Q 40/08 (2013.01) [G06N 3/08 (2013.01); G06N 5/01 (2023.01); G06Q 30/016 (2013.01); G06T 19/006 (2013.01); G10L 15/22 (2013.01)] 13 Claims
OG exemplary drawing
 
1. A method of using an augmented reality enabled device, comprising:
storing first guided instructions for assisting a user with a predetermined process and storing second guided instructions for assisting the user with the predetermined process, wherein the second guided instructions are more detailed than the first guided instructions;
establishing electronic communication with the augmented reality enabled device when the augmented reality enable device is being operated by the user;
receiving information from the augmented reality enabled device indicating that the user has initiated the predetermined process;
monitoring input received from the augmented reality enabled device as the user performs the predetermined process, the input being selected from real time video and real time audio data generated by the augmented reality enabled device;
determining, using an experience determination module, a user level of experience for performing the predetermined process based on the input received from the augmented reality enabled device by analyzing information about the user's speech patterns or body language contained in at least one of the real time video data and the real time audio data;
passing the monitored input and the user level of experience into an assistance classification module and outputting an assistance class, wherein the assistance class has a first classification value and a second classification value, wherein the first classification value is associated with less assistance than the second classification value;
selecting the first guided instructions when the assistance class has the first classification value;
selecting the second guided instructions when the assistance class has the second classification value; and
automatically assisting the user in completing the predetermined process using the selected first guided instructions or the selected second guided instructions, by displaying on the augmented reality enabled device a virtual prompt at each step of the predetermined process.